Usually it's not the fabric but trapped dust that scratches the coating when you wipe.

Dust is harder than hard plastic?