I don’t understand the argument that small amounts don’t matter.

Diversification doesn’t work if you throw in low quality investments you wouldn’t consider on their own. It just lowers returns.