I've seen plenty of evidence that goes against this. From what I have seen, short women will absolutely prefer a man who is taller than other men, regardless of their own height, so something does not add up. In fact, I've seen it often be the other way around; a man will sometimes prefer a woman who is not a midget compared to himself, purely for practical reasons (unless of course he falls in love).

I will go as far as to say that assortative mating is likely a side-effect of people ending up in the same or similar socioeconomic circles, but that is just me reaching. In reality I would guess that it's a bit of that, and being used to people from a certain "class".