That makes some sense. Given the historic sometime scarcity of food and pressure of starvation, and the widespread availability of milk, I would think people would adapt to it.
I guess that lactose-intolerent people today would drink milk rather than starve - do they get zero nutrients from it? - and that evolution would select for those who could survive that way.