Yeah, that claim was always ludicrous to me too. Wisdom-of-crowds isn’t an unbiased decision making strategy, it’s quite biased. Crowd-wisdom works best as a limiter on the bias of other decision making strategies—this is why democracies use representatives rather than direct votes for most decisions.
And polymarket isn’t even the wisdom of crowds lol. At its greatest possible adoption it’s still the wisdom of internet-connected (mostly) white men with time and money to spend on gambling.
If you believe Polymarket odds are wrong in a systematic way then you are free to make a lot of money out of it. Unlike the stock market, where "Markets can remain irrational longer than you can remain solvent", any such irrationality will become irrelevant at settlement time.
The argument isn’t about whether prediction markets can stay well-calibrated.
The argument is that prediction markets incentivize insider trading and backroom power brokership. The “potential energy” behind surprise upsets is most profitably exploited when the outcome sharply differs from the public calibrated consensus, so these two incentives - calibration vs potential for exploitation - are in fundamental tension. I think this tension undermines the whole purpose of prediction markets IMO.