If you believe Polymarket odds are wrong in a systematic way then you are free to make a lot of money out of it. Unlike the stock market, where "Markets can remain irrational longer than you can remain solvent", any such irrationality will become irrelevant at settlement time.
The argument isn’t about whether prediction markets can stay well-calibrated.
The argument is that prediction markets incentivize insider trading and backroom power brokership. The “potential energy” behind surprise upsets is most profitably exploited when the outcome sharply differs from the public calibrated consensus, so these two incentives - calibration vs potential for exploitation - are in fundamental tension. I think this tension undermines the whole purpose of prediction markets IMO.