I don't understand why this is surprising. People didn't go to FiveThirtyEight to marvel the science behind it. The science was just supposed to give you what you came there for: the actual election results.
In the end, it turned out that predicting elections is still very hard, and that for all the fanfare, FiveThirtyEight performed only slightly better than what you could find in any other reputable newspaper, so it kinda lost its appeal.
> FiveThirtyEight performed only slightly better than what you could find in any other reputable newspaper
FiveThirtyEight gave Trump double the odds of the next highest reputable prediction, which was The New York Times Upshot (15%). Princeton Election Consortium gave Trump less than 1%.
That is not "only slightly better" to anyone who's statistically literate.
A FiveThirtyEight reader in 2016 was significantly better calibrated regarding Clinton’s chances than a reader of other reputable newspapers.
This embodies what 538 and its defenders miss about 538's appeal:
People didn't come to 538 for explanations on subtle points of statistical literacy (although those were provided). They came because, for whatever reason, they wanted to know who would win the election.
People not trained in statistics treated like the scoreboard at a football game- it's always better to be winning, but score is a near perfect predictor in the last minute.
Once 538 stopped delivering perfect predictions and started delivering "Actually the difference between 1% and 30% are way bigger than you think" lectures, the appeal disappeared. There are better places to learn math from.
Speak for yourself. That's not why I read FiveThirtyEight.
The purpose of FiveThirtyEight was never to be an oracle for the average person. It was always a deliberately wonky site for a wonky audience. They were very clear about that in the articles they published and topics they covered.
If we’re brutally honest the vast majority of 538 readers read it to be assured that the right outcome was outcoming.
They went to the wrong place then.
----
Nov. 1, 2016 — Election Update: Yes, Donald Trump Has A Path To Victory — https://archive.is/kwdab
> Tuesday was another pretty good day of polling for Donald Trump.
> Trump remains an underdog, but no longer really a longshot: His Electoral College chances are 29 percent in our polls-only model — his highest probability since Oct. 2 — and 30 percent in polls-plus.
> This isn’t a secure map for Clinton at all. In a race where the popular vote is roughly tied nationally, Colorado and New Hampshire are toss-ups, and Clinton’s chances are only 60 to 65 percent in Wisconsin, Michigan and Pennsylvania.
> If you want to debate a campaign’s geographic planning, Hillary Clinton spending time in Arizona is a much worse decision than Trump hanging out in Michigan or Wisconsin.
----
Sept. 16, 2016 — How Trump Could Win The White House While Losing The Popular Vote — https://archive.is/rxP5l
> Using a prototype of a demographic election calculator that FiveThirtyEight will be unveiling in the next few weeks, I decided to simulate a few election scenarios.
> The result? Clinton would carry the popular vote by 1.5 percentage points. However, Trump would win the Electoral College with 280 votes by holding all 24 Romney states and flipping Florida, Ohio, Pennsylvania, Iowa and Maine’s 2nd Congressional District from blue to red.
----
Jun 29, 2016 — Donald Trump Has A 20 Percent Chance Of Becoming President — https://archive.ph/ryIkP
> A 20 percent or 25 percent chance of Trump winning is an awfully long way from 2 percent, or 0.02 percent. It’s a real chance: about the same chance that the visiting team has when it trails by a run in the top of the eighth inning in a Major League Baseball game. If you’ve been following politics or sports over the past couple of years, I hope it’s been imprinted onto your brain that those purported long shots — sometimes much longer shots than Trump — sometimes come through.
----
FiveThirtyEight was probably the worst reputable source to read if you were looking for maximum assurances that Clinton would win.
> it turned out that predicting elections is still very hard
So maybe we shouldn't be doing it. The value of predicting an election in the large out in public seems kind of dubious, and it's more like gambling than actually being useful. A candidate only runs, and continues running, if they think they can win. All predictions like these do is confuse voters leading up to election day and while they are voting. It keep candidates from making strong cases for their platform, keeps the voters from listening to the candidates' platforms, and encourages team-based partisan politics.
Predictions are like exit polls, aren't they, in that they're both able to provide a check on official election results?
Think that's outweighed by the negatives?
538 was never about magically making polls more reliable, and only people that don't understand what polls are could think that (caveat: lots of people don't understand how polls work).
538 was about analyzing and communicating the information from those polls in an easily accessible form. If you came to the site for that, you weren't mad that they "predicted poorly something that was impossible to predict from the data sources they used" ... you were just mad at Trump for winning (despite polls suggesting otherwise).
Again, I don't think any of this matters. People were not coming there to have "information communicated to them". They were coming there for the satisfaction of knowing the results before everyone else. And FiveThirtyEight couldn't realistically deliver on that.
That makes as much sense as visiting ESPN and expecting them to tell you who will definitely win the Super Bowl next year. Anyone expecting that is going to be disappointed often no matter what.
I thought it went without saying but a good analyst can't predict the future in politics, sports, or anything else. What they can do is make good probabilistic estimates of what is likely to happen. 538 wasn't pretending to do anything more than that.
If people want magic predictions there are plenty of touts and scammers willing to offer them, they don't need to waste time with charts and numbers though.
> a good analyst can't predict the future in politics, sports, or anything else. What they can do is make good probabilistic estimates of what is likely to happen. 538 wasn't pretending to do anything more than that.
Well, sure, but how big is the market for that, really? Particularly for a binary outcome like an election, knowing who's going to win is fun, reading a pundit telling you who's going to win can be fun, but ultimately the man in the street is going to take whatever the pundit said and reduce it to candidate X or candidate Y, and you can only do so much better than replacement level at that.
I think anyone who actually operates that way is very misguided, but it’s a fair point. But either way, 538 was such a nice site for just looking at the data in a fresh way at the time, and it’s a shame that went away.
If people are expecting anyone to have a magic prediction algorithm for things like this… I mean there’s only so much one can say. It’s not realistic.
I'm very curious to see how polymarket fairs, compared to the news agencies. I suspect prediction markets will be the norm, going forward. Polls can't fully capture the element of anonymity that's required for an accurate poll of something controversial.
My experience was that prediction markets were lagging indicators and basically followed something akin to an aggregate opinion of polls.
This is especially viewable if you watch them during the 2020 election.
Polls became much less interesting as an Entertainment category once we all had experience with how unreliable they are.