Seems like in addition to whether they're more swingy than they should be, are they being presented in a way that is interpreted as more certain or authoritative than they can possibly claim?
That is actually an issue they've acknowledged and tried to address. People also tended to see percentages and think they reflected vote percents.
To address this, for the 2018 election cycle 538 started displaying their odds as numerical ratios (5 in 9, for example) to try to reduce the aura of certainty.
538 started displaying their odds as numerical ratios
Did that actually make things clearer to people? Speaking only for myself, I find ratios quite hard to works with and reason about. I certainly couldn't tell you instantly for example if 5/9 is more or less than 4/7 or how big the difference between the two are without first doing some mental arithmetic. Perhaps people in the US are more used to working with ratios since their measurement systems tend to be ratio based.
That's the point though. It's meant to require a bit of thinking to digest the information. They didn't set out to make the information clearer and easier to understand, they set out to make the false positive of reading a percentage and considering it final results less frequent.
538 provides pretty in-depth discussions of it forecasts and changes in it. Far more than you would expect - and I think because 538 is ultimately a statistics blog at heart, not a political blog.
The 2016 election was swingy and it was uncertain.
In 2008 and 2012, the political media was myopically focused on the twists and turns of the horse race and called it 50-50 dead even. Nate Silver's 538 modeled the election as basically stable with only small polling shifts, and the only change in the last month was a steady decline in the remaining time that McCain (or Romney) had to significantly shift the polls.
In 2016, the political media treated the election as if Hillary had a lock on it, and entreated us not to get swept up in the daily swings in the polls. Meanwhile, Nate Silver's model swung wildly with the polls, and Nate went on the media to emphasize the uncertainty of the election. Both candidates were disliked, and a large portion of the electorate was undecided right up until the last week of the election.
I remember the following exchange in one interview:
Interviewer: If you say that Donald Trump has a 33% chance to win, what's the chance that he actually wins?
Nate Silver: One in three. I'm predicting a one in three chance that Donald Trump wins in November.
As for the authoritativeness of their presentation, FiveThirtyEight has a really beautiful forecast that presents their 2018 prediction as a probability distribution:
The Real Clear Politics polling aggregator showed nearly a 10 point jump for Hillary Clinton after "grab em by the pussy" came out. That lead then eroded back to the mean over the course of the month.
That's what I mean by swingy. Nothing in 2008 or 2012 shifted the polls like that. The polls in those elections stayed within a 2 point band the whole way. That level of volatility was clear in the polls well in advance of the election.
(And, 15% of the electorate was effectively undecided with 2 weeks to go, which is much higher than it was in 2008 or 2012).
Such swings are also what Taleb and the author of this article are criticizing Nate Silver for.
>The 2016 election was swingy and it was uncertain.
I don't remember it that way, in fact I remember seeing a lot of very shocked faces in the Democratic camp when the results were beginning to take shape. I don't remember having seen similar confused reactions after any previous US presidential elections, not even after the Bush vs Gore one which was a lot closer in terms of electoral votes.
Back to the article, I think Nate Silver's failure only shows to the general public that electoral predictions are rubbish. Maybe "failure" is a strong word because he genuinely seems to be the best at what he's doing, it's just that the domain in which he's involved is turning out to be bogus. I'm sure that there was a crystal-ball viewer that was the best at what he/she was doing, it's just that crystal-ball viewing turned out to be bogus.
However, FiveThirtyEight gave Trump a 30% chance of winning, and in the days before the election was stressing the fact that a Trump win was very possible.
They didn't predict his win, but they did better than anyone else in the mainstream.
> They didn't predict his win, but they did better than anyone else in the mainstream.
Kind of proving my point, as it shows that Nate Silver was the best at crystal-balling the election result. Afaik 30% is still bellow the 50% (or 0.5/1) threshold generally needed to make a decision, as the article also mentions.
I agree. I think people's shock was a failure of imagination. They couldn't imagine a Trump win, so even though the polling data showed he could, they discounted it. That, I think, explains why Silver was criticized so heavily for saying what he did days before the election: they couldn't imagine a Trump win, therefore he must lose, which Silver must also think, therefore Silver is trolling.
