Readit News logoReadit News
qqtt · a year ago
As someone who was super interested in the 538-style of election coverage in 2008, I've kind of fallen "out of love" so to speak with election models and forecasting in general. I'm not really convinced about what it adds to the conversation around elections. We can all look at various polls and get an assessment of who is generally ahead. Weighted polling aggregators and forecasting models just collect all these polls and spit out some data. It's easy to hand wave and think some new information is being revealed, but ultimately it is just a "garbage in garbage out" situation - you are entering polls as input, some hand waving is going on, and you get some forecast as a result.

I think part of my cynicism comes in the wake of the 2016 election, in which the forecast rightfully counted some scenarios in which either candidate could win, upon which conclusion of the model was basically "the result fits in with the forecast, because either candidate could have won according to the model" - in which case I personally concluded, if no matter what the result, we can always just say "the candidate who won could always have won given the forecast" - what are we really adding to the conversation here? We can simply look at polls and understand who is generally ahead, and not be any better or worse off.

alisonatwork · a year ago
538 in its final form was not about predicting election outcomes, it was about the business and science of polling, and contrasting factual data with the way people perceived related issues. It was an interesting outlet that helped illuminate the data side of political science, and at its best also provided some insight into the disconnect between how the general public thinks about a topic versus what is actually happening.

Disney killing 538 is broadly a loss for political journalism in the US, imo, because most other American media is more interested in sensationalism and hyping imaginary culture war issues, i.e. exacerbating exactly the disconnect with reality that 538 was trying to combat with its more evidence-based reporting. From my perspective the only place still doing this kind of work outside of niche, single-topic outlets like SCOTUSblog is ProPublica, and even they don't tend to be as politics-centric as 538 was. So I definitely will miss the site, and the pod. I don't have the stomach for most other American media.

wasabi991011 · a year ago
Respectfully, I disagree on both points.

> We can all look at various polls and get an assessment of who is generally ahead.

I probably could, but there's a lot of polls to look through and I don't really want to spend the time. Much rather have someone else do it for me.

> if no matter what the result, we can always just say "the candidate who won could always have won given the forecast" - what are we really adding to the conversation here?

Isn't this hypothesis testing? If you have a weighted coin and a guess as to which side is heavier and by how much, you're going to need multiple flips to see if you are right. And it doesn't even really make sense to talk about how right/wrong you are about a single flip, only on the aggregate.

It's possible someone has already compiled FiveThirtyEight's results to get some aggregate accuracy, I haven't checked. If they have and he's wrong on average and that's what you are referencing, my apologies.

JeremyNT · a year ago
The trick here is that if the election is close enough that you'd actually want/need multiple pollsters aggregated, the aggregators will indicate high uncertainty. If it's enough of a blowout for the aggregators to indicate low uncertainty, then the individual polls are going to be showing a large gap.

An aggregator saying "foo has a 65% chance of winning" may seem like it's providing more information than a single historically reliable poll (say Reuters/Ipsos) stating "foo is up by 2 points but there's a 3 point margin of error" - but isn't it just an illusion? High quality pollsters very seldom deviate very much.

And even if you grant that the aggregator is closer to being "right" than any single pollster, is that difference actually meaningful enough to impact any real world behaviors? Would you do anything differently with a 50% chance of victory versus a 70% chance?

I've honestly come to think of them as entertainment, with no real value.

tqi · a year ago
I know the folks at 538 meant well, but I think the ultimate impact of their work was to accelerate the politics as entertainment, team sports-ification of elections, to our nations detriment.
sdwr · a year ago
I kinda get your point - statistics suck the air out of the room. If regular people are talking about swing state poll margins of error instead of the actual issues, something's gone wrong.

538 democrasized the numbers that were the domain of political whizzes. I don't know if that's a good thing.

maigret · a year ago
Well swing states are the issues, aren’t they? I’m happy to live in a place where it isn’t a small fraction of individuals deciding the fate of the country.
wslack · a year ago
I think aggregators are useful to tell us what actually moved voters in a campaign - the swings are visible even if baseline error is not.
bnralt · a year ago
Like the Rationalist's "Bayesian priors," the election models were a remnant of the "big data" hype from a decade and a half ago. This article is a decent overview for anyone who forgot about it[1]. Like with many hype cycles, there was something actually important underneath the surface (useful statistical modeling), but then people with a poor understanding of the limitations ran wild thinking it could do things far beyond its capabilities (in this case, the degree to which one could use statistics to predict the future).

Industry gave up on the more extreme claims fairly quickly because it wasn't able to produce. But it lingered on in other places where there was less direct feedback or it was telling people what they wanted it to hear.

To add to this, it became obvious that many of the leaders in this "field" were people who believed they had an expertise that was far beyond their actual capabilities. Nate Silver ended up accusing much of the polling industry of fraud recently, because he wasn't able to do basic statistical math[2].

[1] https://slate.com/technology/2017/10/what-happened-to-big-da... [2] https://x.com/JustinWolfers/status/1853302476406993315

Gimpei · a year ago
I disagree. Before 538 people were still offering lots of election predictions and it was much much worse, because it was based entirely on hunches and vibes. Silvers rates the pollsters and provides confidence intervals far better than a simple average of polls does. I’d much rather read his forecasts than any number of bloviating opeds.
mgfist · a year ago
But that's because elections these days are incredibly close. It's like being upset that the best statistical answer to "who will win a coin toss" is "well it's 50/50".
genewitch · a year ago
Are you sure about that?

89% of counties turned red. Looking at 1.5% total difference is missing the forest for the trees. Trump got all swing states.

To quote: "this is absolutely a mandate."

https://youtu.be/zG3n2IeaTPA

edit: facts cannot be insolent. The youtube link is a guest on the view named Stephen something, from this week, saying the words that i typed into the comment box. Not clicking it is doing yourself a disservice; as it is "source cited."

Don't get mad at me for relaying this information.

nozzlegear · a year ago
I largely agree with your points. Election modelers and forecasters really don't add much to the conversation after 2016, despite their attempts and even purported success at correcting their models and mistakes. The only election forecasting model that I take seriously these days is my own vibes based forecasting.

I have enjoyed the meta-drama around forecasting and modeling that pops up every election season though. It's hard to beat "[Nate Silver] doesn't have the faintest idea how to turn the keys," or "I ran 80,000 simulations."

¹ https://knowyourmeme.com/memes/he-doesnt-have-the-faintest-i...

² https://knowyourmeme.com/memes/i-ran-80000-simulations

Deleted Comment

JumpCrisscross · a year ago
Would note that Silver owned the election model 538 ran on. When he left, he took it with him. The recent election forecasts 538 put out were not Nate Silver’s. (In my opinion, his were more accurate. More importantly, his commentary was more informative on the model’s shortcomings and insights.)
finnthehuman · a year ago
>if no matter what the result, we can always just say "the candidate who won could always have won given the forecast" - what are we really adding to the conversation here?

When the "upset" candidate wins, it will have been statistically likely. Yes. What they're adding are error bars in the collective consciousness.

Ironically enough, the hype pop of 538 was actually driven by people misinterpreting the stats and using them to feel vindicated in their support of Obama. The comedown was finding out that 25% is still 1-in-4 odds.

538 should be killing the horse race coverage by doing the most sober version of it. But horse race is big business. Therefore: boo 538, booooooo, they're harshing the vibe of my favorite reality show.

smt88 · a year ago
The purpose of statistical coverage of elections is to tell campaigns what their strengths and weaknesses are and to entertain the rest of us.

Since we don't run campaigns, the only value is entertainment. But that's not a negligible value for a lot of people.

ks2048 · a year ago
I agree it's mainly entertainment, but it also affects a lot of people "involved" in campaigns who aren't "running" them - mainly people donating their money or time.
CalRobert · a year ago
I find it useful in the general way that having corroborating sources for election data is useful. If the election is reasonably close to models, exit polls, etc. then it is more likely to have been run fairly.
mikepurvis · a year ago
As a resident of Kitchener ON, I'm privileged to live in one of just two Canadian ridings represented by the Green Party— I worked on Mike Morrice's first campaign in 2019, and I remember being frustrated trying to talk to voters on doorsteps and having to explain over and over that the purported "polls" on 338 (Canadian knockoff 538) showing us at 3-4% support in the riding were based on projecting forward previous results and adjusting slightly for national polling trends.

Sure enough, we ended up coming in at a whopping 26% in 2019, and in 2021 won the seat with 33%. Certainly the win was in part because the incumbent was embroiled in a last minute scandal, but I truly believe the polling aggregators have a huge suppressing effect on breakout candidates— without that effect it's possible we could have taken the earlier election too.

Now that seat is "safely" Green, it's been twice affirmed with huge wins for a separate Green candidate at the provincial level:

https://en.wikipedia.org/wiki/Kitchener_Centre_(federal_elec...

https://en.wikipedia.org/wiki/Kitchener_Centre_(provincial_e...

I expect this year's federal election will deliver another 40-50% result for Morrice, as he's very popular locally, but there's 338 again showing a big upswing for the Liberals in Kitchener Centre, when almost certainly there is no such thing, it's all just hallucinated from national polling:

https://338canada.com/35048e.htm

Uehreka · a year ago
I find the modeling super useful, many conventional media outlets still don’t properly communicate probabilities to their audience. For instance, I vividly remember the following exchange between Nate and some news anchor in during one of the 2016 conventions:

Anchor: So Nate, you say Trump has a 25% chance of winning, can you tell us exactly what that means?

Nate: Sure. So imagine I flipped two quarters, and they both came up heads. In that scenario, Trump wins.

Anchor: (shocked) Wait but that’s… that’s a thing that actually happens! You’re saying Trump has an actual chance of winning?

Nate: Well I’d rather be Hillary than Trump right now, but yes, people shouldn’t be that surprised if Trump wins, his chances aren’t insignificant.

I remember people in October still saying that Nate had to be wrong, that there was just no way Trump could win. There was even a growing market for what I would now call “cope forecasts” that “unskewed” the results to show that, really, Hillary had a 99% chance of winning, just like you knew she did (all of these people looked extremely foolish after the election was over).

I also feel like good models provide valuable pushback against media narratives that try to characterize the “closeness” of a race. In 2016, people wanted to hear that Trump had no chance of winning, but Nate/538 correctly pushed back that the race was actually pretty close and both candidates had a good chance. And he did the opposite in 2012: Pundits wanted to cast Obama and Romney as being neck-and-neck (which makes for a more exciting story) and Nate had the stats to push back that actually the race was not very close at all. If Romney had won in 2012, Nate would’ve had to eat crow, but Romney didn’t win.

Nate and 538 also do senate races, which are super valuable if you’re figuring out which candidates to donate money to. Often there are Democratic candidates in totally doomed races against Republicans I really don’t like, and the data helps me look at those situations and go “yeah I hate Lindsey Graham, but his challenger has no chance, I’m going to donate to the milquetoast Nevada senator whose race is on a knife’s edge instead”.

I could probably just look up polls, but the way Nate/538 process the polls into results with error bars and probabilities makes it a lot easier to reason about.

bnralt · a year ago
I see this argument a lot, but it's contradictory. You're simultaneously arguing that people don't understand statistics because they're treating a 25% chance as no chance to win, but then you're doing the same by saying that the other predictions, in the 15% to 2% range[1] are "cope forecasts" that people who followed them "looked extremely foolish" (the only major 99% forecast was PEC, but Wang said that's because the model broke down and the actual forecast was around 5% [1]).

25%, 15%, 5%, even 2% chances happen with a decent amount of frequency. I don't understand how people can say that people don't understand probability because they think a 25% chance won't happen, but then turn around and treat a 15% chance the very same way.

[1] https://www.nytimes.com/interactive/2016/upshot/presidential... [2] https://web.archive.org/web/20171120175008/https://election....

MostlyStable · a year ago
I agree with basically this whole comment, but the sad/ironic/whatever part of it is though even though Nate Silver was doing the thing you are describing ("actually, Trump has a real shot of winning"), afterwards he got constantly dinged (either by people incorrectly conflating him with the 99% models, or by people who just didn't actual listen to him) for "getting it wrong" because he "only" gave trump a 30ish % chance of winning.

I still come across it every once in a while and it probably has the highest ratio of level of infuriating-ness to low-value of the stakes of the opinion of just about any political opinion I can think of.

fitsumbelay · a year ago
agreed on all points
tbrownaw · a year ago
But how else will you know the correct emotional tone to use when complaining / gloating about the upcoming election with friends? Or know how seriously to take various prepper activities like stocking up on Twinkies in case the wrong team wins?
red_hare · a year ago
I started listening to the 538 Politics podcast a lifetime ago when they did The Gerrymandering Project. The deep intertwining of history, intentions, and statistics made the narrative compelling. I learned so much about how our democracy worked that I would never have known otherwise.

So, I kept listening and kept learning. It was sometimes difficult, not because of their storytelling skills, but because the news was hard to consume. But the cold numbers helped me manage my emotions with clarity and not disengage.

There's something wonderful about journalism backed by data. The line between news and editorial has long been blurred beyond visibility. 538 was a rare example of a place where smart people could express strong opinions but always had to show the work behind their conclusions.

I'll miss 538. They were an amazing team.

kemayo · a year ago
Yeah, they've been on my podcast subscription list for at least 5 years now, and I'll miss having them around.

I had growing conflicted feelings about the site's overall impact on media, sadly. It felt like, although it was good that they existed as a dedicated organization, they contributed to (or were a symptom of) the overall media landscape's slide into politics coverage as mostly coverage of the horse-race. Sometimes I want to hear what the Scottish teens think a news story means... but more often I want something deeply reported about policy.

alisonatwork · a year ago
I will miss them too. I saw that Galen is already considering starting his own politics pod, but I fear that by immediately jumping into the Substack black hole it will just end up spiraling into the usual engagement-driven slide to the right: https://www.gdpolitics.com/p/my-thoughts-on-the-end-of-fivet...

Imo part of what made 538 work post-blogosphere heyday was exactly that it had backing from legacy media and the funding to continue sharing information with the public without a paywall. As soon as sites go behind a paywall they become a personification of the "media elite" stereotype, where only rich people have the privilege of being informed. But how otherwise to fund not just a cheerful host but a team of data scientists, editors etc in this day and age? Seems like the only interested billionaires do it with strings attached.

moffers · a year ago
I really enjoyed 538 in its heyday, and am glad to see Nate carry on with some of the work. I know he can be a polarizing in some circles, but keeping the data angle visible helps smooth some of the rougher edges of following politics sometimes.
mmahemoff · a year ago
I learned more from the reaction to Nate and 538’s forecasts than 538 itself. It helped me appreciate how journalists misunderstand and distort basic probability. If a model predicts A, B, and C as having 34%, 33%, and 33% likelihood respectively, the typical report is “538 predicts ‘A’ will win!” and they got it totally wrong when B or C is the victor. Interpretations of 538 were further fuelled by whatever political bias a pundit was coming from.

In a world where Kevin Rose can reboot Digg, Nate has every chance of acquiring and reviving 538. Good luck to Nate.

rconti · a year ago
I really enjoy Nate's podcast with Maria Konnikova - I read her book The Biggest Bluff a few years back and really enjoyed it, and them podcasting together is great. (Despite my feelings about never wanting to hear Malcolm Gladwell's voice ever again, as he's omnipresent in Pushkin podcast network ads, which seem to be the worst of the entire podcasting ecosystem).

I never really followed the "Nate Silver" controversy after 2016, but it basically seems to boil down to a bunch of liberals being mad because they felt lied to for no apparent reason.

nozzlegear · a year ago
> I never really followed the "Nate Silver" controversy after 2016, but it basically seems to boil down to a bunch of liberals being mad because they felt lied to for no apparent reason.

FWIW I don't think he's controversial because of the 2016 polling miss, most people who follow 538 understand what 538 does and that it wasn't his "fault." He's controversial because he posts scalding hot takes on Twitter and then goes to the mat to defend them. He also has a penchant for getting into Twitter beefs with other big names in his industry.

tekla · a year ago
This is an absolute loss. 538 is amazing because it forced people to confront the cold hard data about polls surrounding politics and if you didn't like it, figure out what they did wrong or deal with it.

I'll never forget being called racist because I showed someone a 538 poll that said the presidential election was at best a toss-up to someone who was sure Kamala Harris would sweep the swing states.

kelnos · a year ago
538 hasn't really been 538 since Silver left, as Disney lost the rights to Silver's models (he maintained ownership of them) at that point. I'm honestly not too sad over the ABC News version of 538 folding; Silver's own continuing work will fill the niche that the old-school 538 flled.
vintermann · a year ago
> almost never interfered in our editorial process

Almost. This means that they were perfectly willing to, but rarely had to.

If it were otherwise, as Chomsky said, "You wouldn't be sitting where you're sitting".

BrenBarn · a year ago
FiveThirtyEight was interesting in its time, but in the past few years I felt it ironically became exactly what it was initially trying to oppose: a site full of opinion-based punditry. All their "538 chats" were basically the same as talking heads on TV. Okay, the 538 talking heads maybe paid more attention to data, but the good part of 538 was the intent to cut to the chase, dispense with all the puffery that ordinary news sources shove at you, and just let the data speak for itself. In recent years they moved away from that and became less distinguishable from the opinion section of a mainstream news source.
laweijfmvo · a year ago
it must be a wild experience to have a megacorp buy out your brain child, burn it to ashes and throw it away, and chase after the next shiny thing to slap ads on...
prawn · a year ago
Nate: "I wasn’t technically laid off — but my existing contract was set to expire in June 2023, and there was profound mutual disinterest in negotiating a new deal."
kelnos · a year ago
The interesting thing about his experience, though, is that he had the clout (and foresight?) to craft the deal so that he was merely licensing his models to Disney, and ever since he left, Disney has not been able to use them (well, for some of the sports-related ones, Disney was permitted to keep a copy, but not benefit from further work on it).

So Silver still seems to have the rights to the important stuff that made FiveThirtyEight what it was. He can (and has been) more or less building a new version of it, using all his old IP as a base.

Hamuko · a year ago
Seems like just the natural samsara of the news world. Small outlets are acquired by larger ones, eventually dismantled, staff moving onto build new ones and starting the cycle anew.
TheRealDunkirk · a year ago
<Microsoft clears its throat and glances around nervously...>
fsckboy · a year ago
>538-style of election coverage in 2008... I'm not really convinced about what it adds to the conversation around elections

I'm not really convinced that conversations around elections add anything. <-- period.

for me, 538-style coverage is great substitute for having to listen to myriad opinions that are generally facially flawed, and neither a representative sample.