Monday, October 17, 2016

Nate Silver is full of shit, Part V: Silver's methods can't ask the right questions

For all of my rants here about Nate Silver's methods, the real problem with what he is doing now is that his method cannot ask the right questions.

What Nate Silver is doing now is taking state-by-state polls and playing combinatorics games to try to count up the ways that Trump can get to 270 electoral votes, and ascribing probabilities to each to figure out an estimate of Trump's chances on that basis.  Right now, 538 puts Trump's chances at about 1 in 7, so in the multiverse model of quantum mechanics, one out of seven universes branching out from this moment have President Trumps in them.

The problem, as I keep saying, is that Clinton's lead in the national polls is too big to play that kind of combinatorics game, so the only real question to ask is this:  what would it take for Clinton to not have that lead?  Notice how I phrased that question.  I could have asked, what would it take for Clinton to lose her lead, but I didn't.

Factoring in the crappy LA Times poll, RealClearPolitics currently puts Clinton's lead at around 5.5 points, on average.  If her lead stays that big, the electoral college doesn't matter, and Silver's silly, little games are pointless.  So, what if her lead isn't that big now?  There are two ways that could be true.  Sampling bias, or measurement error.  Sampling bias means that everyone is getting a bad sample because we don't know how to reach the voters.  This is distinguished from randomly generated sampling error in that there is something systematically wrong with everything.  That could be true.  Maybe this year is just so screwy that every poll has a lousy sample.  If so, Silver's method doesn't help because it is all about the combinatorics, but we need to keep that in mind.

Second, measurement error.  Ever hear of the "Bradley effect?"  It is sort of a legend in political terms.  When Tom Bradley ran for Governor of California, there was supposedly a discrepancy between his polling numbers and his vote total.  Why?  Well, maybe it was because he was African-American, and nobody wanted to appear racist.  So, people told pollsters that they were going to vote for him, even if they had no intention of doing so.  The evidence was never that strong for the Bradley effect, but the legend lives on.  The concept, though, is that when people tell pollsters about their voting intentions, that isn't their real intention.  That would be measurement error.  Maybe Trump supporters don't want to admit that they are going to vote for the rapecist (TM-pending).  Can Silver's combinatorics game help with that?  Nope.  The problem with that is that Trump is actually polling OK, all things considered.  Clinton has a clear lead, but this isn't a blowout.

And then, the real trouble, which Silver can't address.  Intervening events.  What would it take for Clinton to lose a real lead?  What are the chances of a major terrorist attack, or something like it?  Silver can't tell you.  This is a really hard question.  Want a good book?  I recommend this one all the time. Philip Tetlock's Expert Political Judgment.  Certain cognitive styles are better at making that kind of forecast than others, but regardless, nothing in Silver's combinatorics game is even intended to capture it.  Tetlock draws on the old distinction between foxes and hedgehogs.  Foxes know a little about a lot of different stuff, and hedgehogs know a lot about one thing.  Foxes make better predictions.  The combinatorics game that Silver is playing?  Straight-up hedgehog thinking.

What is the real probability of Trump winning?  I don't know.  Silver's method, though, is garbage right now.  When one candidate is this far ahead in the national polls, the electoral college is not where you should focus.  It absolutely is where the candidates should focus, but not us.

In order for Trump to win, either the polls need to be systematically wrong, or something needs to change, and Silver's method isn't capturing that, and can't capture that.

No comments:

Post a Comment