Thursday, October 20, 2016

Is Nate Silver still full of shit?

So, you may have noticed that I'm not a fan of Nate Silver.  I wrote a series of posts about it.

Given that series, I've been checking 538 periodically to see how he's doing.  Concern for his health, and all.  And today, I see this.  You may notice that it reads very similarly to how I've been talking about the election in the "Nate Silver is full of shit" series.  Trump has two chances:  massive, systematic polling error, or an intervening event.  So, um, what's the deal?

Here's the deal.  Silver's method of computing the chances of Trump's victory is still hopelessly flawed.  His polls only forecast currently puts Trump's chances at around 14%, which is where it was when I started the "Nate Silver is full of shit" series.  That still describes a quantum-mechanical multiverse in which one out of seven universes branching out from this moment include President Trumps.  And it does so because Silver's method plays combinatorical games with the electoral college, ignoring the near impossibility of an electoral college-popular vote split when a candidate is as far ahead as Clinton is now nationally.  Hence, there is actually a discrepancy between how Silver describes the race in the piece linked above and his normal method, if one recognizes the point I kept beating into the ground in the "Nate Silver is full of shit series" on the relationship between the popular vote and the electoral college when one candidate is as far ahead as Clinton is.

The problem, then, is Silver's failure to connect the substantive political observation to the math.  Combinatorics games of the kind Silver employs with the electoral college are not informative unless the popular vote is close enough to make an electoral college-popular vote split plausible.  Hence, the right question remains as follows:  what is the probability of Trump closing the gap nationally to a level at which 270 electoral votes are within reach for him?  Silver's method doesn't address that question.  What is the probability?  I don't know, but one of of seven?  I call bullshit.

And in his linked piece above?  So does Silver.  On himself.

2 comments:

  1. You're misreading Silver's model.
    He does not assign a 14% probability to an EC/vote split. Right now, it's a 5% chance of that for HRC, and a 0.5% for Trump. That's only about 1/3 of the 14%. (4.5%, since the 0.5% for Trump cuts against it)
    The rest of it is coming from shared variance among states in previous elections. Essentially, your state is doing a lot of fucking it up. Silver's model has a ton of "as goes X, so goes Y" in it, with anticipated correlation coefficients derived from previous election results and demographics. For example, a new poll just came out showing MI at +12 for Clinton (after Silver adjusts it, which is another thing, but it can be separated out for this discussion). This increased Clinton's odds of winning by 0.2%. Seems like a little, but we're at the flat part of the S-curve at this point. Once you realize that, you go "wait! How did one poll in MI move it so much? She's not losing MI; everyone knows that!" In Silver's model, MI +12 implies something like OH +7, so now THAT is baked into it. And earlier today, TN +11 Trump helped him, because as goes TN, so likely moves NC.

    He's using new state polls to derive imputations for other states in the absence of new data for those other states. It's DEFINITELY combinatorics, as you note. But, it's based on likelihoods of outcomes.

    If you want an example of where this is likely making errors, look at OH and NC. (All numbers are + Dem margin, negative is GOP; used election results for 2012 and RCP averages for 2016)

    2012: OH +1.9, NC -2.2
    2016: OH -0.6, NC +2.5

    These two states have moved in different directions. That's actually kinda weird. Even if Silver is using updated demographics and the Census has estimated those correctly, that's still kinda odd. He's got a PVI in there for each state, with a 75% weight for 2012 and 25% for 2008. The map I'm seeing has similarities, but AZ? NC? GA? I think they're all throwing the model off. For example, his model gives Clinton a 15% of winning SC. 4% for states like MS and TN.

    Honestly, though, I think it's the shared correlations between NC and FL that are giving him such high chances for Trump. That said, I color NC, FL, and OH for Trump and....it's still not 270. So, IN, AZ and IA go too.....and we're at 260. He needs MN to get to 270. And Silver's model gives MN a 13% chance of going to the dark side. Which really only happens if all the polls are off. But, if Trump is winning OH and IA, then he might also carry MN.

    But, yeah, how his model is coming up with 13% for MN? I think he's letting his demographics do too much work.

    ReplyDelete
    Replies
    1. 14-4.5=9.5 9% is the chance that Trump has over at PredictWise. He is off by the amount by which he assigns a chance to a popular vote-electoral vote split. That's my point.

      As for MN, the answer is more simple. Nobody is polling on it. Take a look at RCP. There is nothing in the last month, and the last thing is that weird tied poll from those fuckwits at Gravis, and Silver's model mechanically takes that as meaningful. That's not really his fault-- he is stuck with the data he has, in Rumsfeldspeak, but I go back to that 4.5% chance of a popular vote-electoral vote split saving Trump.

      Silver has no business playing the combinatorics game, period. When Clinton is ahead this much, the probability of a popular vote-electoral vote split is realistically zero. Therefore, the only real question is this: what is the probability of Clinton losing enough of her lead to make a Trump victory in the electoral college concievable? That isn't a combinatorics question. Wrong method for the wrong question.

      Delete