A Word Against Nate Silver

Technically minded people have always been primed to become unwitting accessories to corruption, especially when speaking as authorities on subjects beyond their expertise. According to data scientist Cathy O’Neil, Liberals’ new favorite number cruncher is no exception.

Formerly a forecaster for Major League Baseball, Nate Silver won the hearts of mainstream lefties for predicting the outcome of the last presidential election with uncanny accuracy and leaving conservative pundits and politicians kicking and screaming on the floor. He has a new book out, called “The Signal and the Noise: Why Most Predictions Fail but Some Don’t.” As of this posting, it’s ranked No. 8 on the New York Times Best Seller list, which means it’s shaping the opinions of a large portion of the reading and voting public — most of whom probably know very little about Silver’s subject, predictive modeling.

Silver critic O’Neil is an independent data scientist and former hedge fund analyst associated with Columbia University in New York City. Of the two principal sins O’Neil says Silver commits in his book, the first is “naivete,” grounded in his expressed belief that bad modeling — not the incentivized pursuit of self-interest by members of the banking industry — caused the 2008 financial crisis.

“Silver has an unswerving assumption [with one exception], which he repeats several times, that the only goal of a modeler is to produce an accurate model,” O’Neil writes.

“This assumption generally holds in his experience: poker, baseball, and polling are all arenas in which one’s incentive is to be as accurate as possible. But he falls prey to some of the very mistakes he warns about in his book, namely over-confidence and over-generalization. He assumes that, since he’s an expert in those arenas, he can generalize to the field of finance, where he is not an expert.

“The logical result of this assumption,” O’Neil continues, “is his definition of failure as something where the underlying mathematical model is inaccurate. But that’s not how most people would define failure, and it is dangerously naive.”

Politics can never be ignored when attempting to describe and predict social behavior. But by O’Neil’s testimony, Silver doesn’t ground his conclusions about the cause of the fiscal crisis in an adequate understanding of social theory. Silver ignores the broader context, she notes, and instead “chooses to focus on individuals working in a tight competition and their motives and individual biases.”

O’Neil also says Silver “spends very little time on the question of how people act inside larger systems, where a given modeler might be more interested in keeping their job or getting a big bonus than in making their model as accurate as possible.

“In other words,” O’Neil concludes, “Silver crafts an argument which ignores politics. This is Silver’s blind spot: in the real world politics often trump accuracy, and accurate mathematical models don’t matter as much as he hopes they would.”

Silver’s second sin, according to O’Neil, is “to a lesser extent, authority-worship.” His apparent ignorance of political reality is so thorough, she asserts, that he was able to write the following in a particularly nasty entry on his blog, Five Thirty Eight, in the early days of the Obama administration’s response to the fiscal crisis:

“To suggest that Obama or Geithner are tools of Wall Street and are looking out for something other than the country’s best interest is freaking asinine.”

In the same post, he apparently went on to say that the post-crisis period was a time for “expert opinion,” with the experts, who comprised Obama’s economic team, being the very architects of deregulation that created the conditions that led to the bursting of the housing bubble and the crash. It was not, Silver wrote, the “time … for mass movements.”

O’Neil’s conclusion: “Nate Silver is a man who deeply believes in experts, even when the evidence is not good that they have aligned incentives with the public.”

If O’Neil is correct, then she’s done readers the valuable service of revealing Silver’s blind spots, which are dangerous because they mislead those who trust him. But Silver is a big time star with the respect and admiration of people around the country. As far as the public is concerned, O’Neil is a relative nobody, and even if Silver’s readers found her article in its obscure corner of the Internet, they would be faced with the familiar predicament of having to decide which expert to trust. The best way to combat this confusion, O’Neil says, is the vigilant review of Silver’s work by other modelers.

Until such criticism is available and widely comprehended, Silver’s celebrity status will continue to compel readers who find comfort in his reputation as a fact-based liberal hated by conservatives to treat his pronouncements as scripture.

— Posted by Alexander Reed Kelly.

mathbabe:

I’m not criticizing Silver for not understanding the financial system. Indeed one of the most crucial problems with the current system is its complexity, and as I’ve said before, most people inside finance don’t really understand it. But at the very least he should know that he is not an authority and should not act like one.

I’m also not accusing him of knowingly helping cover up the financial industry. But covering for the financial industry is an unfortunate side-effect of his naivete and presumed authority, and a very unwelcome source of noise at this moment when so much needs to be done.

Silver has gone to great lengths to make his message simple, and positive, and to make people feel smart and smug, especially Obama’s supporters.

He gets well-paid for his political consulting work and speaker appearances at hedge funds like D.E. Shaw and Jane Street, and, in order to maintain this income, it’s critical that he perfects a patina of modeling genius combined with an easily digested message for his financial and political clients.

Read more