Experts-shmexperts. Where’s the data?

I wrote yesterday about the kind of expertise that makes its case by relating what other experts and regular folk reckon. Once I started thinking about it, it struck me how similar this kind of expertise is to the kind relied on by the US government and military in Afghanistan. This is a topic that I thought a lot about while working for the Army, and that friends still working in that area did and still do write about. It’s also gotten a lot of attention recently since Rajiv Chandrasekaran’s book about the war in Afghanistan came out. Rajiv criticizes the Obama administration for bringing its “C team” to Afghanistan.

He means C as in third-rate. But the critique would be just as valid if it stood for C as in Commentary team. Because that’s what most of the experts the Obama administration relied on specialized in. In 2011 the National published a perceptive critique of the kind of expertise deployed by US foreign policy. In one part of that piece the author contrasted the experts who built their expertise on their ability to “merge a personal narrative implying site-specific knowledge, avowedly ethnographic in nature, with a deep engagement with the political and analytical clusters of the American and British military” with “greater claims, and greater efforts, towards satellite cameras and listening devices; drones which can hover for days; databases which can track all good Taliban and all bad Taliban.” And then asked the question: “Yet who can decipher this data?”

It was meant as a rhetorical question. But I think it should be a literal one. In the days of big data, US military and national security officials are still incredibly dependent on anecdotes and the masters of anecdote. You can see a list of the experts General McChrystal brought to Afghanistan here. Josh Foust wrote more about them here. You’ll struggle to find much systematically empirical work done by any of those experts, and almost none of the kind of inferential and statistical analyses that would be needed to deal with any data comprised of more than a couple tens of data points.

For all the data the military says it collects, the experts it typically relies on are ones that don’t appear to be capable of doing anything with it. That’s probably why when presented with the results of a systematic analysis, a former secretary of defense turns first of all to what he’s “heard firsthand from countless troops on the battlefields of Iraq and Afghanistan.” Despite all the data supposedly available, personal experiences still regularly trumps analysis in the military. (And when data is referred to, it’s usually classified data so we can’t know about it).

But let’s pretend the military does have a massive amount of data available for those who are able to look at it.  The headlining-experts it picks to travel to Kabul aren’t ones who know how to do anything with it. Which is why I take Mark Liberman’s conclusions about quotations in the New Yorker to work pretty well for most analyses about the war in Afghanistan, including especially the ones where the authors tell you all about what Afghans have told them in Kandahar, or what tribal key leaders think, or what terrorists care about.

 “My conclusion? When you see a passage in quotation marks in a New Yorker article, you should not expect it to be a truthful representation of anything that the alleged speaker ever actually said. Rather, you should take it as the author’s expression of what they want you to believe that the speaker meant. In some cases, these unquotations are “poetically true”, that is, they give an insightful impression of the speaker’s feelings and attitudes, although the writer knows that the words are not original. In other cases, the unquotations are an honest misrepresentation, in the sense that they’re genuinely what the author understood (or at least, remembers) the speaker to have meant to say. And sometimes, the unquotations are completely fictional, in the sense that the author doesn’t care at all what the alleged speaker either said or meant, but puts words in their mouth in order to advance the narrative.”


One thought on “Experts-shmexperts. Where’s the data?

  1. the military is a profession that is much more comfortable with stories well-told than the systematic analysis of facts. Analysis is hard, thinking is hard, military leaders prefer trusting “good people” than applying reasoning and critical thinking. As evidence, look at how they have compartmentalized the Command & General Staff College student population: operations career field officers , “leaders” go to Leavenworth; thinkers, like ORSAs go to a functional school, thus preserving the barrier between thinkiing and action; The leaders learn to rely on cults of personality and trust, instead of on systematic, rigorous thinking

Comments are closed.