Why the best ideas sometimes don’t seem very good

I came across this the other day:

The Pentagon’s new “strategic guidance” document, issued last month in the wake of Mr. Obama’s pledge to cut $485 billion from the defense budget over the coming decade…states, “In the aftermath of the wars in Iraq and Afghanistan, the United States will emphasize nonmilitary means and military-to-military cooperation to address instability and reduce the demand for significant U.S. force commitments to stability operations.” It goes on to note that “U.S. forces will no longer be sized to conduct large-scale, prolonged stability operations.’”

With this paragraph military planners signaled an abrupt end to the post-9/11 era of intervention. Only a few years ago the wars in Iraq and Afghanistan — wars of occupation, nation-building and counterinsurgency — looked like the face of modern conflict. Now they don’t. Americans don’t believe in them and can’t afford them anymore.

When the Army hired me a few years ago, they hired me as an “irregular warfare” analyst. Irregular warfare is a wonderfully undefined topic (just try to make sense of this). Irregular warfare is pretty much every situation of potential or active conflict that doesn’t involve people in uniforms lining up in formation to shoot at each other. Everything from insurgencies to popular protests to online activism gets lumped into the irregular warfare category. But in my office – maybe because that’s just the pattern people had established – irregular warfare meant counterinsurgency. I spent a good two years or so turning out some pretty interesting (in my opinion) assessments on conflict in Afghanistan and Paksitan. I was well within the counterinsurgency lane of the irregular warfare road.

A few months before I switched careers, I had a few opportunities to work on places other than Afghanistan and Pakistan, and on issues outside of insurgency. I put these projects on my list of things to do and was excited about them because they were not only new and interesting, but because they would allow me to transition from being a counterinsurgency analyst to an actual irregular warfare analyst. Shortly thereafter, my supervisor called me in and told me he didn’t feel good about me doing the projects. He said we were all supposed to do counterinsurgency analysis and that what I had chosen was not counterinsurgency analysis. I argued that what I wanted to do was irregular warfare analysis – I even used the Joint Operating Concepts I cited above – but he said that wasn’t the issue: even though we were officially an irregular warfare team, we needed to stick to counterinsurgency.

I switched tactics, and offered up a perspective that I’d been developing over the previous months. I basically said the same thing as the above quote. Iraq and Afganistan were expensive, messy, unpopular wars, and we even had a Secretary of Defense say we shouldn’t fight those wars anymore. We technically had the mandate to work most if not all issues that fell under the umbrella of irregular warfare, and if we artificially limited ourselves to just one little aspect of that subject, then we ran the risk of making ourselves obsolete.

He didn’t buy it, and I tend to file paragraphs like the ones I quoted above more into the “gee-whiz” folder than the “vindication” folder, but my point is that my supervisor – and I think a lot of other people with whom I worked – unnecessarily and (in my view) unwisely limited the possibilities of his team because the product of those limited possibilities was easy to sell at the moment. Even though at the time I had that conversation people were already starting to say that we weren’t going to fight another war like Afghanistan, very few people were saying it out loud. It was easy to see that a lot of people – our customers – wanted to read stuff about Afghanistan and population-centric counterinsurgency.

I’ve been thinking a lot lately about people’s tendencies to embrace tactical victories in exchange for strategic losses. There are companies that provide good products and services – that are simply good at what they do – and there are companies that help shape the products and services that will be offered a year from now. People talked a lot about this stuff a few months ago when Steve Jobs died. One story reported that when asked what market research Apple did to develop the iPad, Jobs replied: “None. It’s not the consumers’ job to know what they want.” In other words, the companies that get lauded as leaders in their respective industries are those that do things because they think those things are good ideas, not because someone else thought it would be a good idea.

Call it whatever you want – thought leadership, innovation, good business sense – but that kind of behavior prioritizes strategy over tactics. It’s not a matter of what’s selling now or what people are asking for now, although of course you shouldn’t ignore those things. It’s a matter of figuring out what opportunities exist in whatever environment you’re operating in, and taking advantage of those opportunities because you can.

At the same time, I can (now that I have some distance) sympathize with my former supervisor. Doing what you’ve been doing all along is a pretty safe way to get minimally positive results – if it hadn’t given you at least a little bit of a positive outcome up to this point, you would have dropped it and moved on to something else. Doing something new doesn’t guarantee positive results: people might not like it, or you might get results that make you look bad. It doesn’t make sense to risk a tactical loss in pursuit of a worthy strategic goal as long as you focus on the satisfactory results that tactical risk-avoidance has brought you so far.

Problem is, risk-averse tactics make minimally positive results more likely, but they don’t guarantee them. In fact, given enough time, risk-averse tactics practically ensure bad outcomes because eventually someone will innovate in ways that you refused to, and what they have to offer then will make what you have to offer look really useless.  It’s not a question of whether the market will stay the same or change. The market’s going to change. The question is whether you’ll follow right behind the change, or far behind it.

I feel like I’m waxing all business-guru here, which I find distasteful but also funny since I’m totally not qualified to give business advice. It’s just that I’ve participated in and overheard a lot of conversations over the last few years that went along the lines of “I don’t want to identify problem areas. I want findings that show that this is the best thing ever and that we shouldn’t change a thing.” Or “that would be a great study…as long as the findings are positive.” The scientist in me screams when I hear that kind of stuff, but I can totally see how that kind of thinking makes sense – at a tactical level. It’s a real temptation to focus entirely on tactics because it’s concrete, and it’s what you already know. Strategy is everything you think, everything you might not know. It doesn’t make any sense from the perspective of day-to-day business operations to offer a product for which your customers never asked, to give people free access to a service just because you can, or to really empirically measure to what extent your operations actually create the outcomes that you say they do. All of those things run the risk of giving you – and everyone else around you – insights that aren’t really flattering.

Products, services, and business practices are a lot like living organisms, in that they operate in ways that were better suited to the environments in which they were developed than were the ways of their competitors. As environments change, practices that used to work well don’t work well any more, and products that used to look good don’t look as good in comparison to their better-adapted competitors. As a researcher, I tend to look at products, practices, etc. fro the population perspective – I’m ok if specific organisms have to die in order for the population as a whole to be better off. I’m not really concerned about whether a particular product is liked or disliked by specific people, or whether a particular practice is kept or discarded. I’m concerned that, taken as a whole, those products and services are succeeding more often than they are failing, and that they are changing in ways appropriate to the changing environments.

I’ve found that most people take the organism perspective. It’s not that they don’t care about the population’s viability –they’re just a whole lot more concerned about whether they live or die than they are about future generations’ ability to cope with changing selective pressures. Their project deserves to be funded because it’s their project. They’ve put a lot of work into it. I understand that perspective. I just happen to disagree with it. It’s a little cold-hearted maybe, but innovation isn’t just good public relations – it’s another word for adaptation. Every organization has to adapt, whether its members want to or not. So the choice is either to take a rigorous and brutally candid look at yourself and your environment and thereby maintain some degree of control over your adaptation, or you can look only at what you want to and thereby cede your control. Good analysis is a strategic priority – it doesn’t always seem like the best or most welcome idea in the present, but it’s the best way to ensure that your organization as a whole can survive into the future.


6 thoughts on “Why the best ideas sometimes don’t seem very good

  1. “I’ve been thinking a lot lately about people’s tendencies to embrace tactical victories in exchange for strategic losses.”

    In our particular case I was always kind of surprised at the strength of responses (usually rejections) to suggestions for innovations. Often the responses were openly based on concern for possible consequences (in terms of the disapproval of superiors or threats to the organization’s future). Once these concerns were expressed and used to justify the original strong negative response, my surprise turned to frustration. It didn’t make sense because when I looked around me there seemed to be little to no evidence that such consequences ever really existed. Teams and organizations rarely, if ever, were closed. People were moved around, but rarely (rarely) fired. Occasionally some superior somewhere issued a reprimand – but the reprimanded got over it quickly enough (with little more hurt than feelings). The superior got over it quickly enough too, after all their success was dependent on their subordinates’, so it was in their interest to protect and support their subordinates. This was hardly a crazy survival of the fittest world, so the strong concern for survival-threatening consequences was surprising, frustrating, and confusing.

    This paper and this paper by the psychologists Dan Gilbert and Tim Wilson helped me think about the kind of responses we received. They call them affective forecasting errors – inaccurate predictions regarding future states/outcomes (especially emotional ones). “Loss aversion occurs because people expect losses to have greater hedonic impact than gains of equal magnitude.” In other words, we’re too easily caught up in tempests in a teapot.

    I also thought this was persuasive: “People overestimated the hedonic impact of losses because they underestimated their tendency to rationalize losses and overestimated their tendency to dwell on losses.” This second claim is especially appealing because that was what I saw people doing all the time when it came to the negative consequences they HADN’T foreseen or necessarily caused – they rationalized and explained and justified them away in no time at all. We all do it. But somehow they didn’t realize they would probably do the same thing with consequences that might arise from risks they take that don’t turn out the way they planned.

    Perhaps I underestimate the potential for serious failure and undervalue legitimate concerns about such potential, but I think when it comes to planning for the future we too often compare the alternatives of ‘awesome success’ and ‘catastrophic failure’. A more realistic comparison would probably be between awesome success and the mundane process of muddling on. And even if it was just between ‘basic success’ and ‘mundane muddling on’ – it’d still make it a better comparison. It’s so easy to imagine futures of catastrophic failure; but such failure isn’t very realistic simply because of our persistent psychological need/ability to make ourselves feel good, and better, about ourselves. Perhaps managers and researchers, etc., would be more willing to take risks if they realized their most legitimate worry if the risks don’t turn out like they plan is just a more mundane future.

  2. I think there’s a problem with point of reference. It seems that people who want to innovate get into an organization and see a bunch of mediocrity – “mundane muddling on” – and want to try for something more impressive. They see achievement of awesome success as the possible good outcome and continuation of mediocrity as the possible bad outcome. Mediocrity isn’t normal for innovators.

    But mediocrity is very normal for other people. They don’t, I think, view their everyday activities as mediocre. I don’t know that they view their activities as one big string of awesome successes…maybe they don’t think about it that much…but it seems that if mediocre is considered to be just fine then even a moderate success can be viewed as an awesome success – acceptance of mediocrity might blows small successes all out of proportion. If that is true, then it would stand to reason that it would also blow small failures all out of proportion.

    On a seven-point scale of “very good,” “good”, “kinda good,” “neutral,” “kinda bad,” “bad,” and “very bad,” I think I tend to pin mediocrity somewhere between “kinda bad” and “bad.” I wonder if a lot of people pin mediocrity between “kinda good” and “good,” or at somewhere on the “good” side of “neutral.” So if I’m already at “kinda bad,” some higher-up getting mad at me will move me to “bad” for a little while and then back to “kinda bad.” But if I’m at “kinda good,” that reprimand still moves me to “bad” for a while, which means that reprimand moves me a lot farther in terms of emotional distance traveled.

  3. Pingback: Organizations may have leaders, but organizational thinking shouldn’t « House of Stones

  4. Pingback: Surveys, Assumptions, and the Need for Data Collection Alternatives « House of Stones

  5. Pingback: Feeling Ineffective … Needing the “Haqqani network” | House of Stones

  6. Pingback: Some helps for running and evaluating Bayesian regression models | House of Stones

Comments are closed.