Good research usually needs walls. Really hard ones.

Tomorrow is my last day as a civilian employee of the U.S. Department of the Army. I’ve spent the last three years researching the effects of insurgent IED capabilities, patterns of consensus among detainee interrogation reports, return-on-investment for development projects, and other conflict-related subjects. I’ve worked on conflict issues ever since I was an undergraduate, and I’m sure I’ll still keep my fingers in a few of those pots. (I’m in the middle of a study of counter-narcotics operations in Mexico and getting ready to start a study of the relationships between disease and conflict in Africa – I’m sure I’ll be writing a lot about those projects on this blog). But on Tuesday I become the new Research Director in the marketing department at WorldStrides. I’m not sure, but I think my decision to move from the defense industry to the educational and travel industry raised even more eyebrows than my more general decision to move from the public to the private sector. I don’t think that change is as big as it might seem. I’ve tried to explain this in casual conversations with people over the last few weeks, but I think it’s a good idea to try to spell it out for myself here.

I study behavior. Specifically, I study the way people form shared patterns of behavior in response to their physical and social environments. At a more theoretical level, I want to know what kinds of ways an environment can vary. At the day-to-day research level, I want to know how variations in an environment can cause or coincide with variations in behavior. At the practical level, I want to know how variations in the environment can be changed or planned around to maximize the behaviors that people decide they want and minimize the behaviors they decide they don’t want.

In that sense, my move to a new industry doesn’t represent a change in research focus at all. I’m still dealing with behavior. I’m still dealing with environments. The stuff I study in the education industry will (I hope) involve fewer guns than the stuff I studied in the defense industry, but to me that’s just context. It’s still just behavioral adaptations to environment.

That begs the question of why I made the move. If it’s all the same stuff, why study it in the private sector instead of the public sector? No one has said it to my face, but as I’ve talked with people in the Defense Department I’ve gotten this vibe…sort of an unspoken question: “you’re leaving the study of national security so you can do marketing research?” Yeah, I am. Some of that has to do with frustrations I had working for the government. If you have access to the A-Space network, look up “iWAR” (the blog I helped maintain on that network) and you can see what I wrote about some of the things that bothered me. But a lot of my decision to move has less to do with specific things that made me unhappy about working for Defense, and more to do with a general need that working for the government couldn’t really fill.

Most people (I think) do research with the idea that it should inform people’s decisions. Research basically tells people that there’s a door through which they can walk to get to where they want to go. Problem is, research is just a conglomeration of researchers’ assumptions and a sample of data (often selected under less-than-ideal conditions). When people do research in organizations or situations where no one actually tries to walk through that door, there’s no way to really evaluate the validity of the research. If the research says there’s a door there, you ought to be able to walk through it and not hit a wall instead. That wall of reality makes for better decisions and better research because it provides a practical way to separate the valid assertions from the invalid ones. I was really surprised to learn that the U.S. government (at least the corner in which I worked) does not have many walls of reality. Analyses were planned, conducted, and disseminated without the findings making their way into any practical implementation that could really provide measures of how well the outcomes of the research-based actions did in comparison to the way things were done before the research was implemented. There were loads of information, and piles of analyses, and very little implementation, and that made it really, really hard to evaluate findings.

I need an environment where I can not only do research, but where I can follow what happens when people act on that research. I think good research can be done in many different environments, but for me at least, it’s easier to do good research when I have that wall – the reality of implementing the findings in a way that will either work or not work. If it works, that lends some support to the validity of the research. It it doesn’t work, that could mean the implementation was bad, but it could just as easily mean that my research was flawed. Either way, that interaction with practical application is what keeps me honest about my assumptions, my methods, and pretty much everything else in my research.


9 thoughts on “Good research usually needs walls. Really hard ones.

  1. One of the goals of this space is to try and lay out our work as we do it, in real-time so to speak. In our work we look primarily at human behavior and in explaining human behavior we generally consider it necessary to go beyond ‘end-of-the-road’ statements about a person’s motivations or traits. We want to get at how particular environments facilitate or constrain particular behaviors. Taking this approach it’s especially ironic how easy it is to get so caught up in the subject of your analysis or research and the why and how of the behavior you’re trying to understand and explain that you forget that your own research behavior is just as in need of explanation. If we care about the quality and value of our research we have to remain aware of our own behavior (our research processes) as much as possible and of the environments that are shaping and constraining those processes. Because sometimes those environments might be leading us to overlook systemic errors and over-value the quality of our work.

    The recognition that it isn’t just about what we WANT to do or how we WANT to conduct our research, but that our environments play a really big part of what we actually do, is a big part of what brought both of us out of government service. And like I said above, it’s a big part of why we’ve developed this space.

    But if we want to pay attention to our own environments in a way that’s useful, we have to do our best to be precise. So I’m going to extend your differentiation between government and the private sector a little. “Government” and “private sector” are just convenient labels we use to refer to two different human organizational patterns. Generally we use them because they work. But too often social science and media commentary use them as if they referred to two kinds of inherent essences and as if they needed no further explanation. In this particular case there are easily discernible differences between the Army organization we worked for and the private organization you work for now. The Army organization we worked for explicitly excluded almost all interaction with what you might call implementation. WorldStrides is a relatively smaller organization in which you will be directly and consistently interacting with people who will be almost entirely involved in making decisions, implementing ideas, and (as you put it) walking through doors or other gaps in walls. That’s a fairly simple difference and you kind of described it; I just wanted to make it a little more explicit.

    There’s no point getting into it here (as this wasn’t the point of the post). However it would be an interesting discussion to think about some of the ways the US army and the intelligence community are organized that basically proscribe the interaction between decision-makers and researchers/analysts and which make it very easy for those organizations (and the sub-organizations they comprise) to facilitate less and less useful analysis and actually constrain useful research. There are whole swaths of the US government (at least in the areas I worked in) that very very rarely think about how they’re organized and whether those organizational structures are sometimes ill-formed. The result is a whole lot of wasted resources, time, and effort.

    Hopefully your new position will allow you to think about the way your own organization and research group are organized and about your own environment (in addition to the behavior of your organization’s clients). If that’s the case, then hopefully your resulting analyses will be able to occasionally help change your organization and environment to improve your ability to do your primary research. …THAT would be an awesome position. :-)

  2. It’s interesting you mention both the idea of our research environments influencing our ability to actually do research and the project of describing how the government environments in which I worked constrained the opportunities for analysis to inform decision makers. The first topic is actually a big part of the reason I don’t yet feel competent to address the second topic. I’ve been wanting to analyze the environment of intelligence analysis for a long time…almost since I began my work for the Army…but I haven’t been able to do it. I took my job in the private sector mostly because it looks to be a really great job, but also partially because I felt I needed some distance from the environments in which I’d been working in order to really be able to see those environments clearly.

    And yes, I’m very excited about the prospect of my research funneling directly into implementation.

  3. Pingback: Research and the tools we use to do it « House of Stones

  4. Pingback: “Opportunistic analysis” sounds easier than it really is « House of Stones

  5. Schaun, Paul,

    I hope you eventually get around to studying the intelligence environment. I want to do the same, but I need distance from it for a while, and some new skills, before I feel ready to dive in. Whatever arc my training takes, I plan to eventually steer it back to at least one research project on the intelligence environment.

  6. Pingback: My problematic relationship with theory « House of Stones

  7. Pingback: Research at Disney | House of Stones

  8. Pingback: Houses of Stones

Comments are closed.