Conflict Resolution & Security | Methodologies

Ask the Survey Doctor: Surveys in Conflict Zones and Non-Permissive Environments

By Craig Charney | Insights | Series II | No. 2 | January 2014

If you are struggling with surveys, evaluations, assessments, or market research in developing countries, email the Survey Doctor with your question. 

Q: Many development, security assistance, or conflict resolution programs necessarily operate in insecure environments. How are you able to conduct surveys in conflict zones and non-permissive environments?

A: This is a very real problem, particularly as development, defense, and diplomatic objectives have become more closely aligned since 2001. It’s important to be able to conduct surveys, assessments, and evaluations — but it’s essential to know how to operate in conflictual or non-permissive environments. It’s trickier than in more normal places — but it can be done.

The first issue in such places is often the lack of a good local organization to conduct research. Conflict zones — Tom Barnett’s famous “gap” — frequently are impoverished, underdeveloped places. Educated people are few and organizations with the capacity to handle sophisticated research methods rare.

Where organizations are weak or absent, the solution is to build capacity, often working together with local or regional expertise. In most of the 45 countries where we’ve worked, there has been some sort of local organization that has done polling or focus groups. When we had to, we’ve strengthened them.

Thus, after the fall of the Taliban, when we were planning the first-ever national poll there, we found an organization that had done dodgy men-only surveys. We helped them to plan a scientifically sampled survey, develop an interviewer training plan and a recruiting plan for female interviewers, and set up a train-the-trainer program. Much of the training was done by Indian pollsters we brought in, since they already knew how to do the things required to implement a survey, like make random selections in a village, in South Asian conditions.

Photo: Training women interviewers

The second issue is keeping interviewers safely out of hot spots. It’s important to remember that even in places where there is a lot of conflict, that doesn’t mean fighting is going on everywhere, all the time. Even in a place like Afghanistan, where there are over 300 districts, on any given day clashes are underway only in a small number of them. Nor is fighting district-wide: it usually occurs in a handful of villages or towns at any given time.

In this kind of environment, only a few sampling points tend to fall in places that are too hot to handle. When they do, the first step is to wait and go elsewhere: things may quiet down in a few days. If the whole area seems too violent for safe interviewing, we can randomly draw another village in the same district. If the whole district is too rough — which is rare — we can randomly substitute another district in the province. In most cases we have been able to reach all but a few districts and almost all of Afghanistan’s 34 provinces.  With the uptick in fighting matters have gotten worse in the past couple of years, but it is still possible to reach upwards of 90% of the population.

The third problem is areas where it is unsafe to send interviewers altogether, because government or opposition groups will not tolerate them. These, too, are fewer than one might think, as long as it is understood that the interviewers are ordinary people making a living, not government or insurgent agents.  (We even have pictures of black-turbaned Taliban smiling at passing interviewers.) This is an area where local knowledge is important. We work closely with interviewers and supervisors to make a district-by-district analysis to identify areas that are off limits and must be substituted.

In places more developed than Afghanistan, where non-permissive environments are more often due to political barriers than overt warfare, there may be work-arounds. For example, when we wanted to poll Iranians about their country’s nuclear program, we couldn’t hire local polling firms. The Iranian government has the nasty habit of locking up pollsters who ask inconvenient questions. So instead, we called in from a Farsi-language phone bank outside the country – and found Iranians were happy to talk.

Indeed, one of the ironies of a conflict society is that the people who are most afraid to speak out are often those who want to the most. In that first Afghan study, for instance, we began with qualitative interviews to determine what people would feel willing to talk about. Not only were they willing to answer all our questions, but when we finished, many of the transcripts end, “Thank you for asking. Nobody was ever interested in my opinion before.”