Relative Risks of State-Led Mass Killing Onset in 2017: Results from a Comparison Survey

The dot plot below displays the survey results in an alternative format; namely, it lists countries in rank order and shows their corresponding survey scores. (NOTE: the survey scores plotted here do not reflect estimated probabilities of a mass-killing onset. Instead, they are assessments of relative risk.)

A few issues should be considered when evaluating the survey results. First, countries with red dots likely had ongoing episodes of state-led mass killing in 2016: Democratic Republic of the Congo, Egypt, Myanmar, Nigeria, North Korea, South Sudan, Sudan, and Syria. Though it is possible these countries will experience a new onset of mass killing in 2017, respondents might have confused the risk of a new episode—in which the state targets a different group than the one(s) already being targeted—with the presence or intensity of an ongoing one. Second, there are several ongoing episodes of mass killing being perpetrated by non-state actors (e.g., Boko Haram in Nigeria and the Islamic State in Iraq). Although non-state actors are outside the scope of our survey, the presence of such violence might have skewed survey respondents’ estimation of the likelihood of a new state-led mass killing. Third and finally, we suspect that some respondents might have confused the distinct countries “Republic of the Congo” and “Democratic Republic of the Congo,” potentially muddling the survey outcomes for those two countries.

Now that we have seen the risk assessments from this year’s survey, let us compare to last year. The plot below shows 2016 rankings compared with 2017. Red arrows identify countries that climbed at least 10 spots, while blue arrows identify countries that fell at least 10. Among the biggest gainers in risk rank, according to our wikisurvey, were Gambia (69th to 22nd), Philippines (65th to 18th), Bosnia (94th to 50th), Djibouti (101st to 59th), Malawi (89th to 52nd), Venezuela (67th to 32nd), and Tajikstan (72nd to 37th). Among the biggest decliners were Senegal (54th to 122nd), Colombia (34th to 94th), Lesotho (61st to 110th), Niger (22nd to 67th), Haiti (49th to 91st), Ukraine (18th to 56th), and Tunisia (53rd to 88th).

The results from our annual wikisurvey do not contribute directly to our statistical risk assessments, but they do inform the selection of cases we track in real time with our Opinion Pool. Throughout 2017, we are launching opinion pool questions about risks of mass killing in roughly 15 countries worldwide. The set of countries includes the top states from the 2016 statistical risk assessments, plus a few additional ones that are salient to some of our core audiences.

As with all our forecasts, we plan to track and report on the accuracy of the assessments this wiki survey produces. Because the survey results are relative rankings, we cannot score them the way we would score probabilities, but we can see if any countries that suffered onsets of state-led mass killing had ranked near the top of this list. At the moment, our researchers are still reviewing information from 2016 to determine where episodes of mass killing began, continued, or ended. After that review has been completed and the results posted, we will take a preliminary look at the accuracy of our various risk-assessment methods—statistical modeling, wiki survey, and opinion pool—over the past year.

Methodology

We used All Our Ideas to conduct this pairwise wiki survey. For more on how a pairwise wiki survey works, see this working paper by the instrument’s creators. For details on what we mean by “state-led mass killing” and “onset,” see the blog post that led respondents to the survey.

We launched this survey with that blog post on December 1, 2016 and closed it at the beginning of 2017. We publicized the survey through a few channels, including emails from the Center for the Prevention of Genocide and Ben Valentino to partners and colleagues who work on this topic; emails to our own relevant professional networks; ads from our Center for the Prevention of Genocide group on Facebook; and occasional tweets throughout the month.

We do not and cannot know the identities of our survey respondents. Participation in All Our Ideas surveys is anonymous, and our blog post was shared publicly. However, we do know that the participants cast 4,540 votes in 124 unique user sessions. The number of votes was lower than last year, but greater than two years ago. Some individuals likely voted in more than one session, so the total count of respondents is some unknown figure less than or equal to 124 (down from 160 last year). Several people affiliated with the Early Warning Project participated, but we actively discouraged them from casting a large number of votes so that our results would not be heavily influenced by our own internal assessments.

The chart below shows that voting began slowly, but it picked up markedly after we began tweeting and publicizing via Facebook more regularly. The map below shows that, as in previous years, most of the votes in our wiki survey came from the United States and Europe. We were heartened to see votes from Asia, Latin America, and the South Pacific as well, but we hope to attract more participation from those regions and from Africa and the Middle East in future surveys.