Relative Risks of Mass Killing Onset in 2018: Results from a Comparison Survey

According to participants, who cast 10,801 votes in a series of head-to-head match-ups, the top 10 countries at risk for mass killing in 2018 are: 

1. South Sudan
2. Democratic Republic of the Congo (Kinshasa)
3. Myanmar/Burma
4. Yemen
5. Central African Republic
6. Syria
7. Afghanistan
8. Somalia
9. Sudan
10. Iraq
11. Burundi
12. Nigeria
13. Pakistan
14. Mali
15. Egypt
16. Libya
17. Zimbabwe
18. Chad
19. Niger
20. Republic of the Congo (Brazzaville)

Countries in italics had ongoing episodes of mass killing at the end of 2017 (see State of the World blog post). Though it is possible these countries will experience a new onset of mass killing in 2018, respondents might have confused the risk of a new episode—in which an armed group or the state targets a different group than the one(s) already being targeted—with the persistence of an ongoing one.

The annual wiki survey is one of three quantitative methods used by the Early Warning Project to assess atrocity risk worldwide (see our statistical risk assessments and public opinion pool). Like the opinion pool, the wiki survey methodology draws on a “wisdom of the crowd” approach. Our survey was open to the public for the month of December 2017, and specifically promoted to experts, policymakers, NGOs, and scholars in international affairs. A pairwise wiki survey involves a single question—in this case, Which country is more likely to see a new episode of mass killing in 2018?—with many possible answers on which participants vote, one pair at a time. The response options presented to participants (every country with a population greater than 500,000) are randomly generated at the beginning of the survey period. As the survey progresses and more people record responses, the survey mechanism “learns” from previous responses and presents increasingly difficult pairs.

The results are mapped below. The darker the shade of red, the greater the risk that a new episode of mass killing will begin in 2018, according to survey respondents. See all results on All Our Ideas. 

EWP Wikisurvey Results Heatmap

Now that we have seen the risk assessments from this year’s survey, let us compare to last year. The table below shows 2017 rankings compared with 2018 for the top 60 countries. Rows highlighted in red identify countries that climbed at least 10 spots, while blue highlights identify countries that fell at least 10. We consider gains and losses to be more meaningful in the high to medium risk countries (top 30 and 60, respectively), in comparison to lower ranking countries where moves up and down are less significant. 

The results from our annual wikisurvey do not contribute directly to our statistical risk assessments, but they do inform the selection of cases we track in real time with our public Opinion Pool. We are currently tracking 17 countries, 15 of which scored in the top 30 in the wikisurvey. To participate in regular forecasting, and see results in real time, create an account on Good Judgment Open.

Methodology

We used All Our Ideas to conduct this pairwise wiki survey. For more on how a pairwise wiki survey works, see this working paper by the instrument’s creators. For details on what we mean by “mass killing” and “onset,” see the blog post that led respondents to the survey. We launched the survey with that blog post on December 1, 2017 and closed it on December 31, 2017.

We do not know the identities of our survey respondents. Participation in All Our Ideas surveys is anonymous, and our blog post was shared publicly. However, we do know that participants cast 10,825 votes in 217 unique user sessions, compared to 4540 last year. Some individuals likely voted in more than one session, so the total count of respondents is some unknown figure less than or equal to 217 (compared to 124 in 2016 and 160 in 2015).

The chart below shows that voting began slowly, but spiked when we promoted the survey via targeted emails and Twitter. As in previous years, most of the votes in our wiki survey came from the United States and Europe. We were heartened to see votes from Asia, Latin America, and the South Pacific as well, but we hope to attract more participation from those regions and from Africa and the Middle East in future surveys.