Thread by @JamesKhalil1

How *NOT* to conduct field research into terrorism and violent extremism

A thread

1/

This thread focuses on UNDP’s “Journey to Extremism in Africa” research as it commits a series of basic methodological errors, and presents its findings with unwarranted degrees of confidence in light of issues with sampling and data reliability.

2/

I have been critical of this research before, and so I should clarify the reason I continue to point to its inadequacies is as it continues to have a major influence on policymakers. The errors are so pronounced they may actually undermine our fight against violent extremism

3/

To be clear, I am not arguing that all the UNDP findings are necessary ‘wrong’, but they are skewed towards a specific “developmentalist” perspective as a consequence of the inherent flaws in the research design. We desperately need to do better than this.

4/

I also want to clarify that the takeaway message is not only that researchers must get better at delivering rigorous research, but also that policymakers and programme implementers must become sufficiently informed to spot inadequacies and limitations in this research.

5/

The UNDP study draws on interviews with 495 current & former violent extremists in Africa, dividing this main sample into ‘voluntary’ and ‘forced’ members. For comparison, it also draws on a ‘reference group’ of 145 individuals loosely matched in terms of demographic traits.

6/

The first key point of criticism is that while this study places a heavy emphasis on structural factors such as poverty & state violence, it simultaneously neglects or entirely overlooks many critical individual incentives and factors that enable this violence.

7/

E.g. psychosocial factors such as fear, excitement and vengeance are only discussed towards the end of the report, where they are inexplicably framed as ‘emotions when joining’ violent extremist groups, rather than key drivers of enlistment that they are widely known to be.

8/

It also neglects how territorial control maintained by such groups often sets the conditions for enlistment (see the work of Stathis Kalyvas). As such, it downplays the relevance of rights-compliant military and counter-terrorism responses that aim to reclaim territory.

9/

The sample is also biased in terms of country of origin, and probably also the members’ rank and role, the extent of their ideological motivation, etc. The ‘reference group’ is also poorly matched in a variety of ways, which places major question marks over all comparisons.

11/

While some of this would be acceptable for a qualitative study offering nuance on individual trajectories, the findings are presented in quantitative terms, with inferences far beyond what the sample allows. The report does offer some caveats, but these are largely buried.

12/

Another issue is that the report places a very heavy emphasis on ‘tipping points’ in journeys to violent extremism. Without doubting that these do occur in specific cases (as my own research shows), the UNDP report probably far exaggerates the extent to which they occur.

13/

The problem is that this finding was derived from the following ‘leading’ question: "What specific thing happened that finally motivated you to join the organization?" This represents an elementary methodological error that even non-researchers should know to avoid.

14/

Finally, the report far understates the extent to which violent extremists may offer false or misleading information, e.g. to be viewed favorably, to discredit others, as they are ill-informed, through unwitting self-deception, as they have flawed memories, etc.

15/

Again, a few caveats are offered in the report, but these are essentially buried from view. Researchers have a responsibility to be clear with readers about the extent to which the information they collect may be unreliable or false, and to explain why this may be the case.

15/

For those interested in finding out more about how this research should be conducted, please check out ...

tandfonline.com/doi/full/10.10…