26 annotated studies in Trust & Safety

Reyns, B. W., Henson, B., & Fisher, B. S. (2011). Being pursued online: Applying cyberlifestyle-routine activities theory to cyberstalking victimization. Justice Quarterly, 28(2), 1–31.

Foundational adaptation of routine activity theory to online victimization. Using survey data, the authors find that engaging in online deviance (boundary-testing or risky online conduct) predicts cyberstalking victimization more powerfully than internet use frequency alone. The cyberlifestyle-routine activities theory (CLRAT) framework has since been used to study a wide range of online harms.

Why a T&S practitioner would cite this: Gives T&S a defensible explanatory model for why the user populations engaging in boundary-testing conduct are also the populations most often reporting victimization. Useful for designing reporting tools that do not penalize victims for risk exposure.

@misc{reyns2011beingpursued,
  title = {Reyns, B. W., Henson, B., & Fisher, B. S. (2011). Being pursued online: Applying cyberlifestyle-routine activities theory to cyberstalking victimization. Justice Quarterly, 28(2), 1–31.},
  doi   = {}
}

Costello, M., Hawdon, J., & Hughes, J. P. (2017). Contesting online extremism: Does arguing with racists help? Social Science Computer Review, 35(5).

Survey of online users finds that routine activities on social networking sites — particularly engaging with or confronting hate material online — significantly predicted being targeted with hate speech. The counter-intuitive finding: users who actively counter-spoke against hate were at higher risk of subsequent retaliation. Methodologically straightforward; conclusions widely cited.

Why a T&S practitioner would cite this: Undermines simple counter-speech interventions as protective. Has direct implications for how platforms route bystander reporting and design pre-emptive safety nudges.

@misc{costello2017contesting,
  title = {Costello, M., Hawdon, J., & Hughes, J. P. (2017). Contesting online extremism: Does arguing with racists help? Social Science Computer Review, 35(5).},
  doi   = {10.1177/0894439316666272}
}

Ballard, M. E., & Welch, K. M. (2022). Youths as targets: Factors of online hate speech victimization among adolescents and young adults. Journal of Computer-Mediated Communication, 27(4).

National quota-based survey (N=1,180) in Germany identified six distinct online hate victimization profiles among young users. Low digital media literacy combined with high political participation were independent predictors of victimization, controlling for minority identity. Profile-based analysis is methodologically useful for T&S harm-segmentation work.

Why a T&S practitioner would cite this: Identifies a specific high-value user segment — politically engaged but digitally less fluent users — where investment in accessible reporting and self-protection tooling would produce outsized harm reduction.

@misc{ballard2022youths,
  title = {Ballard, M. E., & Welch, K. M. (2022). Youths as targets: Factors of online hate speech victimization among adolescents and young adults. Journal of Computer-Mediated Communication, 27(4).},
  doi   = {10.1093/jcmc/zmac012}
}

Hine, G. E., Onaolapo, J., De Cristofaro, E., Kourtellis, N., Leontiadis, I., Samaras, R., Stringhini, G., & Blackburn, J. (2017). Kek, cucks, and God Emperor Trump: A measurement study of 4chan's politically incorrect board and its effects on the web. Proceedings of ICWSM 2017.

Large-scale data collection from 4chan's /pol/ board documents how hate content migrated to Twitter and Reddit, with quantitative measurement of cross-platform diffusion dynamics. Among the first studies to rigorously document the infrastructure of cross-platform hate migration — a phenomenon T&S teams now track routinely but rarely publish data on.

Why a T&S practitioner would cite this: Provides an empirical baseline and methodology for cross-platform harm migration tracking. Useful precedent for any researcher trying to study hate spread without platform-internal data.

@misc{hine2017kek,
  title = {Hine, G. E., Onaolapo, J., De Cristofaro, E., Kourtellis, N., Leontiadis, I., Samaras, R., Stringhini, G., & Blackburn, J. (2017). Kek, cucks, and God Emperor Trump: A measurement study of 4chan's politically incorrect board and its effects on the web. Proceedings of ICWSM 2017.},
  doi   = {}
}

Whitty, M. T. (2018). Do you love me? Psychological characteristics of romance scam victims. Cyberpsychology, Behavior, and Social Networking, 21(2).

Population survey (N=11,780) comparing romance scam victims to non-victims. Victims were disproportionately female, middle-aged, highly educated, and scored high on urgency, sensation-seeking, and addictive disposition. Crucially, loneliness alone was not predictive — a common assumption in the lay literature.

Why a T&S practitioner would cite this: Tells T&S fraud teams that high-education users are not protected by knowledge alone; the manipulation targets psychological rather than informational vulnerabilities. Implication: design friction around escalating commitment, not around initial contact.

@misc{whitty2018doyouloveme,
  title = {Whitty, M. T. (2018). Do you love me? Psychological characteristics of romance scam victims. Cyberpsychology, Behavior, and Social Networking, 21(2).},
  doi   = {10.1089/cyber.2016.0729}
}

Buchanan, T., & Whitty, M. T. (2014). The online dating romance scam: Causes and consequences of victimization. Psychology, Crime & Law, 20(3).

Two-study design (N=853 dating site users; N=397 self-identified victims). Romantic idealization — the tendency to construct an idealized partner — predicted scam vulnerability more strongly than neuroticism or loneliness. Establishes idealization dynamics as the locus of intervention.

Why a T&S practitioner would cite this: Suggests that fraud-friction features should be triggered by escalation patterns (idealization markers, accelerating intimacy) rather than by message content alone — a measurable behavioral signal class largely unused by current fraud detection.

@misc{buchanan2014romance,
  title = {Buchanan, T., & Whitty, M. T. (2014). The online dating romance scam: Causes and consequences of victimization. Psychology, Crime & Law, 20(3).},
  doi   = {10.1080/1068316X.2013.772180}
}

Cross, C., Smith, R. G., & Richards, K. (2014). Challenges of responding to online fraud victimization in Australia. Trends and Issues in Crime and Criminal Justice, No. 474. Australian Institute of Criminology.

Qualitative analysis of online fraud victim responses and institutional barriers to reporting. Documents extensive underreporting driven by embarrassment and victim skepticism that police will act. Establishes a robust dark figure for online fraud and shows the institutional response gap that platforms increasingly fill by default.

Why a T&S practitioner would cite this: Empirical grounding for the claim that platform-level fraud intervention is the de facto primary response for most online fraud victims — and a justification for substantial platform investment in fraud reporting and remediation.

@misc{cross2014challenges,
  title = {Cross, C., Smith, R. G., & Richards, K. (2014). Challenges of responding to online fraud victimization in Australia. Trends and Issues in Crime and Criminal Justice, No. 474. Australian Institute of Criminology.},
  doi   = {}
}

Wolak, J., Finkelhor, D., Mitchell, K. J., & Ybarra, M. L. (2008). Online "predators" and their victims: Myths, realities, and implications for prevention and treatment. American Psychologist, 63(2).

Landmark study correcting media narratives about online predators. Most online-facilitated sexual exploitation of minors does not involve deception about age or intent; victims are often adolescents who knowingly communicate with adults who express interest. Reshaped both public discourse and platform policy on grooming detection.

Why a T&S practitioner would cite this: T&S architectures that assume grooming looks like stranger-danger miss the modal case. This study is the empirical case for detection logic that targets seduction-based manipulation of adolescents rather than stranger contact alone.

@misc{wolak2008onlinepredators,
  title = {Wolak, J., Finkelhor, D., Mitchell, K. J., & Ybarra, M. L. (2008). Online "predators" and their victims: Myths, realities, and implications for prevention and treatment. American Psychologist, 63(2).},
  doi   = {10.1037/0003-066X.63.2.111}
}

Henshaw, M., Ogloff, J. R. P., & Clough, J. (2025). Unmasking the men who produce child sexual abuse material. Sexual Abuse.

Australian study using Victoria Police records (2004–2019) finds CSAM production cases recorded by police increased substantially over the period and documents criminal sub-types among producers. Different sub-groups differ in pathways, risk levels, and relationship to contact offending — taxonomically useful for triage.

Why a T&S practitioner would cite this: Gives T&S investigations teams an evidence-based framework for stratified threat triage. Not all CSAM-related accounts present the same downstream contact-offense risk; this study supports differentiated escalation logic.

@misc{henshaw2025unmasking,
  title = {Henshaw, M., Ogloff, J. R. P., & Clough, J. (2025). Unmasking the men who produce child sexual abuse material. Sexual Abuse.},
  doi   = {}
}

Seto, M. C., & Eke, A. W. (2015). Predicting recidivism among internet offenders and child pornography offenders. Sexual Abuse: A Journal of Research and Treatment.

Longitudinal study of CSAM offenders in Canada, identifying empirical risk factors for recidivism. Establishes that actuarial risk stratification of online sex offenders is empirically tractable using a small number of variables.

Why a T&S practitioner would cite this: T&S ban-evasion and re-offense detection should be grounded in the same actuarial logic that has been developed for offline recidivism. This is the citation that supports moving from heuristic strike-systems to data-driven risk scoring.

@misc{seto2015predicting,
  title = {Seto, M. C., & Eke, A. W. (2015). Predicting recidivism among internet offenders and child pornography offenders. Sexual Abuse: A Journal of Research and Treatment.},
  doi   = {10.1177/1079063215573889}
}

Internet Watch Foundation. (2024). Annual Report 2023. IWF.

Annual data on CSAM URLs identified and removed, including emerging trends. The 2023 report documents a record number of reports and rapid proliferation of AI-generated CSAM. Practitioner-facing but the most authoritative public-facing trend data for the field.

Why a T&S practitioner would cite this: The primary public-facing trend data for the CSAM threat landscape. Every T&S child-safety team should be able to cite the current year's headline numbers from memory.

@misc{iwf2024report,
  title = {Internet Watch Foundation. (2024). Annual Report 2023. IWF.},
  doi   = {}
}

Chen, A. Y., Nyhan, B., Reifler, J., Robertson, R. E., & Wilson, C. (2023). Subscriptions and external links help drive resentful users to alternative and extremist YouTube videos. Science Advances.

Matched survey and web-browsing data showing the YouTube algorithm rarely recommends extremist content to mainstream users; individuals who seek out such content already espoused extremist beliefs. Participation in extremist online spaces correlates with subsequent participation in extremist civil unrest. The strongest empirical challenge to date to the simple algorithmic-pipeline theory.

Why a T&S practitioner would cite this: Implies that recommendation-focused interventions may be less effective than infrastructure disruption (community removal, sharing limits) for the population most at risk. Major implication for extremism policy design.

@misc{chen2023subscriptions,
  title = {Chen, A. Y., Nyhan, B., Reifler, J., Robertson, R. E., & Wilson, C. (2023). Subscriptions and external links help drive resentful users to alternative and extremist YouTube videos. Science Advances.},
  doi   = {10.1126/sciadv.add8080}
}

GWU Program on Extremism. (2023). The third generation of online radicalization. George Washington University.

Framework paper arguing online extremism has moved through distinct generational phases; the current generation is characterized by fragmented, platform-agnostic networks that use mainstream platforms for recruitment and fringe platforms for operational planning. Strategic-orientation reading, not empirical research.

Why a T&S practitioner would cite this: Establishes the cross-platform coordination thesis that now structures most threat-intel work. Single-platform enforcement is necessary but not sufficient against organized extremist networks.

@misc{gwu2023third,
  title = {GWU Program on Extremism. (2023). The third generation of online radicalization. George Washington University.},
  doi   = {}
}

Bail, C. A., Argyle, L. P., Brown, T. W., Bumpus, J. P., Chen, H., Hunzaker, M. B. F., Lee, J., Mann, M., Merhout, F., & Volfovsky, A. (2018). Exposure to opposing views on social media can increase political polarization. PNAS, 115(37).

Randomized experiment (N=1,652) finding that exposure to bots posting opposing political views on Twitter increased polarization, particularly among conservatives with strong priors. Methodologically one of the strongest experimental studies in the social-media-and-politics literature.

Why a T&S practitioner would cite this: Challenges naive counter-speech and feed-diversification interventions. T&S policy teams considering recommendation-diversity tools need this study on the desk before designing them.

@misc{bail2018exposure,
  title = {Bail, C. A., Argyle, L. P., Brown, T. W., Bumpus, J. P., Chen, H., Hunzaker, M. B. F., Lee, J., Mann, M., Merhout, F., & Volfovsky, A. (2018). Exposure to opposing views on social media can increase political polarization. PNAS, 115(37).},
  doi   = {10.1073/pnas.1804840115}
}

Munn, L. (2019). Alt-right pipeline: Individual journeys to extremism online. First Monday, 24(6).

Qualitative analysis of radicalization trajectories through progressive YouTube content. Methodologically limited but historically important: catalyzed the radicalization-pipeline research agenda and shaped YouTube T&S policy. Worth reading alongside Chen et al. (2023) to understand the intellectual history.

Why a T&S practitioner would cite this: Intellectual history reading. Helps T&S researchers understand why the pipeline frame became dominant — and what its empirical limits are.

@misc{munn2019altright,
  title = {Munn, L. (2019). Alt-right pipeline: Individual journeys to extremism online. First Monday, 24(6).},
  doi   = {10.5210/fm.v24i6.10108}
}

Dragiewicz, M., Harris, B., Woodlock, D., Salter, M., Easton, H., Lynch, A., Campbell, H., Leach, J., & Milne, L. (2022). Technology-facilitated coercive control: Domestic violence and the competing roles of digital media platforms.

Systematic review establishing that TFA sits on a continuum with in-person IPV. Perpetrators use smartphones, social media, GPS tracking, and hacking to manufacture a sense of omnipresence and omniscience in victims. Foundational synthesis for the TFA harm category.

Why a T&S practitioner would cite this: Establishes the analytical category T&S teams now use for TFA harm. Safety features designed for third-party attackers fail when the threat actor is a credentialed intimate; this study explains why.

@misc{dragiewicz2022tfa,
  title = {Dragiewicz, M., Harris, B., Woodlock, D., Salter, M., Easton, H., Lynch, A., Campbell, H., Leach, J., & Milne, L. (2022). Technology-facilitated coercive control: Domestic violence and the competing roles of digital media platforms.},
  doi   = {}
}

Freed, D., Palmer, J., Minchala, D., Levy, K., Ristenpart, T., & Dell, N. (2018). "A stalker's paradise": How intimate partner abusers exploit technology. Proceedings of the ACM on Human-Computer Interaction, 2 (CSCW).

Focus groups with 39 IPV survivors and 50 professionals documenting the specific ways abusers exploit account access — shared accounts, tracking apps presented as family-safety tools, surveillance via smart-home devices. Standard reading for T&S product teams building account recovery and safety features.

Why a T&S practitioner would cite this: The specific failure modes documented here drive product requirements for account recovery, location-sharing, and family-safety features at every major platform with a TFA-aware safety team.

@misc{freed2018digital,
  title = {Freed, D., Palmer, J., Minchala, D., Levy, K., Ristenpart, T., & Dell, N. (2018). "A stalker's paradise": How intimate partner abusers exploit technology. Proceedings of the ACM on Human-Computer Interaction, 2 (CSCW).},
  doi   = {10.1145/3274304}
}

McGlynn, C., Johnson, K., & Rackley, E. (2024). "It's torture for the soul": The harms of image-based sexual abuse. European Journal of Criminology.

Working with the UK Revenge Porn Helpline, the first study to quantitatively examine NCII distribution across platforms. Meta-owned platforms (Facebook and Instagram) facilitated 78% of social-media NCII distribution cases. Reporting and image-removal processes vary enormously across platforms.

Why a T&S practitioner would cite this: Establishes specialist victim-support organizations as essential T&S partners — not adversaries. Also a benchmark for which platforms shoulder which share of NCII enforcement burden.

@misc{mcglynn2024ncii,
  title = {McGlynn, C., Johnson, K., & Rackley, E. (2024). "It's torture for the soul": The harms of image-based sexual abuse. European Journal of Criminology.},
  doi   = {10.1177/14773708241255821}
}

Eaton, A. A., Jacobs, H., & Ruvalcaba, Y. (2017). 2017 Nationwide online study of nonconsensual porn victimization and perpetration. Cyber Civil Rights Initiative.

National survey finding that 1 in 8 social media users have been victims of NCII. Has been replicated in subsequent surveys. The empirical grounding for substantial engineering investment in NCII hash-matching infrastructure analogous to the CSAM infrastructure built decades earlier.

Why a T&S practitioner would cite this: Prevalence data justifying T&S investment in NCII detection and hash-matching. If you are arguing internally for an NCII infrastructure budget, this is the citation.

@misc{eaton2017ncii,
  title = {Eaton, A. A., Jacobs, H., & Ruvalcaba, Y. (2017). 2017 Nationwide online study of nonconsensual porn victimization and perpetration. Cyber Civil Rights Initiative.},
  doi   = {}
}

Europol & EMCDDA. (2017). Drugs and the darknet: Perspectives for enforcement, research and policy. Joint publication.

Joint EU report providing empirical data on darknet drug markets (2011–2017), including supplier concentration, commodity mix, and market resilience after law-enforcement takedowns. Markets exhibited remarkable resilience: when AlphaBay and Hansa were seized, activity rapidly migrated to alternatives.

Why a T&S practitioner would cite this: The whack-a-mole dynamic documented here is directly analogous to ban-evasion patterns on surface-web platforms. Useful empirical anchor when arguing for cross-platform coordination over single-platform enforcement.

@misc{europol2017darknet,
  title = {Europol & EMCDDA. (2017). Drugs and the darknet: Perspectives for enforcement, research and policy. Joint publication.},
  doi   = {}
}

Martin, J. (2014). Lost on the Silk Road: Online drug distribution and the "cryptomarket." Criminology & Criminal Justice, 14(3).

Early qualitative study documenting community norms, trust mechanisms (reputation systems, escrow), and harm-reduction ethos of Silk Road participants. Illicit platforms develop internal social control systems that parallel and sometimes exceed those of legitimate platforms.

Why a T&S practitioner would cite this: Reminds T&S that bad-actor ecosystems are not unstructured — they have governance, dispute resolution, and reputation. Understanding these mechanisms sharpens detection and disruption strategy.

@misc{martin2014silkroad,
  title = {Martin, J. (2014). Lost on the Silk Road: Online drug distribution and the "cryptomarket." Criminology & Criminal Justice, 14(3).},
  doi   = {10.1177/1748895813505234}
}

Guriev, S., & Treisman, D. (2024). Online disinformation predicts inaccurate beliefs about electoral integrity. Comparative Political Studies.

Multi-level analysis of 80,000+ respondents across 74 democracies finding that higher volumes of online disinformation in a country's media ecology decoupled citizen perceptions of election fairness from actual electoral integrity. The strongest macro-level empirical case to date.

Why a T&S practitioner would cite this: Provides the macro-evidence that disinformation is structurally corrosive to democratic accountability — useful when justifying election-integrity investment to leadership that wants quantified harm estimates.

@misc{guriev2024disinformation,
  title = {Guriev, S., & Treisman, D. (2024). Online disinformation predicts inaccurate beliefs about electoral integrity. Comparative Political Studies.},
  doi   = {10.1177/00104140231193008}
}

Cinelli, M., Cresci, S., Quattrociocchi, W., Tesconi, M., & Zola, P. (2023). Coordinated inauthentic behavior: An innovative manipulation tactic to amplify COVID-19-related vaccine narratives. Online Social Networks and Media.

Analysis of a Meta-removed anti-vaccine adversarial network. CIB operates across three domains: structuring inauthentic communities, exploiting platform technology, and deceiving algorithms. The manipulation is not primarily about fake content but about the inauthentic amplification structure surrounding real grievances.

Why a T&S practitioner would cite this: Reframes CIB detection strategy: pure content-classification will miss most coordination. Detection must target amplification-structure signatures, not just message content.

@misc{cinelli2023cib,
  title = {Cinelli, M., Cresci, S., Quattrociocchi, W., Tesconi, M., & Zola, P. (2023). Coordinated inauthentic behavior: An innovative manipulation tactic to amplify COVID-19-related vaccine narratives. Online Social Networks and Media.},
  doi   = {}
}

Jhaver, S., Boylston, C., Yang, D., & Bruckman, A. (2021). Evaluating the effectiveness of deplatforming as a moderation strategy on Twitter. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW2).

Empirical study of deplatforming effects after Twitter removed three high-profile accounts. Removing prominent accounts did reduce their followers' downstream hate speech, but did not eliminate it. Mixed but real effect — deplatforming works partially, in immediate networks.

Why a T&S practitioner would cite this: Empirical baseline for the deplatforming-effectiveness question. T&S leadership being asked to justify high-profile removals can cite measured downstream harm reduction.

@misc{jhaver2021deplatforming,
  title = {Jhaver, S., Boylston, C., Yang, D., & Bruckman, A. (2021). Evaluating the effectiveness of deplatforming as a moderation strategy on Twitter. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW2).},
  doi   = {10.1145/3479525}
}

Jiang, S., Robertson, R. E., & Wilson, C. (2024). An in-depth study of online platforms' content moderation policies. arXiv.

First systematic study of content moderation policies across the 43 largest user-generated-content platforms. Platforms rarely explicitly define what they intend to moderate, rely heavily on users for moderation of some content, and provide little meaningful recourse after action — except for copyright.

Why a T&S practitioner would cite this: Empirically substantiates the procedural-justice critique of T&S enforcement. Useful citation for any platform attempting to design appeal and recourse systems that meet a higher procedural-justice bar.

@misc{jiang2024moderation,
  title = {Jiang, S., Robertson, R. E., & Wilson, C. (2024). An in-depth study of online platforms' content moderation policies. arXiv.},
  doi   = {}
}

Spence, R., Bifulco, A., Hamid, N., et al. (2024). Content moderator mental health, secondary trauma, and well-being. Cyberpsychology, Behavior, and Social Networking.

Survey establishing psychological baselines in commercial content moderators. Dose-response relationship: daily exposure to disturbing content predicted significantly higher psychological distress and secondary trauma symptoms. Supportive colleagues and meaningful feedback buffered the effect.

Why a T&S practitioner would cite this: Quantifies what every moderator already knows. Mandatory quotas requiring daily exposure to extreme content are not only an HR problem — they produce measurable psychological harm. Critical for designing humane operations infrastructure.

@misc{spence2024moderator,
  title = {Spence, R., Bifulco, A., Hamid, N., et al. (2024). Content moderator mental health, secondary trauma, and well-being. Cyberpsychology, Behavior, and Social Networking.},
  doi   = {}
}