Mike Denly

  Tuesday, 7th May 2024

  16:30 - 17:30

   Nuffield Fellows Dining Room

   External Validity for Social Inquiry

ABSTRACT

External validity is both everywhere and nowhere. It is everywhere because very few studies aim to only make inferences for one specific case or sample. External validity is also nowhere because the number of studies that make credible inferences beyond the cases or samples that they analyze are so few. Building on insights from the “credibility revolution” associated with an increased focus on causality, we demonstrate how researchers can make their inferences more generalizable to larger populations. When such inferences are impossible, we also show researchers how to make transportable inferences to new target populations. Achieving these end goals necessitates better conceptualization. Notably, it requires more attention to the distinction between populations and samples, generalizability and transportability, and the dimensions of external validity at their different levels of abstraction: that is, Mechanisms, Settings, Treatments, Outcomes, Units, and Time (M-STOUT). To ensure consistent uptake of our recommendations and more reliable assessment of external validity, we develop three new evaluative criteria: Model Utility, Scope Plausibility, and Specification Credibility. Thereafter, we show how to use the new evaluative criteria in core experimental, quantitative observational, qualitative, and synthesis methods. Finally, we provide a roadmap for better reporting on external validity that will benefit researchers, reviewers, journal editors, and the general public.