This is a followup to my previous post on external validity and rigor, and a further attempt to pretend that this blog is not just about productivity, apps, and hacks.
Jed Friedman has a great piece on the Development Impact blog on a working paper by Hunt Alcott (who I cited in my previous post). Alcott describes a concept called “External Unconfoundedness”. This perfectly articulates what I was trying to get at in my previous post, an attempt to bring statistical notions of unbiasedness to questions of external validity. A lot of the conditions for external unconfoundedness have to do with the environment of the original study, and Alcott is particularly interested in site selection bias - the extent to which the setting for a study is chose because of favorable conditions.
Both the working paper and the post are great reads.
Link: Toward a more systematic approach to external validity: Understanding site-selection bias