The credibility revolution has emphasized the exogeneity of the variation used for identification. This paper turns to its amount: it argues that by discarding variation to isolate exogenous sources, causal identification strategies reshape the underlying effective sample. This sample can be small, reducing statistical power and leading significant estimates to exaggerate true effects. Through theory and calibrated simulations, I show that the resulting exaggeration bias can exceed the confounding bias the strategy eliminates and formalize this confounding-exaggeration trade-off. I then introduce a ready-to-use diagnostic tool to characterize this effective sample in applied studies: the observations actually driving the causal estimate.
This paper identifies design parameters that can lead to inaccurate estimates of small effects. Low statistical power not only makes such effects difficult to detect but resulting significant estimates also necessarily exaggerate true effect sizes on average. Through the literature on short-term health effects of air pollution, I explore this issue and its policy implications. Exaggeration can be substantial and power low even with large sample sizes. Real-data simulations highlight key additional drivers: the number of exogenous shocks, instrument strength, and outcome count. I propose a workflow to evaluate and mitigate exaggeration risk in non-experimental studies.
Selected work in progress
Climate Change Narratives: Environmental News in French Newscasts
