Simulations for regression analysis
What is the usefulness of simulations in regression analysis and how to implement them?
After this session, you should be able to implement a basic simulation for regression analysis in R and use it to test some of the hypotheses you made in you own analysis.
Summary
In this session, we first discuss why we should implement simulations. The basic idea behind simulations is that the Data Generating Process is known: in particular, that allows us to evaluate the performance of our analysis and to study what happens if an hypothesis made does not hold, etc. Simulations can thus be helpful at the design stage of an econometric analysis but can also be leveraged as checks or to build intuition on econometrics mechanisms. To build simulations, one should start with a simple DGP, wondering whether their analysis performs well in a rather “pristine” setting. Then, they can complexify the DGP. In the second part of the lecture, we implement a simulation in R.
Session Outline
- What are simulations?
- Why do simulations?
- To understand econometric concepts
- To design a study
- For tests and checks
- As a rhetorical tool
- How to implement a simulation?
- Coding a simulation in R together
Main take-aways
- Whenever we do not understand an econometric mechanism, a simulation can help understand it: plays with simulations, often
- They are helpful before designing a study, even a non-experimental one but also after the analysis
- I advise you to use them quasi-systematically
- They are quite easy to implement
Materials
App
A small app to explore hurdles through simulations can be found here.
Exercise
During the lecture, we emulate an RCT studying the impact of additional courses on students’ grades, in this document.
In addition, you have a graded assignment for next week.
Specific resources for this lecture
Handbooks chapters
- Chapter 15 of The Effect (Huntington-Klein 2022) provides an intuitive explanation of the rationale behind simulations.
- Chapter 5 of Regression and Other Stories (Gelman, Hill, and Vehtari 2021) also discusses the rationale behing simulations.x
Papers using simulations
- Bertrand, Duflo, and Mullainathan (2004): a canonical paper, implementing placebo simulations on an actual data set to document the impact of serial correlation on the standard errors
- Black et al. (2022): implement power simulations building on the concrete case of the expansion of the affordable Care Act Medicaid and provide concrete advice to build such power simulations