Lecture 3 - Power and Causal Inference

class: right, middle, inverse, title-slide

.title[
# Lecture 3 - Power and Causal Inference
]
.subtitle[
## <br> Topics in Econometrics
]
.author[
### Vincent Bagilet
]
.date[
### 2024-09-25
]

---

# Exam

- **Final project** in pairs

- Research question and data `\(\to\)` fake data simulations `\(\to\)` analysis on the actual dataset

- Come up with a research question and data by October 9th

- We will suggest some data sets

- 3 pages reports & code in Quarto

- November 20th: **15 min presentations**

---
class: titled, middle

# Summary from last week

- **Goal** in applied research: accurately estimate the parameters of interest

- Capturing these parameters can be challenging

- **Simulations** can help avoid some pitfalls

- *Can we hope to accurately estimate the true effect of interest?*

- If the analysis faces issues in simulations, it will probably also in an actual setting 
  
---
class: titled, middle

# Usefulness of simulations

- Accurate estimation, at least in a "pristine" setting

- What about a more complex setting?

- What happens to our conclusions if some hypotheses do not hold?

- Simulations can help to understand statistical methods

---
# Steps of the simulation approach

1. Define a DGP and the distribution of variables

1. Set parameters values (`baseline_param`)

1. Generate a data set  (`generate_data()`)

1. Estimate the effect in the generated data set  (`run_estim()`)

1. Repeat many times (`compute_sim()` and `pmap()`)

1. Compute the measure of interest

1. Change parameters values (potentially)

1. Complexify the DGP

--
  
1. Repeat

---
class: right, middle, inverse
layout:false

# Exercise
## Simulating an RCT

---
class: titled, middle

## Setting

- Impact of receiving extra lessons on students’ grades

- Simulate an experiment (RCT):

`\(\forall i \in \{1, .., n\}, \quad Grade_i =  \alpha_0 + \beta_0 Treat_i + u_i\)`

- Which sample size and proportion of treated to have a high probability of detecting the effect?

## How?

- Vary the parameters values to find "correct" ones

---
class: titled, middle

# Coding in Quarto

- Create a new `.qmd` document

- Copy/paste (from the website) the **necessary** lines of code to run a simulation

- Modify parameters values

- Plot how power evolves with `\(n\)`

---
class: right, middle

# Switch to Quarto document for coding

---
# Simulations so far

- We have implemented a simple simulation

- We learned that:

- It can help detect design issues

- They are actually pretty simple to implement

- Now we can:

- Complexify the data generating process (mostly modify `generate_data()`)
  
  - Explore other challenges
  
???

- We generated fake data, estimated our parameter of interest, repeated with different parameters

---
class: right, middle, inverse

# A primer on exaggeration

---

---

---

---

---

---

---
class: titled, middle

# Evolution

- Exaggeration is a decreasing function of statistical power

- Decreases with precision

- Decreases with effect size

- Can be result from the use of causal methods

---
class: right, middle, inverse

# Causal Exaggeration

---
# The Confounding-Exaggeration Trade-Off

.pull-left[

- Challenge of empirical economics: identifying causal effects

- Only use part of the variation: the exogenous part

- Reduces precision and statistical power

- `\(\mathbb{E}[\hat{\beta}] = \beta\)` but `\(\mathbb{E}[\hat{\beta} | \text{ Significant}] \neq \beta\)`
]
--
.pull-right[
<img src="data:image/png;base64,#images/intuition_trade_off.png" width="600" style="display: block; margin: auto;" />
]

---

---

---

---

---
class: right, middle, inverse

# Thank you!