Goal of the session

Air pollution and health

An instrument for air pollution

Formal intuition

• $y_i = \alpha + \beta x_i + u_i$ with $cov(x, u) \neq 0$

• Problem: $x_i$ is endogenous

• IV solution:

  • Find a variable $z$ that shifts $x$ for reasons unrelated to unobservables in $y$
  • It isolates variation in $x$ that is unrelated to $u$ and recover $\beta$
  • Focuses on the variation that is explained by the instrument

Potential outcomes

• Effect of the instrument on the treatment varies across individuals
• We only observe $z_i$ and $d_i$, not these groups

Identifying assumptions

• Exclusion restriction $\Rightarrow ITT_{\text{never-takers}} = ITT_{\text{always-takers}} = 0$.
• Motonicity $\Rightarrow$ compliers are the only individuals whose treatment status could be altered by the instrument

Exclusion restriction

• We can never show that the exclusion restriction holds
• Need to convince the audience that it does with logic arguments
• Need to find “clever” instruments
• But some instruments may turn out to be not so great

Exclusion restriction violations

• Sometimes an instrument affects many (many) other variables $\Rightarrow$ it may affect the outcome through another path (Mellon 2024)

What does the IV estimate?

$$\begin{aligned} \beta_{IV} & := \dfrac{\text{Cov}(y_i, z_i)}{\text{Cov}(d_i, z_i)} = \ ... \\ & = \dfrac{\mathbb{E}[y_i \mid z_i = 1] - \mathbb{E}[y_i \mid z_i = 0]}{\mathbb{E}[d_i \mid z_i = 1] - \mathbb{E}[d_i \mid z_i = 0]} = \ ... \\ & = \underbrace{\mathbb{E}\left[y_i(1) - y_i(0) \;\middle|\; d_i(1) = 1,\; d_i(0) = 0 \right]}_{\text{LATE on the compliers}} \end{aligned}$$

Intuition

• The instrument isolates the variation in treatment driven by $z$
• If treatment effects are heterogeneous, IV does not recover the overall average treatment effect (ATE)
• Instead, it identifies the effect local to those influenced by the instrument

Example

• Want to estimate the causal effect of education ($x$) on earnings ($y$)

• Education not randomly assigned

• Studied in Card (1993)

• Solution:

  • Use an instrument $z$, such as proximity to a college
  • Is it a valid instrument? Yes, arguably. It affects the likelihood of getting more education but does not affect earnings directly

• It estimates:

  • The ATE of education on earnings for those whose education decisions are changed by the instrument
  • ie students who attend college because they live nearby

Differences in estimands

Why instrument a randomised treatment?

Different estimands

• Find much larger effects with the IV than it’s reduced form
• Estimate different estimands

2SLS actually often not a LATE

IV Estimator and 2SLS

• Wald estimator: sample analog of the estimand

$$\widehat{\beta}_W = \dfrac{\widehat{\text{Cov}}(y, z)}{\widehat{\text{Cov}}(d, z)}$$

• Numerically equivalent to Two-Stage Least Squares (2SLS) estimator $\widehat{\beta}_{2SLS}$ obtained through:

  1. Regress $d$ on instrument(s) $z$ and controls and retrieve predicted values

  $$d_i = \pi_0 + \pi_1 z_i + v_i \quad \Rightarrow \quad \widehat{d}_i = \widehat{\mathbb{E}}[d_i \mid z_i]$$

  1. Regress $y$ on predicted treatment $\hat{d}$

  $$y_i = \alpha + \beta \widehat{d}_i + e_i$$

Control function approach

• Alternative approach for the IV
• Steps:
  1. Regress $d$ on $z$ and controls
  2. Retrieve the residuals (the part of $d$ unexplained by $z$)
  3. Regress $y$ on $d$ and these residuals
• Adjusting for these residuals partials this unexplained variation out
• Only leaves out the explained variation

Practical advice

Exercise

1. Run two regressions on the wooldridge::card data using fixest::feols to compare the 2SLS and the control function approaches
   • Key variables:
     • Outcome: lwage (log wage)
     • Treatment: educ (years of schooling)
     • Instrument: nearc4 (near 4-year college)
     • Controls: exper, black, south, smsa, fatheduc, motheduc
2. Build a simple and naive simulation to explore the impact of the use of instruments on measurement error (depending on the correlation between measurement error and the treatment)

Intuition

Definition

Consequences

1. Large variance: Confidence intervals can be very wide
2. Finite-sample bias: IV can be biased towards OLS (especially in small samples) because the IV leverages endogenous variation
3. Distorted inference: Standard t-tests and F-tests become unreliable (non-normal errors)

• Testing for relevance: compute the first-stage F-stat but use the Olea and Pflueger (2013) effective F-stat
• Report identification-robust Anderson-Rubin confidence intervals
• But basically look for a stronger instrument

Behind the F-stat

• With one instrument, the first-stage $F$ is (Keane and Neal 2024)

$$F = N \times \dfrac{R^2_{\text{partial}}}{1 - R^2_{\text{partial}}}$$

• $\Rightarrow$ $F$ grows with the sample size. Instrument strength does not

• A negligible $R^2_{\text{partial}}$ passes $F > 10$ if $N$ is large enough (Bound, Jaeger, and Baker 1995)

• And $10$ is the wrong number: valid 5% $t$-inference needs $F > 100$ (Lee et al. 2022)

Exercise

Bartik’s setting

• Shift-share instruments also named Bartik instruments after Bartik (1991)

• Explores the effect of immigration ($I$) on local wages ($y$)

$$y_{l,t} = \alpha + \delta I_{l,t} + \rho X_{l,t} + \varepsilon_{l,t}$$

• Problem: endogeneity of $I$, eg migrants locate where the economy is doing well

• Bartik instrument interacts:

  • Shares: where migrants from each origin $k$ historically settled
  • Shifts: the national inflow from each origin

$$B_{l,t} = \sum_{k=1}^{K} \underbrace{z_{l,k,t_0}}_{\text{share}} \times \underbrace{m_{k,t}}_{\text{shift}}$$

• A city’s predicted inflow is a weighted average of national flows, weighted by its initial composition

Intuition

Identifying assumptions

To check

• Depending on the approach, check:

  • Shares approach: which shares actually drive the estimate (Rotemberg weights)?
  • Shocks approach: are there enough shocks, are they plausibly as-good-as-random?

• Shares often correlate with a lot of things

• With hundreds of industries and many periods, “the exclusion restriction holds” is a big claim

Air quality alerts in Mexico City

Why does that work?

• Nothing else about a day changes at 155 ppb

  • ie local continuity

• There are days just above and just below

  • ie local support

• Crossing 155 ppb does raise the chance of an alert, by 74 pp

  • ie relevance. But here not mechanical $\Rightarrow$ fuzzy design

• Crossing it only matters because it triggers the alert

  • ie exclusion restriction

Other examples of discontinuities?

In a nutshell

• Treatment is not randomly assigned, it is assigned by a rule:

  • A continuous running variable $x_i$ crosses a cutoff $c$
  • $d_i = \mathbb{1}\{x_i \geq c\}$
  • eg a county is regulated if its pollution exceeds a federal threshold

• Estimand:

$$\beta_{RD} = \lim_{x \to c^+} \mathbb{E}[y_i \mid x_i = x] - \lim_{x \to c^-} \mathbb{E}[y_i \mid x_i = x] = \underbrace{\mathbb{E}[y_i(1) - y_i(0) \mid x_i = c]}_{\text{ATE at the cutoff}}$$

• Econometric model:

$$y_i = \alpha + \beta d_i + f_l(x_i) + f_r(x_i)d_i + e_i$$

Identifying assumptions

Threats to local continuity

No overlapping support

Locality

• An RDD identifies an effect at the cutoff $\Rightarrow$ local

• Says nothing about units far from the threshold

• Extrapolating away from $c$ requires extra assumptions, not more data

• In a fuzzy RD two forms of locality combinded: effect on compliers at the cutoff

Sharp and fuzzy

• Crossing only makes the treatment more likely

  • eg individuals above the threshold are designated, but not all comply

Fuzzy RD as an IV

Functional form near the cutoff

base_plot <- window |> 
  ggplot(aes(x = xc, y = y)) + 
  geom_point(alpha = 0.35) +
  geom_vline(xintercept = 0, linetype = "dashed") +
  labs(
    title = "Raw data",
    subtitle = "One draw, within a bandwidth of 15. The true jump is 3",
    x = "Distance to the cutoff", 
    y = "Outcome"
  )

base_plot
base_plot +
  geom_smooth(
    aes(group = above), 
    method = "lm", 
    se = FALSE,
    formula = y ~ poly(x, 1, raw = TRUE)
  ) +
  labs(
    title = "Linear fit",
    subtitle = "The true jump is 3"
  )
base_plot +
  geom_smooth(
    aes(group = above), 
    method = "lm", 
    se = FALSE,
    formula = y ~ poly(x, 2, raw = TRUE)
  ) +
  labs(
    title = "Quadratic fit",
    subtitle = "The true jump is 3"
  )
base_plot +
  geom_smooth(
    aes(group = above), 
    method = "lm", 
    se = FALSE,
    formula = y ~ poly(x, 4, raw = TRUE)
  ) +
  labs(
    title = "Flexible fit, order 4",
    subtitle = "The true jump is 3"
  )
result_sim |> 
  summarise(
    `Average estimate` = mean(estimate),
    `True effect` = mean(beta_0),
    `Standard deviation` = sd(estimate),
    .by = c(curvature, order)
  ) |> 
  rename(DGP = curvature, Model = order) |> 
  arrange(DGP, Model) |> 
  kable(digits = 2)

Bandwidth size

Statistical power of RD designs

Addressing heaping

Checking the design

Strengths and weaknesses - IV

Strengths and weaknesses - RDD