Short feedback form

https://forms.gle/wxwZNFXyPxprLELT7

Steps of an Econometrics Analysis

Goal of the session

Modelling assumptions matter

Implications

Normally distributed estimator

Exercise

Non-linear

n <- 1000
alpha <- 10
beta <- 2
mu_x <- 2
sigma_x <- 1
sigma_u <- 2

data_non_linear <- tibble(
  x = rnorm(n, mu_x, sigma_x),
  u = rnorm(n, 0, sigma_u),
  y = alpha + beta*x^2 + u  
)

reg_non_linear <- lm(y ~ x, data = data_non_linear) 

# list("Non-linear" = reg_non_linear) |> 
#   modelsummary(gof_omit = "IC|Adj|F|RMSE|Log")

Collinearity

gamma <- 0.2

data_collin <- tibble(
  x = rnorm(n, mu_x, sigma_x),
  w = 0.3*x,
  u = rnorm(n, 0, sigma_u),
  y = alpha + beta*x + gamma*w + u  
)

reg_collin <- lm(y ~ x + w, data = data_collin)

# list("Perfect collin." = reg_collin) |> 
#   modelsummary(gof_omit = "IC|Adj|F|RMSE|Log")

data_almost_collin <- tibble(
  x = rnorm(n, mu_x, sigma_x),
  w = 0.3*x + rnorm(n, 0, 0.01),
  u = rnorm(n, 0, sigma_u),
  y = alpha + beta*x + gamma*w + u  
)

reg_almost_collin <- lm(y ~ x + w, data = data_almost_collin)

# list("Almost perfect collin." = reg_almost_collin) |> 
#   modelsummary(gof_omit = "IC|Adj|F|RMSE|Log")

Endogeneity

data_endog <- tibble(
  x = rnorm(n, mu_x, sigma_x),
  u = 0.5 * x + rnorm(n, 0, sigma_u),
  y = alpha + beta*x + u  
)

reg_endog <- lm(y ~ x, data = data_endog)

# list("Endogeneity" = reg_endog) |> 
#   modelsummary(gof_omit = "IC|Adj|F|RMSE|Log")

Autocorrelation

u_auto <- numeric(n)
for (i in 2:n) u_auto[i] <- 0.9*u_auto[i-1] + rnorm(1, 0, sigma_u)

data_auto <- tibble(
  x = rnorm(n, mu_x, sigma_x),
  u = u_auto,
  y = alpha + beta*x + u
) 

reg_auto <- lm(y ~ x, data = data_auto) 

# reg_auto |>
#   modelsummary(gof_omit = "IC|Adj|F|RMSE|Log", vcov = c("classical", "HAC"))

Heteroskedasticity

data_heterosked <- tibble(
  x = rnorm(n, mu_x, sigma_x),
  u = rnorm(n, 0, sigma_u + x^2),
  y = alpha + beta*x + u  
)

reg_heterosked <- lm(y ~ x, data = data_heterosked) 

# reg_heterosked |> 
#   modelsummary(gof_omit = "IC|Adj|F|RMSE|Log", vcov = c("classical", "robust"))

Non-normal errors

df <- 2

data_non_normal <- tibble(
  x = rnorm(n, mu_x, sigma_x),
  u = rt(n, df), #fatter tails
  y = alpha + beta*x + u  
)

reg_non_normal <- lm(y ~ x, data = data_non_normal)

# reg_non_normal |> 
#   modelsummary(gof_omit = "IC|Adj|F|RMSE|Log", vcov = c("classical", "bootstrap"))

Limited outcome models

Limited outcome models

Binary outcome

Example

• The outcome follows a Bernoulli distribution: $y | X \sim Ber(\pi)$
  
• A regression model expresses the conditional probability $\pi = P[y = 1|X]$ as a function of $X$ and $\beta$: $\pi = g^{-1}(X'\beta)$

Binary outcome

Models

• Linear Probability Model
  • $g : x \mapsto x$
  • Almost always yields biased and inconsistent estimates
• Logistic regression model
  • $g : x \mapsto log\left(\dfrac{x}{1-x}\right)$, the logit function
• Probit regression model
  • g: probit, ie the quantile function of the standard normal distribution (and $g^{-1}$ is the CDF of the normal)
  • Rule of thumb: coefs $\simeq$ logit coefs divided by 1.6
  • Very similar to logit; sometimes easier to implement

Binary outcome models

Interpreting coefficients

• $\beta$ gives the direction BUT not the magnitude
  
• The marginal effect depends on $X$: effects largest in the middle of the distribution

Binary outcome models

Example fits

Count data models

A note on controls

• Overall two reasons to include controls:
  1. To ensure random allocation of the treatment
     • Necessary for identification
  2. To improve precision
     • To better explain $y$: $\nearrow$ $R^2 \ (= 1 - FUV)$ and $\searrow$ $\sigma_u^2$)
• Adjusting for pre-treatment covariates may
  • Reduce the variation in $y$, increases precision
  • Reduce the variation in $x$, decreases precision

Simulating bad controls

#reuse the parameters from above
n <- 10000 #to limit sampling variation
mu_b <- 3
sigma_b <- 1
gamma <- 5
kappa <- 4
sigma_a <- 2
delta <- 0

data_bad_control <- tibble(
  x = rnorm(n, mu_x, sigma_x),
  u = rnorm(n, 0, sigma_u),
  b = rnorm(n, mu_b, sigma_b),
  a = kappa*x + rnorm(n, 0, sigma_a),
  y = alpha + beta*x + gamma*b + delta*a + u  
)

reg_short <-  lm(data = data_bad_control, y ~ x)
reg_pre <- lm(data = data_bad_control, y ~ x + b)
reg_post <- lm(data = data_bad_control, y ~ x + a)
reg_pre_post <- lm(data = data_bad_control, y ~ x + b + a)

SEs: what are they?

Why care about them?

(Non-)spherical errors

• Under assumptions 1 $\to$ 3 we saw earlier, the asymptotic distribution of $\widehat{\beta}_{OLS}$ is

$$\widehat{\beta}_{OLS} \overset{a}{\sim} \mathcal{N}(\beta_0, (X'X)^{-1} X'\Sigma X (X'X)^{-1})$$

• If spherical errors (ie $\Sigma = \sigma^2 I$), use unbiased sample variance
• If non-spherical errors, need a covariance matrix estimator that is consistent under this misspecification
• $\Rightarrow$ use sandwich estimators of the variance ($\underbrace{(X'X)^{-1}}_{bread} \underbrace{X'\widehat{\Sigma} X}_{meat} \underbrace{(X'X)^{-1}}_{bread}$)
• Heteroskedasticity: compute White SEs
• Autocorrelation: compute Newey-West/Conley SEs if correlated in time/space

Why clustering SEs

• When errors are correlated within groups (e.g. individuals, firms, regions), clustering adjust for this

$$\mathbb{E}[u_iu_j| X] \neq 0 \text{ for } i, j \text{ in the same cluster}$$

• Examples
  • Panel data: repeated measures of same unit
  • Group-level treatment, eg policy at regional level with many individuals per region
• Intuition
  • We do not have $N$ independent observations but $G$ clusters
  • eg 10 classrooms of 30 students does not correspond to 300 independent observations
  • The real sample size is closer to the number of clusters, not observations

What clustering does

• Do not affect point estimates
• Allows for intra-cluster correlation and adjusts the effective number of independent observations
• Increases SEs to reflect within-group dependence $\Rightarrow$ wider CIs

$$\widehat{\text{Var}_{CR}} (\widehat{\beta}) = (X'X)^{-1} \left( \dfrac{1}{n-p} \sum_{g \in G} X_g'r_gr_g' X_g \right) (X'X)^{-1}$$ where $X_g$ and $r_g$ are data and residuals for cluster $g$

• Intuition: each cluster provides one independent piece of information about how residuals co-evolve with regressors

Which level to cluster at?

When is clustering difficult to implement?

Bootstrap

Summary of today

Goal of the whole course

Give us a deeper understanding of:

• How regression works “under the hood”: intuition
• Causal identification strategies and their assumptions
• How design, modeling, and analysis choices shape empirical results
• Common pitfalls and challenges in empirical work
• How to use simulations to explore estimator behavior and diagnose potential problems specific to your own cases
• Existing references and where to find additional information on a specific topic

To mention when pitching your analysis

To also mention in your pitch

Summary of the entire course