Lecture 1 - Pitfalls and Simulations Basics

class: right, middle, inverse, title-slide

.title[
# Lecture 1 - Pitfalls and Simulations Basics
]
.subtitle[
## Topics in Econometrics
]
.author[
### Vincent Bagilet
]
.date[
### 2024-09-11
]

---

class: right, middle

# Introduction

???

I introduce myself, they introduce themselves (ask them what they'd like to do later)

---
class: titled, middle

# Structure

- I will teach the 5 first lectures, Sarra Ghaddab will teach the 3 last ones

- General theme for my section: 
 
**Avoiding common pitfalls in applied econometrics research**

- Centered around applied research

- Simulations = cornerstone for this class

---
class: titled, middle

# Outline

1. Introduction and Simulation basics

1. More advanced simulations

1. Low statistical power and exaggeration

1. Design beyond identification

1. Data viz

???

1. Simulation basics: goals, general approach and implementation of simple simulations for regressions in R
1. More advanced simulations: simulations of IV, RDD, DID and their implementation in R
1. Low statistical power and exaggeration: some challenges with the estimation of small effects
1. Multiple goals: how do heterogeneity, a multiplicity of outcomes and external validity may affect design?
1. Data visualization: how can we use data viz to put some of our modeling and identification hypotheses to test?

---
class: right, middle, inverse

# Quick Logistics

---
class: titled, middle

# A typical lecture

1. Introduce concepts and intuition

1. Some R coding together

1. Exercise in R on your own

---
# Website

https://vincentbagilet.github.io/metrics_m2_2024/

</center>

---
class: right, middle, inverse

# R: Why and How?

---
class: titled, middle

# THE statistical analysis software

- Open source

- Can do anything linked to data: wrangling, cleaning

- Huge online community

- Packages for anything

---
# Massive capabilities

### Websites ([Quarto](https://quarto.org/)) and slides ([Quarto](https://quarto.org/), [Xaringan](https://slides.yihui.org/xaringan/#1))

---
### Awesome graphs ([ggplot](https://ggplot2.tidyverse.org/))

.pull-left[
<img src="data:image/png;base64,#images/ggplot_ex_small_multiples.png" width="60%" style="display: block; margin: auto;" />
]
.pull-right[
<img src="data:image/png;base64,#images/rayshader.jpeg" width="2560" />
]

---
### Interactive graphs ([Plotly](https://plotly.com/r/))

<div class="plotly html-widget html-fill-item" id="htmlwidget-35f08b17f0a057f88ca1" style="width:80%;height:504px;"></div>
<script type="application/json" data-for="htmlwidget-35f08b17f0a057f88ca1">{"x":{"data":[{"x":[29,29,31,30,26,26,27,26,25,28,27,25,25,25,25,24,25,23,20,15,20,17,17,26,23,26,25,24,19,14,15,17,27,30,26,29,26,24,24,22,22,24,24,17,22,21,23,23,19,18,17,17,19,19,12,17,15,17,17,12,17,16,18,15,16,12,17,17,16,12,15,16,17,15,17,17,18,17,19,17,19,19,17,17,17,16,16,17,15,17,26,25,26,24,21,22,23,22,20,33,32,32,29,32,34,36,36,29,26,27,30,31,26,26,28,26,29,28,27,24,24,24,22,19,20,17,12,19,18,14,15,18,18,15,17,16,18,17,19,19,17,29,27,31,32,27,26,26,25,25,17,17,20,18,26,26,27,28,25,25,24,27,25,26,23,26,26,26,26,25,27,25,27,20,20,19,17,20,17,29,27,31,31,26,26,28,27,29,31,31,26,26,27,30,33,35,37,35,15,18,20,20,22,17,19,18,20,29,26,29,29,24,44,29,26,29,29,29,29,23,24,44,41,29,26,28,29,29,29,28,29,26,26,26],"y":[1.8,1.8,2,2,2.7999999999999998,2.7999999999999998,3.1000000000000001,1.8,1.8,2,2,2.7999999999999998,2.7999999999999998,3.1000000000000001,3.1000000000000001,2.7999999999999998,3.1000000000000001,4.2000000000000002,5.2999999999999998,5.2999999999999998,5.2999999999999998,5.7000000000000002,6,5.7000000000000002,5.7000000000000002,6.2000000000000002,6.2000000000000002,7,5.2999999999999998,5.2999999999999998,5.7000000000000002,6.5,2.3999999999999999,2.3999999999999999,3.1000000000000001,3.5,3.6000000000000001,2.3999999999999999,3,3.2999999999999998,3.2999999999999998,3.2999999999999998,3.2999999999999998,3.2999999999999998,3.7999999999999998,3.7999999999999998,3.7999999999999998,4,3.7000000000000002,3.7000000000000002,3.8999999999999999,3.8999999999999999,4.7000000000000002,4.7000000000000002,4.7000000000000002,5.2000000000000002,5.2000000000000002,3.8999999999999999,4.7000000000000002,4.7000000000000002,4.7000000000000002,5.2000000000000002,5.7000000000000002,5.9000000000000004,4.7000000000000002,4.7000000000000002,4.7000000000000002,4.7000000000000002,4.7000000000000002,4.7000000000000002,5.2000000000000002,5.2000000000000002,5.7000000000000002,5.9000000000000004,4.5999999999999996,5.4000000000000004,5.4000000000000004,4,4,4,4,4.5999999999999996,5,4.2000000000000002,4.2000000000000002,4.5999999999999996,4.5999999999999996,4.5999999999999996,5.4000000000000004,5.4000000000000004,3.7999999999999998,3.7999999999999998,4,4,4.5999999999999996,4.5999999999999996,4.5999999999999996,4.5999999999999996,5.4000000000000004,1.6000000000000001,1.6000000000000001,1.6000000000000001,1.6000000000000001,1.6000000000000001,1.8,1.8,1.8,2,2.3999999999999999,2.3999999999999999,2.3999999999999999,2.3999999999999999,2.5,2.5,3.2999999999999998,2,2,2,2,2.7000000000000002,2.7000000000000002,2.7000000000000002,3,3.7000000000000002,4,4.7000000000000002,4.7000000000000002,4.7000000000000002,5.7000000000000002,6.0999999999999996,4,4.2000000000000002,4.4000000000000004,4.5999999999999996,5.4000000000000004,5.4000000000000004,5.4000000000000004,4,4,4.5999999999999996,5,2.3999999999999999,2.3999999999999999,2.5,2.5,3.5,3.5,3,3,3.5,3.2999999999999998,3.2999999999999998,4,5.5999999999999996,3.1000000000000001,3.7999999999999998,3.7999999999999998,3.7999999999999998,5.2999999999999998,2.5,2.5,2.5,2.5,2.5,2.5,2.2000000000000002,2.2000000000000002,2.5,2.5,2.5,2.5,2.5,2.5,2.7000000000000002,2.7000000000000002,3.3999999999999999,3.3999999999999999,4,4.7000000000000002,2.2000000000000002,2.2000000000000002,2.3999999999999999,2.3999999999999999,3,3,3.5,2.2000000000000002,2.2000000000000002,2.3999999999999999,2.3999999999999999,3,3,3.2999999999999998,1.8,1.8,1.8,1.8,1.8,4.7000000000000002,5.7000000000000002,2.7000000000000002,2.7000000000000002,2.7000000000000002,3.3999999999999999,3.3999999999999999,4,4,2,2,2,2,2.7999999999999998,1.8999999999999999,2,2,2,2,2.5,2.5,2.7999999999999998,2.7999999999999998,1.8999999999999999,1.8999999999999999,2,2,2.5,2.5,1.8,1.8,2,2,2.7999999999999998,2.7999999999999998,3.6000000000000001],"text":["hwy: 29 displ: 1.8","hwy: 29 displ: 1.8","hwy: 31 displ: 2.0","hwy: 30 displ: 2.0","hwy: 26 displ: 2.8","hwy: 26 displ: 2.8","hwy: 27 displ: 3.1","hwy: 26 displ: 1.8","hwy: 25 displ: 1.8","hwy: 28 displ: 2.0","hwy: 27 displ: 2.0","hwy: 25 displ: 2.8","hwy: 25 displ: 2.8","hwy: 25 displ: 3.1","hwy: 25 displ: 3.1","hwy: 24 displ: 2.8","hwy: 25 displ: 3.1","hwy: 23 displ: 4.2","hwy: 20 displ: 5.3","hwy: 15 displ: 5.3","hwy: 20 displ: 5.3","hwy: 17 displ: 5.7","hwy: 17 displ: 6.0","hwy: 26 displ: 5.7","hwy: 23 displ: 5.7","hwy: 26 displ: 6.2","hwy: 25 displ: 6.2","hwy: 24 displ: 7.0","hwy: 19 displ: 5.3","hwy: 14 displ: 5.3","hwy: 15 displ: 5.7","hwy: 17 displ: 6.5","hwy: 27 displ: 2.4","hwy: 30 displ: 2.4","hwy: 26 displ: 3.1","hwy: 29 displ: 3.5","hwy: 26 displ: 3.6","hwy: 24 displ: 2.4","hwy: 24 displ: 3.0","hwy: 22 displ: 3.3","hwy: 22 displ: 3.3","hwy: 24 displ: 3.3","hwy: 24 displ: 3.3","hwy: 17 displ: 3.3","hwy: 22 displ: 3.8","hwy: 21 displ: 3.8","hwy: 23 displ: 3.8","hwy: 23 displ: 4.0","hwy: 19 displ: 3.7","hwy: 18 displ: 3.7","hwy: 17 displ: 3.9","hwy: 17 displ: 3.9","hwy: 19 displ: 4.7","hwy: 19 displ: 4.7","hwy: 12 displ: 4.7","hwy: 17 displ: 5.2","hwy: 15 displ: 5.2","hwy: 17 displ: 3.9","hwy: 17 displ: 4.7","hwy: 12 displ: 4.7","hwy: 17 displ: 4.7","hwy: 16 displ: 5.2","hwy: 18 displ: 5.7","hwy: 15 displ: 5.9","hwy: 16 displ: 4.7","hwy: 12 displ: 4.7","hwy: 17 displ: 4.7","hwy: 17 displ: 4.7","hwy: 16 displ: 4.7","hwy: 12 displ: 4.7","hwy: 15 displ: 5.2","hwy: 16 displ: 5.2","hwy: 17 displ: 5.7","hwy: 15 displ: 5.9","hwy: 17 displ: 4.6","hwy: 17 displ: 5.4","hwy: 18 displ: 5.4","hwy: 17 displ: 4.0","hwy: 19 displ: 4.0","hwy: 17 displ: 4.0","hwy: 19 displ: 4.0","hwy: 19 displ: 4.6","hwy: 17 displ: 5.0","hwy: 17 displ: 4.2","hwy: 17 displ: 4.2","hwy: 16 displ: 4.6","hwy: 16 displ: 4.6","hwy: 17 displ: 4.6","hwy: 15 displ: 5.4","hwy: 17 displ: 5.4","hwy: 26 displ: 3.8","hwy: 25 displ: 3.8","hwy: 26 displ: 4.0","hwy: 24 displ: 4.0","hwy: 21 displ: 4.6","hwy: 22 displ: 4.6","hwy: 23 displ: 4.6","hwy: 22 displ: 4.6","hwy: 20 displ: 5.4","hwy: 33 displ: 1.6","hwy: 32 displ: 1.6","hwy: 32 displ: 1.6","hwy: 29 displ: 1.6","hwy: 32 displ: 1.6","hwy: 34 displ: 1.8","hwy: 36 displ: 1.8","hwy: 36 displ: 1.8","hwy: 29 displ: 2.0","hwy: 26 displ: 2.4","hwy: 27 displ: 2.4","hwy: 30 displ: 2.4","hwy: 31 displ: 2.4","hwy: 26 displ: 2.5","hwy: 26 displ: 2.5","hwy: 28 displ: 3.3","hwy: 26 displ: 2.0","hwy: 29 displ: 2.0","hwy: 28 displ: 2.0","hwy: 27 displ: 2.0","hwy: 24 displ: 2.7","hwy: 24 displ: 2.7","hwy: 24 displ: 2.7","hwy: 22 displ: 3.0","hwy: 19 displ: 3.7","hwy: 20 displ: 4.0","hwy: 17 displ: 4.7","hwy: 12 displ: 4.7","hwy: 19 displ: 4.7","hwy: 18 displ: 5.7","hwy: 14 displ: 6.1","hwy: 15 displ: 4.0","hwy: 18 displ: 4.2","hwy: 18 displ: 4.4","hwy: 15 displ: 4.6","hwy: 17 displ: 5.4","hwy: 16 displ: 5.4","hwy: 18 displ: 5.4","hwy: 17 displ: 4.0","hwy: 19 displ: 4.0","hwy: 19 displ: 4.6","hwy: 17 displ: 5.0","hwy: 29 displ: 2.4","hwy: 27 displ: 2.4","hwy: 31 displ: 2.5","hwy: 32 displ: 2.5","hwy: 27 displ: 3.5","hwy: 26 displ: 3.5","hwy: 26 displ: 3.0","hwy: 25 displ: 3.0","hwy: 25 displ: 3.5","hwy: 17 displ: 3.3","hwy: 17 displ: 3.3","hwy: 20 displ: 4.0","hwy: 18 displ: 5.6","hwy: 26 displ: 3.1","hwy: 26 displ: 3.8","hwy: 27 displ: 3.8","hwy: 28 displ: 3.8","hwy: 25 displ: 5.3","hwy: 25 displ: 2.5","hwy: 24 displ: 2.5","hwy: 27 displ: 2.5","hwy: 25 displ: 2.5","hwy: 26 displ: 2.5","hwy: 23 displ: 2.5","hwy: 26 displ: 2.2","hwy: 26 displ: 2.2","hwy: 26 displ: 2.5","hwy: 26 displ: 2.5","hwy: 25 displ: 2.5","hwy: 27 displ: 2.5","hwy: 25 displ: 2.5","hwy: 27 displ: 2.5","hwy: 20 displ: 2.7","hwy: 20 displ: 2.7","hwy: 19 displ: 3.4","hwy: 17 displ: 3.4","hwy: 20 displ: 4.0","hwy: 17 displ: 4.7","hwy: 29 displ: 2.2","hwy: 27 displ: 2.2","hwy: 31 displ: 2.4","hwy: 31 displ: 2.4","hwy: 26 displ: 3.0","hwy: 26 displ: 3.0","hwy: 28 displ: 3.5","hwy: 27 displ: 2.2","hwy: 29 displ: 2.2","hwy: 31 displ: 2.4","hwy: 31 displ: 2.4","hwy: 26 displ: 3.0","hwy: 26 displ: 3.0","hwy: 27 displ: 3.3","hwy: 30 displ: 1.8","hwy: 33 displ: 1.8","hwy: 35 displ: 1.8","hwy: 37 displ: 1.8","hwy: 35 displ: 1.8","hwy: 15 displ: 4.7","hwy: 18 displ: 5.7","hwy: 20 displ: 2.7","hwy: 20 displ: 2.7","hwy: 22 displ: 2.7","hwy: 17 displ: 3.4","hwy: 19 displ: 3.4","hwy: 18 displ: 4.0","hwy: 20 displ: 4.0","hwy: 29 displ: 2.0","hwy: 26 displ: 2.0","hwy: 29 displ: 2.0","hwy: 29 displ: 2.0","hwy: 24 displ: 2.8","hwy: 44 displ: 1.9","hwy: 29 displ: 2.0","hwy: 26 displ: 2.0","hwy: 29 displ: 2.0","hwy: 29 displ: 2.0","hwy: 29 displ: 2.5","hwy: 29 displ: 2.5","hwy: 23 displ: 2.8","hwy: 24 displ: 2.8","hwy: 44 displ: 1.9","hwy: 41 displ: 1.9","hwy: 29 displ: 2.0","hwy: 26 displ: 2.0","hwy: 28 displ: 2.5","hwy: 29 displ: 2.5","hwy: 29 displ: 1.8","hwy: 29 displ: 1.8","hwy: 28 displ: 2.0","hwy: 29 displ: 2.0","hwy: 26 displ: 2.8","hwy: 26 displ: 2.8","hwy: 26 displ: 3.6"],"type":"scatter","mode":"markers","marker":{"autocolorscale":false,"color":"rgba(11,43,81,1)","opacity":1,"size":5.6692913385826778,"symbol":"circle","line":{"width":1.8897637795275593,"color":"rgba(11,43,81,1)"}},"hoveron":"points","showlegend":false,"xaxis":"x","yaxis":"y","hoverinfo":"text","frame":null}],"layout":{"margin":{"t":34.897637795275585,"r":18.897637795275585,"b":56.091494167214165,"l":49.117646097475671},"paper_bgcolor":"rgba(254,254,241,1)","font":{"color":"rgba(6,22,41,1)","family":"Lato","size":18.596928185969286},"xaxis":{"domain":[0,1],"automargin":true,"type":"linear","autorange":false,"range":[10.4,45.600000000000001],"tickmode":"array","ticktext":["20","30","40"],"tickvals":[20,30,40],"categoryorder":"array","categoryarray":["20","30","40"],"nticks":null,"ticks":"outside","tickcolor":"rgba(6,22,41,1)","ticklen":4.6492320464923216,"tickwidth":0.74387712743877132,"showticklabels":true,"tickfont":{"color":"rgba(6,22,41,1)","family":"Lato","size":13.947696139476964},"tickangle":-0,"showline":false,"linecolor":null,"linewidth":0,"showgrid":false,"gridcolor":null,"gridwidth":0,"zeroline":false,"anchor":"y","title":{"text":"hwy","font":{"color":"rgba(6,22,41,1)","family":"Lato","size":18.596928185969286}},"hoverformat":".2f"},"yaxis":{"domain":[0,1],"automargin":true,"type":"linear","autorange":false,"range":[1.3300000000000001,7.2699999999999996],"tickmode":"array","ticktext":["2","3","4","5","6","7"],"tickvals":[2,3,4,5,6,7],"categoryorder":"array","categoryarray":["2","3","4","5","6","7"],"nticks":null,"ticks":"","tickcolor":null,"ticklen":4.6492320464923216,"tickwidth":0,"showticklabels":true,"tickfont":{"color":"rgba(6,22,41,1)","family":"Lato","size":13.947696139476964},"tickangle":-0,"showline":false,"linecolor":null,"linewidth":0,"showgrid":true,"gridcolor":"rgba(199,203,207,1)","gridwidth":0.37193856371938566,"zeroline":false,"anchor":"x","title":{"text":"displ","font":{"color":"rgba(6,22,41,1)","family":"Lato","size":18.596928185969286}},"hoverformat":".2f"},"shapes":[{"type":"rect","fillcolor":null,"line":{"color":null,"width":0,"linetype":[]},"yref":"paper","xref":"paper","x0":0,"x1":1,"y0":0,"y1":1}],"showlegend":false,"legend":{"bgcolor":null,"bordercolor":null,"borderwidth":0,"font":{"color":"rgba(6,22,41,1)","family":"Lato","size":14.505603985056041}},"hovermode":"closest","barmode":"relative","plot_bgcolor":"transparent"},"config":{"doubleClick":"reset","modeBarButtonsToAdd":["hoverclosest","hovercompare"],"showSendToCloud":false},"source":"A","attrs":{"107932a3eb768":{"x":{},"y":{},"type":"scatter"}},"cur_data":"107932a3eb768","visdat":{"107932a3eb768":["function (y) ","x"]},"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.20000000000000001,"selected":{"opacity":1},"debounce":0},"shinyEvents":["plotly_hover","plotly_click","plotly_selected","plotly_relayout","plotly_brushed","plotly_brushing","plotly_clickannotation","plotly_doubleclick","plotly_deselect","plotly_afterplot","plotly_sunburstclick"],"base_url":"https://plot.ly"},"evals":[],"jsHooks":[]}</script>

---
### Interactive maps ([Leaflet](https://rstudio.github.io/leaflet/) and [Mapview](https://r-spatial.github.io/mapview/))

<div class="leaflet html-widget html-fill-item" id="htmlwidget-d6756dbd4e6fd9ff3b3e" style="width:100%;height:504px;"></div>
<script type="application/json" data-for="htmlwidget-d6756dbd4e6fd9ff3b3e">{"x":{"options":{"minZoom":1,"maxZoom":52,"crs":{"crsClass":"L.CRS.EPSG3857","code":null,"proj4def":null,"projectedBounds":null,"options":{}},"preferCanvas":false,"bounceAtZoomLimits":false,"maxBounds":[[[-90,-370]],[[90,370]]]},"calls":[{"method":"addProviderTiles","args":["CartoDB.Positron","CartoDB.Positron","CartoDB.Positron",{"errorTileUrl":"","noWrap":false,"detectRetina":false,"pane":"tilePane"}]},{"method":"addProviderTiles","args":["CartoDB.DarkMatter","CartoDB.DarkMatter","CartoDB.DarkMatter",{"errorTileUrl":"","noWrap":false,"detectRetina":false,"pane":"tilePane"}]},{"method":"addProviderTiles","args":["OpenStreetMap","OpenStreetMap","OpenStreetMap",{"errorTileUrl":"","noWrap":false,"detectRetina":false,"pane":"tilePane"}]},{"method":"addProviderTiles","args":["Esri.WorldImagery","Esri.WorldImagery","Esri.WorldImagery",{"errorTileUrl":"","noWrap":false,"detectRetina":false,"pane":"tilePane"}]},{"method":"addProviderTiles","args":["OpenTopoMap","OpenTopoMap","OpenTopoMap",{"errorTileUrl":"","noWrap":false,"detectRetina":false,"pane":"tilePane"}]},{"method":"createMapPane","args":["point",440]},{"method":"addCircleMarkers","args":[45.73360071410955,4.833428242327759,6,null,"ENS",{"crs":{"crsClass":"L.CRS.EPSG3857","code":null,"proj4def":null,"projectedBounds":null,"options":{}},"pane":"point","stroke":true,"color":"#333333","weight":1,"opacity":0.9,"fill":true,"fillColor":"#6666FF","fillOpacity":0.6},null,null,"<div class='scrollableContainer'><table class=mapview-popup id='popup'><tr class='coord'><td><\/td><th>Feature ID <\/b><\/th><td>1 <\/td><\/tr><tr><td>1<\/td><th>geometry <\/th><td>sfc_POINT <\/td><\/tr><\/table><\/div>",{"maxWidth":800,"minWidth":50,"autoPan":true,"keepInView":false,"closeButton":true,"closeOnClick":true,"className":""},"1",{"interactive":false,"permanent":false,"direction":"auto","opacity":1,"offset":[0,0],"textsize":"10px","textOnly":false,"className":"","sticky":true},null]},{"method":"addScaleBar","args":[{"maxWidth":100,"metric":true,"imperial":true,"updateWhenIdle":true,"position":"bottomleft"}]},{"method":"addHomeButton","args":[4.833428242327759,45.73360071410955,4.833428242327759,45.73360071410955,true,"ENS","Zoom to ENS"," ENS <\/strong>","bottomright"]},{"method":"addLayersControl","args":[["CartoDB.Positron","CartoDB.DarkMatter","OpenStreetMap","Esri.WorldImagery","OpenTopoMap"],"ENS",{"collapsed":true,"autoZIndex":true,"position":"topleft"}]},{"method":"addLegend","args":[{"colors":["#6666FF"],"labels":["ENS"],"na_color":null,"na_label":"NA","opacity":1,"position":"topright","type":"factor","title":"","extra":null,"layerId":null,"className":"info legend","group":"ENS"}]}],"limits":{"lat":[45.73360071410955,45.73360071410955],"lng":[4.833428242327759,4.833428242327759]},"setView":[[45.73360071410955,4.833428242327759],18,[]]},"evals":[],"jsHooks":{"render":[{"code":"function(el, x, data) {\n return (\n function(el, x, data) {\n // get the leaflet map\n var map = this; //HTMLWidgets.find('#' + el.id);\n // we need a new div element because we have to handle\n // the mouseover output separately\n // debugger;\n function addElement () {\n // generate new div Element\n var newDiv = $(document.createElement('div'));\n // append at end of leaflet htmlwidget container\n $(el).append(newDiv);\n //provide ID and style\n newDiv.addClass('lnlt');\n newDiv.css({\n 'position': 'relative',\n 'bottomleft': '0px',\n 'background-color': 'rgba(255, 255, 255, 0.7)',\n 'box-shadow': '0 0 2px #bbb',\n 'background-clip': 'padding-box',\n 'margin': '0',\n 'padding-left': '5px',\n 'color': '#333',\n 'font': '9px/1.5 \"Helvetica Neue\", Arial, Helvetica, sans-serif',\n 'z-index': '700',\n });\n return newDiv;\n }\n\n\n // check for already existing lnlt class to not duplicate\n var lnlt = $(el).find('.lnlt');\n\n if(!lnlt.length) {\n lnlt = addElement();\n\n // grab the special div we generated in the beginning\n // and put the mousmove output there\n\n map.on('mousemove', function (e) {\n if (e.originalEvent.ctrlKey) {\n if (document.querySelector('.lnlt') === null) lnlt = addElement();\n lnlt.text(\n ' lon: ' + (e.latlng.lng).toFixed(5) +\n ' | lat: ' + (e.latlng.lat).toFixed(5) +\n ' | zoom: ' + map.getZoom() +\n ' | x: ' + L.CRS.EPSG3857.project(e.latlng).x.toFixed(0) +\n ' | y: ' + L.CRS.EPSG3857.project(e.latlng).y.toFixed(0) +\n ' | epsg: 3857 ' +\n ' | proj4: +proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs ');\n } else {\n if (document.querySelector('.lnlt') === null) lnlt = addElement();\n lnlt.text(\n ' lon: ' + (e.latlng.lng).toFixed(5) +\n ' | lat: ' + (e.latlng.lat).toFixed(5) +\n ' | zoom: ' + map.getZoom() + ' ');\n }\n });\n\n // remove the lnlt div when mouse leaves map\n map.on('mouseout', function (e) {\n var strip = document.querySelector('.lnlt');\n if( strip !==null) strip.remove();\n });\n\n };\n\n //$(el).keypress(67, function(e) {\n map.on('preclick', function(e) {\n if (e.originalEvent.ctrlKey) {\n if (document.querySelector('.lnlt') === null) lnlt = addElement();\n lnlt.text(\n ' lon: ' + (e.latlng.lng).toFixed(5) +\n ' | lat: ' + (e.latlng.lat).toFixed(5) +\n ' | zoom: ' + map.getZoom() + ' ');\n var txt = document.querySelector('.lnlt').textContent;\n console.log(txt);\n //txt.innerText.focus();\n //txt.select();\n setClipboardText('\"' + txt + '\"');\n }\n });\n\n }\n ).call(this.getMap(), el, x, data);\n}","data":null},{"code":"function(el, x, data) {\n return (function(el,x,data){\n var map = this;\n\n map.on('keypress', function(e) {\n console.log(e.originalEvent.code);\n var key = e.originalEvent.code;\n if (key === 'KeyE') {\n var bb = this.getBounds();\n var txt = JSON.stringify(bb);\n console.log(txt);\n\n setClipboardText('\\'' + txt + '\\'');\n }\n })\n }).call(this.getMap(), el, x, data);\n}","data":null}]}}</script>

---
### Interactive apps ([Shiny](https://shiny.posit.co/))

---
layout: true

# Literate programming

---

- Combine code and natural language

---

- R Package: [Quarto](https://quarto.org/)

- We will use this in this class

- Helpful for economic research. Allows to:

- Clearly describe **why** your are doing what you are doing
  
  - Store details and information for future-self (data sources, data structure, etc)
  
  - Analyse your results
  
  - Communicate

---
class: titled, middle
layout: false

.pull-left[
<img src="data:image/png;base64,#images/r4ds.jpg" width="450" style="display: block; margin: auto;" />
]
.pull-right[
 
- Install R and RStudio.

- Instructions [here](https://r4ds.hadley.nz/intro#prerequisites).
]

---
class: right, middle, inverse

# Potential pitfalls

---
# Goals and pitfalls of econ research

- **Goal**: inform a theory (answer a "why", to some extent)

- In applied research and causal inference: estimate the effect of one factor on another

- We want **accurate estimates** (*eg* because inform public policy)

- But potential **pitfalls**, *eg*:
--
.pull-left[
  - Spurious correlation
  
- Reverse causality
  
- Confounders
  
]
.pull-right[
- Model miss-specification

- External validity

- Not detecting the effect

]

???

- Spurious correlation: that's in part why theory matter

- Examples of estimates informing policy: 
  - Impact of a public policy on an outcome (eg Social Cost of Carbon)
  - If want to choose between different policies which one to implement, etc

---
## Spurious correlation

---
## Reverse causality

---
layout: true

## Confounders

---

[https://forms.gle/tHcTYqaKPeDTAUn56](https://forms.gle/tHcTYqaKPeDTAUn56)

---

---

---

---

---
layout: false

## Model miss-specification

---
# Where can we face pitfalls?

### Steps of applied economics analyses

.pull-left[
1 Define question/topic

2 Find, get, wrangle and clean data

3 Summary statistics

4 Define an identification strategy
]
.pull-right[

5 Make a regression model

6 Estimation

7 Specification checks

8 Additional inference
]

???

- I need a running example here. My MA thesis on gasoline prices?

- Here, we will mostly talk about 2, 3, 4, 5 and 8 
- 1: I am not the best for this
- 4: You had a class on this last year, that's fun, that's also very much discussed in economics. Same for 7
- 6: You had a lot of classes and there are a lot of resources on this. That's also very much discussed in economics but clearly, not fun

- 5: Talked about it last year but will talk more about it this year

- For other points, pitfalls less discussed

---
class: right, middle, inverse

# Avoiding pitfalls

---
# How to avoid pitfalls?

### Way before implementing the study

**Learn**, understand metrics and applied research

???

- Seriously, that can be helpful to avoid doing some wrong stuff
- Knowing that some things do not work, etc

---
class: center
 
<a href="https://en.wikipedia.org/wiki/The_Barque_of_Dante_(Manet)" target="_blank"><img src="data:image/png;base64,#images/manet_delacroix.jpg" width="580" /></a>

*The Barque of Dante* by Manet, after a painting by Delacroix

**Replication**, a helpful learning tool

???

- La barque de Dante d'après Eugène Delacroix par Edouard Manet

---
layout: true

# How to avoid pitfalls?

---

### Just ahead of the study

Think about and evaluate its **design**:

- Identification strategy and related hypotheses

--
  
- But NOT ONLY!
  
- **Design broader than identification**
  
  - Magnitude of the effect
  
  - Practical significance
  
  - Precision and statistical power
  
  
???

- Id strat: the fun part! Think about potential confounders etc. But do not only think about this 
  
---
### During the study

.pull-left[
- Check if the model seems to represent the DGP

- Check if our identification hypotheses seem to hold

- Check if the hypotheses for estimation seem to hold
]

.pull-right[
 
Look at the consequences if this does not hold
]

-------

- Robustness checks

- Evaluate the design retrospectively

???

- Think about: identification strategy, but other aspects as well
- We will study these other

---
class: titled, middle
layout: false

# Overall, how to detect risks of pitfalls?

- Think

- Maths

- Data viz

- Simulations

???

- You can think about your identification straategy, potential risks of confounders, etc. 
- If want to study

---
class: titled, middle

# Objectives for this class

- Build a mindful mindset

- Learn how to identify and test for potential pitfalls

- Learn how to implement simulations

- Learn something that may not be often discussed but that is really helpful

???

- Sim: learn how to implement them but also learn about their usefulness
- Objective for me (beyond helping you learn important material): develop one of my ongoing research articles

---
class: right, middle, inverse

# Simulations

## Usefulness through an example

---
class: titled, middle

# Idea behind simulations

1. You define a (fake) true effect

1. Generate data

1. Pretend you do not know how it was generated

1. Evaluate the ability of your analysis to **retrieve this true effect**

1. If it does not, it probably will not either on actual data

1. Identify the limiting factors and issues

---
class: titled, middle

# A simple example

### Setting

- Impact of receiving extra lessons on students’ grades

- Simulate an experiment (RCT)

- Which sample size and proportion of treated to have a high probability of detecting the effect?

### How?

- Simulate many experiments

- Compute the proportion of effects detected

---
class: titled, middle

# Data Generating Process (DGP)

`$$Grade_i = \alpha_0 + \beta_0 Treat_i + u_i$$`

Implicit assumptions in this DGP?

???

- No confounders
- Linear treatment

---
class: right, middle

# Switch to Quarto document for coding

---
class: titled, middle

# Summary of the simulation approach

1. Define a DGP and the distribution of variables

1. Set parameters values

1. Generate a data set

1. Estimate the effect in the generated data set

1. Repeat many times

1. Compute the measure of interest (here the statistical power)

---
class: right, middle, inverse

# Lecture summary

---
class: titled, middle

# What did we learn today?

- Learned about or reviewed some common pitfalls encountered in applied research

- How to implement a simple simulation

---
class: titled, middle

## What did you learned, liked, disliked?

???

- First class here, with you
- First time teaching my own material
- First time teaching something like this
- I want to have your opinion for next time
- Was it too simple, too complex?

---
class: titled, middle

# Side quests

- Make you enjoy it a bit and maybe have a bit of fun coding

- Learn some new R stuff

- Usefulness of literate programming

- Hint that desing matters beyond identification

---
class: right, middle, inverse

# Thank you!