2 Exchangeability and experiments

2.1 Randomized experiments

Sep 1. Slides. After class, read Hernán and Robins 2020 Chapter 2 through the end of 2.1.

Much of this course will address observational studies with non-randomized treatments. To set the stage, today we will first discuss why randomized experiments are so powerful when they are possible.

2.2 Lab: Analyze a randomized experiment

Sep 2. Discussion

This lab will use R to analyze data from a randomized experiment estimating the affect of generative AI on acquiring new skills (Bastani et al. 2025). Download the R Markdown file here.

2.3 Exchangeability and conditional randomization

Sep 3. Slides. After class, read Hernán and Robins 2020 Chapter 2.2 & 2.3.

We continue discussing why experiments are powerful: they are a setting in which a key identification assumption (exchangeability) holds by design. We further discuss why exchangeability is important: it allows us to link causal quantities to observable data. We discuss exchangeability in simple randomized experiments and in experiments that are conditionally randomized so that treatment assignment probabilities are functions of pre-existing characteristics.

2.4 Standardization

Sep 8. Slides After class, read Hernán and Robins 2020 Chapter 2.3 and 4.1-4.3.

Although we can use the ATE to describe the average over the entire population, the treatment effect may vary across sub-populations. This is called treatment effect heterogeneity and is an important consideration when making policy decisions.

Stratification allows us to estimate the average causal effect within a subpopulation, or strata, also known as the conditional average treatment effect. Standardization is an important statistical procedure that allows us to estimate the population average treatment effect by taking a weighted average over the subpopulations.

In conditionally randomized experiments, standardization is essential to yield unbiased estimates of the population average causal effect. This strategy will also be essential in observational studies that we will discuss soon.

2.5 Lab: Treatment Effect Heterogeneity and Asking Causal Questions

Sep 9 Discussion and discussion slides

This lab will use the randomized experiment (Gerber, Green, & Larimer 2008) to explore the treatment effect across sub-population, to determine whether there is treatment effect heterogeneity. The slides layout an overview of the course project, explore the Add Health Codebook Explorer, and go through the in-class assignment of asking a causal question.

2.6 Inverse probability weighting

Sep 10. Slides After class, read Hernán and Robins 2020 Chapters 3.1 and 3.2.

This class will introduce inverse probability weighting as an approach to estimate average causal effects when conditional exchangeability holds. We will also discuss alternative ways to measure a causal effect. Finally, we will discuss when conditional exchangeability might also hold in observational studies.