# Discussion 1. Prob & Stats Review
## STSCI/INFO/ILRST 3900: Causal Inference 
#### August 27, 2025

```{r echo=FALSE}
library(ggplot2)
library(tibble)

set.seed(1234)
```


### Probability and Statistics Review
- Expectation
- Variance
- Conditional Expectation 
- Independence
- Bernoulli Random Variables
- Law of Total Expectation
- Confidence Intervals
- Regression (OLS, logistic)


#### 1. Expectation
##### (Expected Value, Population Mean, Average)

- Notation: $E(X), \mu$
- The expected value of a finite random variable
  $$\mu=E(X) := \sum_{i=1}^N x_i P(x_i) \quad \text{where} \quad P(x_i):=\text{prob}(X=x_i)$$
- The expected value of a countable random variable, i.e. the (long run) average
  $$E(X):= \sum_{i=1}^\infty x_i P(x_i)$$
- For $n$ independent and identically distributed (i.i.d.) random variables $X_1,\ldots,X_N$
the sample mean is
$$\bar X = \frac{1}{N}\sum_{i=1}^N X_i$$
- Law of Large Numbers (LLN): the sample mean converges to the expected value (population mean) as $N \to \infty$
- Example: $X_i$ are random draws from $\sim \mathcal{N}(2,5)$ (a Normal r.v. with mean 2, variance 5)

```{r, warning=FALSE, fig.dim=c(4, 3), fig.align='center'}
true_mean <- 2
true_var <- 5

x <- seq(-8, 12 , length=1000)
y <- dnorm(x, mean=true_mean, sd=sqrt(true_var))

ggplot() + geom_line(aes(x,y)) + 
    geom_vline(xintercept = true_mean, color = "red") +
    theme_bw() +
    labs(y="Density")


sample_seq <- 1:3000
means <- numeric(length(sample_seq))
vars <- numeric(length(sample_seq))


for(i in 1:length(sample_seq)){
  n <- sample_seq[i]
  data <- rnorm(n = n, mean = true_mean, sd = sqrt(true_var))
  
  sample_mean <- mean(data)
  sample_var <- sum((data - sample_mean)^2)/length(data)
  
  means[i] <- sample_mean
  vars[i] <- sample_var
}


means <- tibble("N" = sample_seq, "Sample Mean" = means)
vars <- tibble("N" = sample_seq, "Sample Variance" = vars)

colors <- c("Sample Mean" = "lightblue", "Population Mean" = "red")

ggplot(means, aes(y = `Sample Mean`, x = N)) +
  geom_line(color = "lightblue") +
  geom_abline(slope = 0, intercept = true_mean, color = "red") +
  theme_bw()

```


#### 2. Variance
##### Describes the spread of the data

- Notation: $V(X), Var(X),\sigma^2$
- Variance is the average of the squared differences from the mean
- For a random variable $X$ with expected value $\mu:=E(X)$, the variance is
  $$\sigma^2 = Var(X) := E\Big[\big(X-\mu\big)^2\Big] = E\big[X^2\big] - \mu^2$$
  More explicitly
  $$Var(X) = \sum_{i=1}^n P(x_i)\cdot (x_i-\mu)^2 \quad \text{where} \quad P(x_i):=\text{prob}(X=x_i)$$

#### 3. Sample (Empirical) Variance
##### For a finite dataset or finite sample

- In practice, you can compute the variance of a finite dataset as
  $$\sigma^2 = \Big(\frac{1}{N}\sum_{i=1}^N x_i^2\Big)-\bar{X}^2 \quad \text{where} \quad \bar{X} := \frac{1}{N}\sum_{i=1}^N x_i$$ 
- You don’t need to have the formula memorized, just be aware of it
- Likely you’ll never have to explicitly compute it this way, just use an R function
- Example: $X_i$ are random draws from $\sim \mathcal{N}(2,5)$ (a Normal r.v. with mean 2, variance 5)


```{r, warning=FALSE, fig.dim=c(4, 3), fig.align='center'}
ggplot(vars, aes(y = `Sample Variance`, x = N)) +
  geom_line(color = "lightblue") +
  geom_abline(slope = 0, intercept = true_var, color = "red") +
  theme_bw()
```


#### 4. Conditional Expectation

- Notation: $E(X|Y)$
- The expected value given a set of “conditions” 
- Read as “the expectation of $X$ given (or conditioned on) $Y$” 
  $$E(X|Y) = \sum_{i=1}^n x_i \cdot P(X=x_i | Y) \quad \text{where} \quad P(X=x_i|Y) = \frac{P(X=x_i \text{ and } Y)}{P(Y)}$$
- Example: Roll a fair dice
  - Let $A=1$ if you roll an even number, $0$ otherwise
  - Let $B=1$ if you roll a prime number,  $0$ otherwise
  - Then
    $$E[A] = \sum_{i=1}^6 a_i\cdot P(a_i) = \frac{0+1+0+1+0+1}{6} = \frac{1}{2}$$
    and the conditional expectation of $A$ given $B=1$ (i.e. we rolled 2, 3, or 5)
    $$E[A | B=1]= \sum_{i=1}^3 a_i\cdot P(a_i|B=1) = \frac{1 + 0 + 0}{3}= \frac{1}{3}$$
- Visualization in R for $E(X)=25$, $E[X| \text{group 1}] = 20$, $E[X| \text{group 2}] = 30$


```{r, warning=FALSE, fig.dim=c(5, 3), fig.align='center', echo=FALSE}
group1_means <- rnorm(100, mean = 20, sd = 5)
group2_means <- rnorm(100, mean = 30, sd = 5)
group_means <- data.frame(
  "Group" = c(rep("Group 1", 100), rep("Group 2", 100)),
  "Values" = c(group1_means, group2_means),
  "x" = rnorm(200, 5, sd = 3)
)
ggplot(group_means, aes(x = x, y = Values, color = Group)) +
  geom_point() +
  geom_abline(
    slope = 0,
    intercept = mean(group_means$Values),
    show.legend = TRUE,
    color = "gray30"
  ) +
  geom_abline(
    slope = 0,
    intercept = mean(group_means[group_means$Group == "Group 1", ]$Values),
    show.legend = TRUE,
    color = "#F8766D"
  ) +
  geom_abline(
    slope = 0,
    intercept = mean(group_means[group_means$Group == "Group 2", ]$Values),
    show.legend = TRUE,
    color = "#00BFC4"
  ) +
  theme_bw()
```
