Homework 3 Solutions

Chapter 12, Coding Question 6

Part (a)

In class, we derived that \[\begin{align*} \hat{\beta}_0 = \bar{Y} - \bar{X} \hat{\beta}_1 \quad \text{ and } \quad \hat{\beta}_1 = \frac{\widehat{\mathrm{cov}}(X,Y)}{\widehat{\mathrm{var}(X)}}, \end{align*}\] which we can use to answer this question.

data(Caschool, package="Ecdat")
Y <- Caschool$testscr
X <- Caschool$str
barY <- mean(Y)
barX <- mean(X)
beta1 <- cov(X,Y) / var(X)
beta0 <- barY - barX * beta1

# show results
round(data.frame(beta0=beta0, beta1=beta1),3)
    beta0 beta1
1 698.933 -2.28
# confirm that we get the same results with `lm`
reg <- lm(testscr ~ str, data=Caschool)
summary(reg)

Call:
lm(formula = testscr ~ str, data = Caschool)

Residuals:
    Min      1Q  Median      3Q     Max 
-47.727 -14.251   0.483  12.822  48.540 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 698.9330     9.4675  73.825  < 2e-16 ***
str          -2.2798     0.4798  -4.751 2.78e-06 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 18.58 on 418 degrees of freedom
Multiple R-squared:  0.05124,   Adjusted R-squared:  0.04897 
F-statistic: 22.58 on 1 and 418 DF,  p-value: 2.783e-06

Part (b)

In class, we showed that \[\begin{align*} \sqrt{n}(\hat{\beta}_1 - \beta_1) \rightarrow \mathcal{N}(0,V) \end{align*}\] where \[\begin{align*} V = \frac{\mathbb{E}\big[(X-\mathbb{E}[X])^2 U^2\big]}{\mathrm{var}(X)^2}, \end{align*}\] which we can estimate using \[\begin{align*} \hat{V} = \dfrac{\displaystyle \frac{1}{n} \sum_{i=1}^n (X_i - \bar{X})^2 \hat{U}_i^2}{\widehat{\mathrm{var}}(X)} \end{align*}\] and then \[\begin{align*} \text{se}(\hat{\beta}_1) = \frac{\sqrt{\hat{V}}}{\sqrt{n}} \end{align*}\] We just need to calculate \(\text{se}(\hat{\beta}_1)\) by applying these formulas to the data that we have.

# calculate residual
Uhat <- Y - beta0 - X * beta1

varX <- mean( (X - barX)^2 )
Vhat <- mean( (X - barX)^2 * Uhat^2 ) / ( varX^2 )
n <- nrow(Caschool)
se <- sqrt(Vhat) / sqrt(n)

# report standard error
round(data.frame(se=se), 3)
     se
1 0.518

Compare to what we get using estimatr::lm_robust

library(estimatr)
regb <- lm_robust(testscr ~ str, data=Caschool, se_type="HC0")
summary(regb)

Call:
lm_robust(formula = testscr ~ str, data = Caschool, se_type = "HC0")

Standard error type:  HC0 

Coefficients:
            Estimate Std. Error t value   Pr(>|t|) CI Lower CI Upper  DF
(Intercept)   698.93    10.3397  67.597 3.803e-227  678.609  719.257 418
str            -2.28     0.5183  -4.399  1.382e-05   -3.299   -1.261 418

Multiple R-squared:  0.05124 ,  Adjusted R-squared:  0.04897 
F-statistic: 19.35 on 1 and 418 DF,  p-value: 1.382e-05

Part (c)

Recall from class that \[\begin{align*} \text{p-val} = 2 \Phi\big(-|t|\big) \end{align*}\] which we can calculate as follows

# calclute t-statistic and p-value
t_stat <- beta1 / se
p_val <- 2 * pnorm(-abs(t_stat))

# print p-value
data.frame(p_val=p_val)
        p_val
1 1.08729e-05

which is very close to 0.

Part (d)

Recall from class that a 95% confidence interval is given by

\[\begin{align*} CI = [\hat{\beta}_1 - 1.96 \text{se}(\hat{\beta}_1), \ \hat{\beta}_1 + 1.96 \text{se}(\hat{\beta}_1)] \end{align*}\]

which we can calculate as follows

ci_L <- beta1 - 1.96*se
ci_U <- beta1 + 1.96*se

# print confidence interval
round(data.frame(ci_L=ci_L, ci_U=ci_U), 3)
    ci_L   ci_U
1 -3.296 -1.264

Chapter 19, Coding Question 1

load("../data/rand_hie.RData")
rand_hie_subset <- subset(rand_hie, plan_type %in% c("Catastrophic", "Free"))
rand_hie_subset$free <- 1 * (rand_hie_subset$plan_type == "Free")

Part (a)

Yes, you can estimate the ATT for the Free plan relative to Catastrophic just by running a regression of total_med_expenditure on free. The reason this works is because the plan type is randomly assigned in this problem, and, if you run a regression of an outcome on the single binary variable free, the coefficient on the binary variable will give you the difference in the average outcomes for units with free=1 relative to those with free=0—which is exactly equal to \(\widehat{ATT}\).

Part (b)

exp_reg <- lm(total_med_expenditure ~ free, data = rand_hie_subset)
summary(exp_reg)

Call:
lm(formula = total_med_expenditure ~ free, data = rand_hie_subset)

Residuals:
    Min      1Q  Median      3Q     Max 
 -532.9  -392.8  -299.4    38.4 17987.6 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   392.77      40.19   9.773   <2e-16 ***
free          140.12      49.74   2.817   0.0049 ** 
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 993.4 on 1758 degrees of freedom
Multiple R-squared:  0.004493,  Adjusted R-squared:  0.003927 
F-statistic: 7.935 on 1 and 1758 DF,  p-value: 0.004903

Relative to having only “catastrophic” insurance coverage, total medical expenditure (notice that total medical expenditure includes both how much the person paid themselves plus how much their insurance paid) is estimated to be substantially higher, on average, for individuals assigned to “free” insurance (i.e., that paid nothing for medical care); in particular, we estimate that “free” insurance results about $140 more, on average, than those with only catastrophic coverage. In my view, this difference is large in magnitude as the average expenditure is $393 for those with catastrophic coverage, which implies that those with free coverage have 36% higher total medical expenditures. Since individuals were randomly assigned to a type of plan, it seems reasonable to interpret these results as being a causal effect of plan type on total medical spending.

Part (c)

visits_reg <- lm(face_to_face_visits ~ plan_type, data = rand_hie_subset)
summary(visits_reg)

Call:
lm(formula = face_to_face_visits ~ plan_type, data = rand_hie_subset)

Residuals:
   Min     1Q Median     3Q    Max 
-4.928 -3.192 -1.792  0.808 91.672 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept)     3.1917     0.2562  12.457  < 2e-16 ***
plan_typeFree   1.7361     0.3171   5.475 5.01e-08 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 6.333 on 1758 degrees of freedom
Multiple R-squared:  0.01676,   Adjusted R-squared:  0.01621 
F-statistic: 29.98 on 1 and 1758 DF,  p-value: 5.007e-08

These results are broadly similar to the ones before. Individuals assigned to the “free” insurance plan had, on average, 1.7 more face to face visits with doctors. This is 54% more than individuals randomly assigned to the “catastrophic” insurance plan. As in part (a), it seems reasonable to interpret these as causal effects due to the random assignment.

Part (d)

health_reg <- lm(health_index ~ plan_type, data = rand_hie_subset)
summary(health_reg)

Call:
lm(formula = health_index ~ plan_type, data = rand_hie_subset)

Residuals:
    Min      1Q  Median      3Q     Max 
-60.525  -9.784   1.516  10.616  32.216 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept)    68.5247     0.6190 110.698   <2e-16 ***
plan_typeFree  -0.7407     0.7661  -0.967    0.334    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 15.3 on 1758 degrees of freedom
Multiple R-squared:  0.0005315, Adjusted R-squared:  -3.708e-05 
F-statistic: 0.9348 on 1 and 1758 DF,  p-value: 0.3338

These results are different from the previous ones. Although individuals assigned to the “free” insurance plan appear to be utilizing more medical care, it does not appear to be improving their health (at least according to this measure of an individual’s health). The results here are not statistically significant and quantitatively small; for example, here we estimate that individuals in the “free” insurance plan have about 1% lower health index, on average, than those in the “catastrophic” plan.

Part (e)

Parts (b)-(d) seem to suggest that “free” insurance increased medical care usage without much of an effect on health (at least in the way that we were able to measure health).

Chapter 19, Extra Question 1

Treatment effect heterogeneity means that the effect of the treatment can be different across different units. Treatment effect homogeneity means that the effect of the treatment is the same across all units. Most applications (at least the ones we have considered) likely exhibit treatment effect heterogeneity.

Chapter 19, Extra Question 2

Most researchers give up on estimating individual-level treatment effects as it is too difficult. To give a particular example, suppose that we are interested in the causal effect of going to college on a person’s income. For me, I went to college, so I know my treated potential outcome. However, even I do not know exactly what my untreated potential outcome is — this means that I do not know my own treatment effect. Now compare that to a researcher which might just observe just a few of my characteristics. They have much less information with which to learn about my individual level treatment effect than I do; if I do not know my own treatment effect, it suggests that this is an essentially impossible task for a researcher.