Additional Notation for Covariates

Start with the case with only two time periods

More notation about covariates:

• \(X_{it=2}\) and \(X_{it=1}\) — time-varying covariates

• \(Z_i\) — time-invariant covariates

What We’ll Do in this Part

Two identification results:

1. Direct identifcation strategy:

   • Recover \(ATT\) using DiD identification strategy at all values of the covariates, and then averaging them

2. Indirect identification strategy:

   • Based on re-weighting the untreated group to make it have the same distribution of covariates as the treated group

Covariates in the Parallel Trends Assumption

Identification under Conditional Parallel Trends, Strategy 1

Under conditional parallel trends, we have that \[ \begin{aligned} ATT &= \E[\Delta Y | G=1] - \E[\Delta Y(0) | G=1] \hspace{150pt} \end{aligned} \]

Identification under Conditional Parallel Trends, Strategy 1

Under conditional parallel trends, we have that \[ \begin{aligned} ATT &= \E[\Delta Y | G=1] - \E[\Delta Y(0) | G=1] \hspace{150pt}\\ &=\E\Big[ \E[\Delta Y | X,G=1] \Big| G=1\Big] - \E\Big[ \E[\Delta Y(0) | X, G=1] \Big| G=1\Big] \end{aligned} \]

Identification under Conditional Parallel Trends, Strategy 1

Under conditional parallel trends, we have that \[ \begin{aligned} ATT &= \E[\Delta Y | G=1] - \E[\Delta Y(0) | G=1] \hspace{150pt}\\ &=\E\Big[ \E[\Delta Y | X,G=1] \Big| G=1\Big] - \E\Big[ \E[\Delta Y(0) | X, G=1] \Big| G=1\Big]\\ &=\E\Big[ \underbrace{\E[\Delta Y | X,G=1] - \E[\Delta Y(0) | X, G=0]}_{=:ATT(X)} \Big| G=1\Big] \end{aligned} \]

Identification under Conditional Parallel Trends, Strategy 1

From the last slide, we had that

\[ ATT = \E\Big[ \underbrace{\E[\Delta Y | X,G=1] - \E[\Delta Y(0) | X, G=0]}_{=:ATT(X)} \Big| G=1\Big] \]

Identification Strategy 2: Covariate Balancing

Example: Momentarily, suppose that the distribution of \(X\) was the same for both groups, then

Identification Strategy 2: Covariate Balancing

Example: Momentarily, suppose that the distribution of \(X\) was the same for both groups, then

\[ \begin{aligned} ATT &= \E[\Delta Y | G=1] - \E[\Delta Y(0) | G=1] \hspace{150pt}\\ &= \E[\Delta Y | G=1] - \E\Big[ \E[\Delta Y(0) | X, G=0 ] \Big| G=1\Big] \end{aligned} \]

Identification Strategy 2: Covariate Balancing

Example: Momentarily, suppose that the distribution of \(X\) was the same for both groups, then

\[ \begin{aligned} ATT &= \E[\Delta Y | G=1] - \E[\Delta Y(0) | G=1] \hspace{150pt}\\ &= \E[\Delta Y | G=1] - \E\Big[ \E[\Delta Y(0) | X, G=0 ] \Big| G=1\Big]\\ &= \E[\Delta Y | G=1] - \E\Big[ \E[\Delta Y(0) | X, G=0 ] \Big| G=0\Big] \end{aligned} \]

Identification Strategy 2: Covariate Balancing

Example: Momentarily, suppose that the distribution of \(X\) was the same for both groups, then

\[ \begin{aligned} ATT &= \E[\Delta Y | G=1] - \E[\Delta Y(0) | G=1] \hspace{150pt}\\ &= \E[\Delta Y | G=1] - \E\Big[ \E[\Delta Y(0) | X, G=0 ] \Big| G=1\Big]\\ &= \E[\Delta Y | G=1] - \E\Big[ \E[\Delta Y(0) | X, G=0 ] \Big| G=0\Big]\\ &= \E[\Delta Y | G=1] - \E[\Delta Y(0) | G=0] \end{aligned} \]

Alternative Identification Strategy: Covariate Balancing

More generally: We would not expect the distribution of covariates to be the same across groups.

Alternative Identification Strategy: Covariate Balancing

More generally: We would not expect the distribution of covariates to be the same across groups.

However the idea of covariate balancing is to come up with balancing weights \(\nu_0(X)\) such that the distribution of \(X\) is the same in the untreated group as it is in the treated group after applying the balancing weights. Then we would have that

\[ \begin{aligned} ATT &= \E[\Delta Y | G=1] - \E[\Delta Y(0) | G=1] \hspace{150pt}\\ &= \E[\Delta Y | G=1] - \E\Big[ \E[\Delta Y(0) | X, G=0 ] \Big| G=1\Big] \end{aligned} \]

Alternative Identification Strategy: Covariate Balancing

More generally: We would not expect the distribution of covariates to be the same across groups.

However the idea of covariate balancing is to come up with balancing weights \(\nu_0(X)\) such that the distribution of \(X\) is the same in the untreated group as it is in the treated group after applying the balancing weights. Then we would have that

\[ \begin{aligned} ATT &= \E[\Delta Y | G=1] - \E[\Delta Y(0) | G=1] \hspace{150pt}\\ &= \E[\Delta Y | G=1] - \E\Big[ \E[\Delta Y(0) | X, G=0 ] \Big| G=1\Big]\\ &= \E[\Delta Y | G=1] - \E\Big[ \nu_0(X) \E[\Delta Y(0) | X, G=0 ] \Big| G=0\Big] \end{aligned} \]

Alternative Identification Strategy: Covariate Balancing

More generally: We would not expect the distribution of covariates to be the same across groups.

However the idea of covariate balancing is to come up with balancing weights \(\nu_0(X)\) such that the distribution of \(X\) is the same in the untreated group as it is in the treated group after applying the balancing weights. Then we would have that

\[ \begin{aligned} ATT &= \E[\Delta Y | G=1] - \E[\Delta Y(0) | G=1] \hspace{150pt}\\ &= \E[\Delta Y | G=1] - \E\Big[ \E[\Delta Y(0) | X, G=0 ] \Big| G=1\Big]\\ &= \E[\Delta Y | G=1] - \E\Big[ \nu_0(X) \E[\Delta Y(0) | X, G=0 ] \Big| G=0\Big]\\ &= \E[\Delta Y | G=1] - \E[\nu_0(X) \Delta Y(0) | G=0] \end{aligned} \]

Discussion

The arguments about suggest that, in order to estimate the \(ATT\), we will either need to

1. Correctly model \(\E[\Delta Y(0) | X, G=0]\) (i.e, specifiy a model for \(m_0(X)\))

2. Balance the distribution of \(X\) to be the same for the untreated group relative to the treated group.

Limitations of TWFE Regressions

In this setting, it is common to run the following TWFE regression:

\[Y_{it} = \theta_t + \eta_i + \alpha D_{it} + X_{it}'\beta + e_{it}\]

Limitations of TWFE Regressions

Focusing on the case with two periods, to estimate the model, we take first-differences to eliminate the unit fixed effects and ultimately estimate the regression

\[\Delta Y_{it} = \Delta \theta_t + \alpha D_{it} + \Delta X_{it}'\beta + \Delta e_{it}\]

Limitations of TWFE Regressions

Can show that the coefficient \(\alpha\) in the TWFE regression can be decomposed as

\[ \small \alpha = \underbrace{\E\Big[ w_1(\Delta X) ATT(X_{t=2},X_{t=1},Z)\Big| G=1 \Big]}_{\textrm{weighted avg. of $ATT(X)$}} + \underbrace{\E\Big[ w_1(\Delta X) \Big( \E[\Delta Y | X_{t=2}, X_{t=1}, Z, G=0] - \L_0(\Delta Y | \Delta X) \Big) \Big| G=1 \Big]}_{\textrm{misspecification bias}}\]

Limitations of TWFE Regressions

Can show that the coefficient \(\alpha\) in the TWFE regression can be decomposed as

\[ \small \alpha = \underbrace{\E\Big[ w_1(\Delta X) ATT(X_{t=2},X_{t=1},Z)\Big| G=1 \Big]}_{\textrm{weighted avg. of $ATT(X)$}} + \underbrace{\E\Big[ w_1(\Delta X) \Big( \E[\Delta Y | X_{t=2}, X_{t=1}, Z, G=0] - \L_0(\Delta Y | \Delta X) \Big) \Big| G=1 \Big]}_{\textrm{misspecification bias}}\]

where \[ w_1(\Delta X) := \frac{\big(1-\L(D|\Delta X)\big) \pi}{\E\big[(D-\L(D|\Delta X))^2\big]}\]

About the first term:

Ideally, we would like \(w_1(\Delta X)=1\), which would imply that this term is equal to \(ATT\).

Relative to this baseline, these weights have some drawbacks:

• The weights can be negative

• The weights suffer from a form of weight reversal (e.g., Słoczyński (2022)):

Limitations of TWFE Regressions

\[ \small \alpha = \underbrace{\E\Big[ w_1(\Delta X) ATT(X_{t=2},X_{t=1},Z)\Big| G=1 \Big]}_{\textrm{weighted avg. of $ATT(X)$}} + \underbrace{\E\Big[ w_1(\Delta X) \Big( \E[\Delta Y | X_{t=2}, X_{t=1}, Z, G=0] - \L_0(\Delta Y | \Delta X) \Big) \Big| G=1 \Big]}_{\textrm{misspecification bias}}\]

The misspecification bias component is equal to 0 if either:

1. \(\E[\Delta Y|X_{t=2},X_{t=1},Z,G=0] = \L_0(\Delta Y| \Delta X)\) (i.e., the model for untreated potential outcomes is linear in \(\Delta X\))

Limitations of TWFE Regressions

Consider the condition

\[ \E[\Delta Y | X_{t=2}, X_{t=1}, Z, G=0] = \L_0(\Delta Y | \Delta X) \]

Notice that this condition involves two things:

1. A condition about linearity (makes sense…and similar to the cross-sectional case)

2. Changing the covariates that show up from \((X_{t=2}, X_{t=1}, Z)\) to \(\Delta X\)

Limitations of TWFE Regressions

Next, a property of implicit regression weights is that they balance the means of regressors included in the model (Chattopadhyay and Zubizarreta (2023))

• This is good property in the cross-sectional setting and suggests that typically misspecification bias is likely to be small in that case

• In our case, this means that the TWFE regression will balance the mean of \(\Delta X\) across groups

Limitations of TWFE Regressions

In Caetano and Callaway (2023), we refer to the misspecification bias term above as hidden linearity bias.

Limitations of TWFE Regressions

What is going wrong with the TWFE regression?

\[Y_{it} = \theta_t + \eta_i + \alpha D_{it} + X_{it}'\beta + e_{it}\]

The source of the issues with the TWFE regression is that, when we difference out the unit fixed effect, we also transform the covariates.

• We have focused on the case with two periods and estimation in first differences, but similar issues apply in cases with more periods and with other transformations (e.g., within transformation)

Limitations of TWFE Regressions

One last question: How much does this matter in practice?

• Not all that easy to check how far away \(\E[\Delta Y | X_{t=1}, X_{t=2}, Z, G=0]\) is from \(\L_0(\Delta Y|\Delta X)\)

Alternative Estimation Strategies

Given the limitations of TWFE regressions, we will consider alternative estimation strategies:

1. Regression Adjustment (RA)

2. Propensity Score Weighting (IPW)

3. Augmented-Inverse Propensity Score Weighting (AIPW)

Regression Adjustment (RA)

Recall our first identification result above:

\[ATT = \E[\Delta Y | G=1] - \E\Big[ \underbrace{\E[\Delta Y(0) | X, G=0]}_{=:m_0(X)} \Big| G=1\Big]\]

The most direct way to proceed is by proposing a model for \(m_0(X)\). For example, \(m_0(X) = X'\beta_0\).

Regression Adjustment (RA)

Recall our first identification result above:

\[ATT = \E[\Delta Y | G=1] - \E\Big[ \underbrace{\E[\Delta Y(0) | X, G=0]}_{=:m_0(X)} \Big| G=1\Big]\]

This expression suggests a regression adjustment estimator, based on:

\[ATT = \E[\Delta Y | G=1] - \E[X'\beta_0|G=1]\]

Inverse Propensity Score Weighting (IPW)

Alternatively, recall our identification strategy based on re-weighting: \[ ATT = \E[\Delta Y | G=1] - \E[ \nu_0(X) \Delta Y | G=0] \]

Augmented-Inverse Propensity Score Weighting (AIPW)

You can show an additional identification result:

\[ATT = \E\left[ \Delta Y_{t} - \E[\Delta Y_{t} | X, G=0] \big| G=1\right] - \E\left[ \frac{p(X)(1-\pi)}{(1-p(X))\pi} \big(\Delta Y_{t} - \E[\Delta Y_{t} | X, G=0]\big) \Big| G=0\right]\]

This requires estimating both \(p(X)\) and \(\E[\Delta Y|X,G=0]\).

Discussion

Regarding the previous issues with TWFE regressions, RA, IPW, and AIPW satisfy:

Issue 1: Multiple periods ✔

Issue 2: Time-invariant covariates ✔

Issue 3: Covariates affected by the treatment ?

Issue 4: Hidden linearity bias ✔

Issue 5: Weighted average of \(ATT(X)\) ✔

Review of Results under Unconditional Parallel Trends

With multiple periods and variation in treatment timing, and when parallel trends holds across all groups and time periods, we previously showed that:

\[ ATT(g,t) = \E[Y_t - Y_{g-1} | G=g] - \E[Y_t - Y_{g-1} | U=1] \]

Multiple Time Periods and Variation in Treatment Timing

Under this assumption, using similar arguments to the ones above, one can show that

\[ATT(g,t) = \E\left[ \left( \frac{\indicator{G=g}}{\pi_g} - \frac{p_g(\mathbf{X},Z)U}{(1-p_g(\mathbf{X},Z))\pi_g}\right)\Big(Y_{t} - Y_{g-1} - m_{gt}^0(\mathbf{X},Z)\Big) \right]\]

where \(p_g(\mathbf{X},Z) := \P(G=g|\mathbf{X},Z,\indicator{G=g}+U=1)\) and \(m_{gt}^0(\mathbf{X},Z) := \E[Y_{t}-Y_{g-1}|\mathbf{X},Z,U=1]\).

Practical Considerations

Because \(\mathbf{X}_i\) contains \(X_{it}\) for all time periods, terms like \(m_{gt}^0(\mathbf{X},Z)\) can be quite high-dimensional (and hard to estimate) in many applications.

Empirical Example: Minimum Wages and Employment

• This is the same application that we considered in Session 1

Empirical Example: Minimum Wages and Employment

Let’s start by assuming that parallel trends holds conditional on a county’s population and average income (sometimes we’ll add region too)

• i.e., we would like to compare treated and untreated counties with similar populations and average incomes

TWFE Results without Covariates

twfe_res2 <- fixest::feols(lemp ~ post | id + year,
                           data=data2,
                           cluster="id")

modelsummary(list(twfe_res2), gof_omit=".*")

Event Study without Covariates (Callaway and Sant’Anna)

attgt <- did::att_gt(yname="lemp",
                     idname="id",
                     gname="G",
                     tname="year",
                     data=data2,
                     control_group="nevertreated",
                     base_period="universal")
attes <- aggte(attgt, type="dynamic")
ggdid(attes)

Overall ATT without Covariates (Callaway and Sant’Anna)

attO <- did::aggte(attgt, type="group")
summary(attO)

Call:
did::aggte(MP = attgt, type = "group")

Reference: Callaway, Brantly and Pedro H.C. Sant'Anna.  "Difference-in-Differences with Multiple Time Periods." Journal of Econometrics, Vol. 225, No. 2, pp. 200-230, 2021. <https://doi.org/10.1016/j.jeconom.2020.12.001>, <https://arxiv.org/abs/1803.09015> 


Overall summary of ATT's based on group/cohort aggregation:  
     ATT    Std. Error     [ 95%  Conf. Int.]  
 -0.0571        0.0085    -0.0738     -0.0404 *


Group Effects:
 Group Estimate Std. Error [95% Simult.  Conf. Band]  
  2004  -0.0888     0.0189       -0.1302     -0.0475 *
  2006  -0.0427     0.0083       -0.0610     -0.0245 *
---
Signif. codes: `*' confidence band does not cover 0

Control Group:  Never Treated,  Anticipation Periods:  0
Estimation Method:  Doubly Robust

Two periods TWFE Regression with Covariates

# run TWFE regression
data2_subset <- subset(data2, year %in% c(2003,2004))
data2_subset <- subset(data2_subset, G %in% c(0, 2004))
twfe_x <- fixest::feols(lemp ~ post + lpop + lavg_pay | id + year,
                        data=data2_subset,
                        cluster="id")
modelsummary(twfe_x, gof_omit=".*")

Diagnose covariate balance

library(twfeweights)
tp_wts <- two_period_reg_weights(
  yname = "lemp",
  tname = "year",
  idname = "id",
  gname = "G",
  xformula = ~lpop + lavg_pay,
  extra_balance_vars_formula = ~region,
  data = data2_subset
)

Diagnose covariate balance

ggtwfeweights(tp_wts,
              absolute_value=FALSE,
              standardize=TRUE,
              plot_relative_to_target=FALSE) +
  xlim(c(-2,2))

TWFE with more periods and covariates

# run TWFE regression
twfe_x <- fixest::feols(lemp ~ post + lpop + lavg_pay | id + year,
                        data=data2,
                        cluster="id")
modelsummary(twfe_x, gof_omit=".*")

Diagnose covariate balance

twfe_wts <- implicit_twfe_weights(
  yname = "lemp",
  tname = "year",
  idname = "id",
  gname = "G",
  xformula = ~lpop + lavg_pay,
  data = data2,
  base_period = "gmin1"
)
covariate_balance <- twfe_cov_bal(twfe_wts, ~ region + lpop + lavg_pay + -1)

Diagnose covariate balance

ggtwfeweights(covariate_balance,
              absolute_value = FALSE,
              standardize = TRUE,
              plot_relative_to_target = FALSE) +
  xlim(c(-1,1))

CS (2021) AIPW, \(X_{g-1}, Z\)

# callaway and sant'anna including covariates
cs_x <- att_gt(yname="lemp",
               tname="year",
               idname="id",
               gname="G",
               xformla=~region + lpop + lavg_pay,
               control_group="nevertreated",
               base_period="universal",
               est_method="dr",
               data=data2)
cs_x_res <- aggte(cs_x, type="group")
summary(cs_x_res)
cs_x_dyn <- aggte(cs_x, type="dynamic")
ggdid(cs_x_dyn)

CS (2021) AIPW, \(X_{g-1}, Z\)

Call:
aggte(MP = cs_x, type = "group")

Reference: Callaway, Brantly and Pedro H.C. Sant'Anna.  "Difference-in-Differences with Multiple Time Periods." Journal of Econometrics, Vol. 225, No. 2, pp. 200-230, 2021. <https://doi.org/10.1016/j.jeconom.2020.12.001>, <https://arxiv.org/abs/1803.09015> 


Overall summary of ATT's based on group/cohort aggregation:  
     ATT    Std. Error     [ 95%  Conf. Int.]  
 -0.0317        0.0075    -0.0463      -0.017 *


Group Effects:
 Group Estimate Std. Error [95% Simult.  Conf. Band]  
  2004  -0.0509     0.0204       -0.0928     -0.0089 *
  2006  -0.0230     0.0076       -0.0387     -0.0074 *
---
Signif. codes: `*' confidence band does not cover 0

Control Group:  Never Treated,  Anticipation Periods:  0
Estimation Method:  Doubly Robust

CS (2021) AIPW, \(X_{g-1}, Z\)

Check covariate balance

# similar code as before...check course materials

Additional Results and Bonus Material

Additional Results:

• Add region as a covariate in TWFE [details]

• One-shot imputation estimators [details]

• Regression adjustment [details]

• Inverse propensity score weighting [details]

Bonus Material:

• “Bad Controls” [details]

Conclusion

• Including covariates in the parallel trends assumption can make difference-in-differences identification strategies more plausible

• If you want to include covariates in the parallel trends assumption, it is better to use approaches that directly include the covariates relative to estimation strategies that transform the covariates

• In the minimum wage application, we did better in terms of covariate balance with regression adjustment and AIPW; however, there still seemed to be violations of parallel trends in pre-treatment periods

  • I know some ways to get rid of these apparent violations of parallel trends, but I want to use this as an angle to get invited back for Session 3…

Side-Discussion: Imputation Estimators

Regression adjustment is closely related to imputation estimators, which we talked about in the first session.

Side-Discussion: Imputation Estimators

One-shot imputation estimators have similar limitations as TWFE regressions when it comes to covariates:

\[Y_{it}(0) = \theta_t + \eta_i + X_{it}'\beta + e_{it}\]

Issue 1: Multiple periods ✔

Issue 2: Time-invariant covariates ⚠

Issue 3: Covariates affected by the treatment ❌

Issue 4: Hidden linearity bias ❌

• You can see that it implicitly relies on \(\E[\Delta Y(0) | X_{t=2}, X_{t=1}, Z, G=0] = \E[\Delta Y(0) | \Delta X, G=0]\)

Issue 5: Weighted average of \(ATT(X)\) ✔

[back]

Understanding Double Robustness

To understand double robustness, we can rewrite the expression for \(ATT\) as \[\begin{align*} ATT = \E\left[ \frac{D}{\pi} \Big(\Delta Y - m_0(X)\Big) \right] - \E\left[ \frac{p(X)(1-D)}{(1-p(X))\pi} \Big(\Delta Y - m_0(X)\Big)\right] \end{align*}\]

The first term is exactly the same as what comes from regression adjustment

• If we correctly specify a model for \(m_0(X)\), it will be equal to \(ATT\).

• If \(m_0(X)\) not correctly specified, then, by itself, this term will be biased for \(ATT\)

The second term can be thought of as a de-biasing term

• If \(m_0(X)\) is correctly specified, it is equal to 0

• If \(p(X)\) is correctly specified, it reduces to \(\E[\Delta Y_{t}(0) | G=1] - \E[m_0(X)|G=1]\) which both delivers counterfactual untreated potential outcomes and removes the (possibly misspecified) second term from the first equation

[Back]

Add region as a covariate

We’ll allow for path of outcomes to depend on region of the country

# run TWFE regression
twfe_x <- fixest::feols(lemp ~ post + lpop + lavg_pay | id + region^year,
                        data=data2,
                        cluster="id")
modelsummary(twfe_x, gof_omit=".*")

Relative to previous results, this is much smaller—this is (broadly) in line with the literature where controlling for region often matters a great deal (e.g., Dube, Lester, and Reich (2010)).

Check covariate balance

# similar code as before...check course materials

[back]

One-shot imputation (i.e., regression adjustment with only \(\Delta X\) as covariate)

# it's reg. adj. even though the function says aipw...
ra_wts <- implicit_aipw_weights(
  yname = "lemp",
  tname = "year",
  idname = "id",
  gname = "G",
  xformula = ~ 1,
  d_covs_formula = ~ lpop + lavg_pay,
  pscore_formula = ~1,
  data = data2
)
ra_wts$est

[1] -0.06098144

i.e., we estimate a somewhat larger effect of the minimum wage on teen employment

One-shot imputation (i.e., regression adjustment with only \(\Delta X\) as covariate)

ra_cov_bal <- aipw_cov_bal(ra_wts, ~ region + lpop + lavg_pay + -1)
ggtwfeweights(ra_cov_bal, absolute_value = FALSE,
              standardize = TRUE,
              plot_relative_to_target = FALSE) +
  xlim(c(-1,1))

[back]

CS (2021) Regression Adjustment, \(X_{g-1}, Z\)

# callaway and sant'anna including covariates
cs_x <- att_gt(yname="lemp",
               tname="year",
               idname="id",
               gname="G",
               xformla=~region + lpop + lavg_pay,
               control_group="nevertreated",
               base_period="universal",
               est_method="reg",
               data=data2)
cs_x_res <- aggte(cs_x, type="group")
summary(cs_x_res)
cs_x_dyn <- aggte(cs_x, type="dynamic")
ggdid(cs_x_dyn)

CS (2021) Regression Adjustment, \(X_{g-1},Z\)

Call:
aggte(MP = cs_x, type = "group")

Reference: Callaway, Brantly and Pedro H.C. Sant'Anna.  "Difference-in-Differences with Multiple Time Periods." Journal of Econometrics, Vol. 225, No. 2, pp. 200-230, 2021. <https://doi.org/10.1016/j.jeconom.2020.12.001>, <https://arxiv.org/abs/1803.09015> 


Overall summary of ATT's based on group/cohort aggregation:  
     ATT    Std. Error     [ 95%  Conf. Int.]  
 -0.0321        0.0079    -0.0477     -0.0166 *


Group Effects:
 Group Estimate Std. Error [95% Simult.  Conf. Band]  
  2004  -0.0596     0.0194       -0.1007     -0.0185 *
  2006  -0.0197     0.0080       -0.0367     -0.0027 *
---
Signif. codes: `*' confidence band does not cover 0

Control Group:  Never Treated,  Anticipation Periods:  0
Estimation Method:  Outcome Regression

CS (2021) Regression Adjustment, \(X_{g-1},Z\)

Check covariate balance

# similar code as before...check course materials

[back]

CS (2021) IPW, \(X_{g-1}, Z\)

# callaway and sant'anna including covariates
cs_x <- att_gt(yname="lemp",
               tname="year",
               idname="id",
               gname="G",
               xformla=~region + lpop + lavg_pay,
               control_group="nevertreated",
               base_period="universal",
               est_method="ipw",
               data=data2)
cs_x_res <- aggte(cs_x, type="group")
summary(cs_x_res)
cs_x_dyn <- aggte(cs_x, type="dynamic")
ggdid(cs_x_dyn)

CS (2021) IPW, \(X_{g-1}, Z\)

Call:
aggte(MP = cs_x, type = "group")

Reference: Callaway, Brantly and Pedro H.C. Sant'Anna.  "Difference-in-Differences with Multiple Time Periods." Journal of Econometrics, Vol. 225, No. 2, pp. 200-230, 2021. <https://doi.org/10.1016/j.jeconom.2020.12.001>, <https://arxiv.org/abs/1803.09015> 


Overall summary of ATT's based on group/cohort aggregation:  
     ATT    Std. Error     [ 95%  Conf. Int.]  
 -0.0313        0.0083    -0.0475     -0.0151 *


Group Effects:
 Group Estimate Std. Error [95% Simult.  Conf. Band]  
  2004  -0.0514     0.0179       -0.0902     -0.0127 *
  2006  -0.0222     0.0072       -0.0377     -0.0067 *
---
Signif. codes: `*' confidence band does not cover 0

Control Group:  Never Treated,  Anticipation Periods:  0
Estimation Method:  Inverse Probability Weighting

CS (2021) IPW, \(X_{g-1}, Z\)

[back]

Covariates Affected by the Treatment

So far, our discussion has been for the case where the time-varying covariates evolve exogenously.

• Many (probably most) covariates fit into this category: in the minimum wage example, a county’s population probably fits here.

Covariates Affected by the Treatment

The most common practice is to just completely drop these covariates from the analysis

• Not clear if this is the right idea though…

We will consider some alternatives

• Condition on pre-treatment value of bad control
• Treat bad control as an outcome (i.e., use some identification strategy), then feed this into the main analysis as a covariate

Additional Notation

To wrap our heads around this, let’s go back to the case with two time periods.

Define treated and untreated potential covariates: \(X_{it}(1)\) and \(X_{it}(0)\). Notice that in the “textbook” two period setting, we observe \[X_{it=2} = D_i X_{it=2}(1) + (1-D_i) X_{it=2}(0) \qquad \textrm{and} \qquad X_{it=1} = X_{it=1}(0)\]

Then, we will consider parallel trends in terms of untreated potential outcomes and untreated potential covariates:

Identification Issues

Following the same line of argument as before, it follows that

\[ATT = \E[\Delta Y | G=1] - \E\Big[ \E[\Delta Y(0) | X_{t=2}(0), X_{t=1}(0), Z, G=0] \Big| G=1\Big]\]

The second term is the tricky one. Notice that:

• The inside conditional expectation is identified — we see untreated potential outcomes and covariates for the untreated group

• However, we cannot average over \(X_{t=2}(0)\) for the treated group, because we don’t observe \(X_{t=2}(0)\) for the treated group

Option 1: Ignore

One idea is to just ignore that the covariates may have been affected by the treatment:

The limitations of this approach are well known (even discussed in MHE), and this is not typically the approach taken in empirical work

Option 2: Drop

It is more common in empirical work to drop \(X{it}(0)\) entirely from the parallel trends assumption

In my view, this is not attractive either though. If we believe this assumption, then we have basically solved the bad control problem by assuming that it does not exist.

Option 3: Tweak

Perhaps a better alternative identifying assumption is the following one

Option 4: Extra Assumptions

Another option is to keep the original identifying assumption, but add additional assumptions where we (in some sense) treat \(X_t\) as an outcome and as a covariate.

Recall:

\[ATT = \E[\Delta Y | G=1] - \E\Big[ \E[\Delta Y(0) | X_{t=2}(0), X_{t=1}(0), Z, G=0] \Big| G=1\Big]\]

Option 4: Dealing with \(X_{t=2}(0)\)

Intuition: For the treated group, the time-varying covariate would have evolved in the same way over time as it actually did for the untreated group, conditional on \(X_{t=1}\) and \(Z\).

• Notice that this assumption only concerns untreated potential covariates \(\implies\) it allows for \(X_{t=2}\) to be affected by the treatment

• Making an assumption like this indicates that \(X_{t=2}(0)\) is playing a dual role: (i) start by treating it as if it’s an outcome, (ii) have it continue to play a role as a covariate

Under this assumption, can show that we can recover the \(ATT\):

\[ATT = \E[\Delta Y | G=1] - \E\left[ \E[\Delta Y | X_{t=1}, Z, G=0] \Big| G=1 \right]\]

This is the same expression as in Option 3

Option 4: Additional Discussion

In some cases, it may make sense to condition on other additional variables (e.g., the lagged outcome \(Y_{t=1}\)) in the covariate unconfoundedness assumption. In this case, it is still possible to identify \(ATT\), but it is more complicated

It could also be possible to use alternative identifying assumptions besides covariate unconfoundedness — at a high-level, we somehow need to recover the distribution of \(X_{t=2}(0)\)

• e.g., Brown, Butts, and Westerlund (2023)

See Caetano et al. (2022) for more details about bad controls.

[Back]