Appendix B — Appendix for Chapter 22
B.1 Staggered treatment designs
B.1.1 Introduction
In this section, we consider a staggered treatment design in which units receive treatment at different time periods. In this design, treatment timing varies across units, such that some units may receive treatment in the first period, while others may receive treatment in the second period, and so on. This design is particularly useful for evaluating the effects of policies or interventions implemented at different times across units, such as states or countries. This setting allows us to define distinct treatment groups (cohorts) based on treatment timing. For example, we can define one cohort consisting of units that receive treatment in the first period, another cohort consisting of units that receive treatment in the second period, and so on, and then compare these cohorts to a control group that never receives treatment.
We use \(Y_{it}\) to denote the observed outcome for unit \(i\) at time \(t\), for \(i=1, \ldots, n\) and \(t = 1, \ldots, T\). We use \(Y_{it}(\infty)\) to denote the never-treated potential outcome for unit \(i\) at time \(t\), and \(Y_{it}(g)\) to denote the potential outcome for unit \(i\) at time \(t\) if it receives treatment in period \(g\). Let \(G_i\) denote the period in which unit \(i\) receives treatment, with \(G_i = \infty\) if unit \(i\) never receives treatment, and let \(\mathcal{G}\) denote the set of all possible treatment periods. Thus, the set \(\mathcal{G}\) defines the cohorts of treated units and untreated units in the staggered treatment design. Each element of \(\mathcal{G}\) corresponds to a specific treatment period, and the control group corresponds to the element \(\infty\).
For example, suppose there are three time periods: 2019, 2020, 2021, and 2022. Further, suppose that some units receive treatment in 2020, some in 2021, some in 2022, and some never receive treatment. In this case, we can define three cohorts of treated units: the first, second, and third cohorts consist of units treated in 2020, 2021, and 2022, respectively. The never-treated group consists of units that never receive treatment (i.e., \(G_i = \infty\)). Thus, in this example, we have \(\mathcal{G} = {2020, 2021, 2022, \infty}\).
Using this notation, for the \(i\)th unit that belongs to cohort \(g\), we can express its observed outcome as \[ Y_{it} = \begin{cases} Y_{it}(g) & \text{if } t \geq g,\\ Y_{it}(\infty) & \text{if } t < g. \end{cases} \]
In our example, where \(\mathcal{G} = \{2020, 2021, 2022, \infty\}\), the observed outcome for unit \(i\) at time \(t\) can be expressed as follows: \[ Y_{it} = \begin{cases} Y_{it}(2020) & \text{if } t \geq 2020 \text{ and } G_i = 2020,\\ Y_{it}(2021) & \text{if } t \geq 2021 \text{ and } G_i = 2021,\\ Y_{it}(2022) & \text{if } t \geq 2022 \text{ and } G_i = 2022,\\ Y_{it}(\infty) & \text{if } t < G_i. \end{cases} \]
For our example with \(\mathcal{G} = {2020, 2021, 2022, \infty}\), suppose that the first, second, and third treated cohorts consist of 5, 3, and 8 units, respectively, while the never-treated group consists of 4 units. In the following example, we use the panelview function from the panelView package to visualize treatment timing across units and over time. In Figure B.1, the first 5 rows, the next 3 rows, and the following 8 rows correspond to the first, second, and third treated cohorts, respectively, while the last 4 rows correspond to the never-treated group. In addition, the not-yet-treated group at each time period corresponds to the rows associated with treated cohorts that have not yet received treatment by that period. For example, in 2020, the not-yet-treated group consists of the units in rows 1 to 16; in 2021, it consists of the units in rows 6 to 16; and in 2022, it consists of the units in rows 9 to 16.
set.seed(123)
n <- 20
G <- c(rep(2020, 5), rep(2021, 3), rep(2022, 8), rep(Inf, 4))
time <- 2019:2022
data_example <- expand.grid(id = 1:n, time = time) %>%
mutate(G = G[id]) %>%
mutate(D = ifelse(time >= G, 1, 0))
# Define the outcome variable
data_example <- data_example %>%
mutate(Y = rnorm(nrow(data_example), mean = 0, sd = 1))
# Create a panel view plot
panelview(
data = data_example,
formula = Y ~ D,
index = c("id", "time"),
main = "",
xlab = "Year",
ylab = "Units",
display.all = FALSE,
by.timing = TRUE,
pre.post = TRUE,
theme = "red",
cex.legend = 9
)
In the staggered treatment design, the parameter of interest is often the group-time average treatment effect, which is defined as the average treatment effect for units in cohort \(g\) at time \(t\), and is given by \[ ATT(g,t) = \mathbb{E}[Y_{it}(g) - Y_{it}(\infty) | G_i = g]. \] This parameter captures the average effect of treatment for units in cohort \(g\) at time \(t\), relative to the never-treated potential outcome.
B.1.2 Identification and estimation without covariates
To identify and estimate \(ATT(g,t)\), we assume the following assumptions.
The no anticipation assumption states that units do not anticipate future treatment and thus their potential outcomes before treatment are the same as if they were never treated. As in the case of common treatment timing, this assumption suggests that \(ATT(g,t)=0\) for all \(t < g\). Due to the staggered nature of the treatment, we can have different comparison groups at each time period, yielding different parallel trends assumptions.
The first assumption, based on the never-treated group, states that in the post-treatment period, the average change in the never-treated potential outcome from period \(t-1\) to period \(t\) is the same for units in cohort \(g\) and units in the never-treated group. The second assumption, based on the not-yet-treated groups, states that the average change in the never-treated potential outcome is the same for units in cohort \(g\) and units in the not-yet-treated cohorts. Finally, the parallel trends assumption for every period and group states that the average change in the never-treated potential outcome is the same for units in cohort \(g\) and units in any other cohort.
Theorem B.1 (Identification of the group-time average treatment effects)
- Under the no anticipation assumption and the parallel trends assumption based on the never-treated group, for all \(g \in \mathcal{G}\) and \(t \geq g\), we have \[ ATT(g,t) = \E[Y_{it} - Y_{i,g-1}|G_i = g] - \E[Y_{it} - Y_{i,g-1}|G_i = \infty]. \]
- Under the no anticipation assumption and the parallel trends assumption based on the not-yet-treated groups, for all \(g \in \mathcal{G}\) and \(t \geq g\), we have \[ ATT(g,t) = \E[Y_{it} - Y_{i,g-1}|G_i = g] - \E[Y_{it} - Y_{i,g-1}|G_i > \max\{g,t\}]. \]
- Under the no anticipation assumption and the parallel trends assumption for every period and group, for all \(g, g' \in \mathcal{G}\) and time periods \(t\) such that \(g'> t\) and \(t \geq g\), we have \[ ATT(g,t) = \E[Y_{it} - Y_{i,t_0}|G_i = g] - \E[Y_{it} - Y_{i,t_0}|G_i = g'], \] where \(t_0\) is any time period such that \(t_0 < g\).
Proof (Proof of Theorem Theorem B.1). This theorem can be proved as in the case of common treatment timing, by adding and subtracting the appropriate never-treated potential outcomes, and then applying the no anticipation and the appropriate parallel trends assumption. We provide the proof of the first result, and the proofs of the second and third results are similar.
For the first result, we assume that the no anticipation assumption and the parallel trends assumption based on the never-treated group hold. Then, for all \(g \in \mathcal{G}\) and \(t \geq g\), we have \[\begin{align*} ATT(g,t) &= \E[Y_{it}(g) - Y_{it}(\infty)|G_i = g]\\ &= \E[Y_{it}(g) - Y_{i,g-1}(\infty)|G_i = g] - \E[Y_{it}(\infty) - Y_{i,g-1}(\infty)|G_i = g]\\ &= \E[Y_{it}(g) - Y_{i,g-1}(\infty)|G_i = g] - \E[Y_{it}(\infty) - Y_{i,g-1}(\infty)|G_i = \infty]\\ &= \E[Y_{it} - Y_{i,g-1}|G_i = g] - \E[Y_{it} - Y_{i,g-1}|G_i = \infty], \end{align*}\] where the second equality follows from adding and subtracting \(Y_{i,g-1}(\infty)\), the third equality follows from the parallel trends assumption based on the never-treated group, and the last equality follows from the no anticipation assumption and the definition of the observed outcome.
Given a random sample of \(n\) units, we can estimate \(ATT(g,t)\) using the sample analog of the identification results in Theorem Theorem B.1. Under the no anticipation assumption and the parallel trends assumption based on the never-treated group, for all \(g \in \mathcal{G}\) and \(t \geq g\), we can estimate \(ATT(g,t)\) as \[ \begin{align} \widehat{ATT}(g,t) &= \frac{1}{n_g}\sum_{i=1}^n (Y_{it} - Y_{i,g-1})\mathbf{1}\{G_i = g\} \\ &- \frac{1}{n_\infty}\sum_{i=1}^n (Y_{it} - Y_{i,g-1})\mathbf{1}\{G_i = \infty\}, \end{align} \] where \(n_g = \sum_{i=1}^n \mathbf{1}\{G_i = g\}\) and \(n_\infty = \sum_{i=1}^n \mathbf{1}\{G_i = \infty\}\) are the number of units in cohort \(g\) and the never-treated group, respectively. The second result in Theorem B.1, for all \(g \in \mathcal{G}\) and \(t \geq g\), we can estimate \(ATT(g,t)\) as \[ \begin{align} \widehat{ATT}(g,t) &= \frac{1}{n_g}\sum_{i=1}^n (Y_{it} - Y_{i,g-1})\mathbf{1}\{G_i = g\} \\ &- \frac{1}{n_{\max\{g,t\}}}\sum_{i=1}^n (Y_{it} - Y_{i,g-1})\mathbf{1}\{G_i > \max\{g,t\}\}, \end{align} \] where \(n_{\max\{g,t\}} = \sum_{i=1}^n \mathbf{1}\{G_i > \max\{g,t\}\}=\sum_{i=1}^n \mathbf{1}\{G_i > t\}\) is the number of units in the not-yet-treated cohorts. Finally, under the parallel trends assumption for every period and group, for all \(g, g' \in \mathcal{G}\) and time periods \(t\) such that \(g'> t\) and \(t \geq g\), we can estimate \(ATT(g,t)\) as \[ \begin{align} \widehat{ATT}(g,t) &= \frac{1}{n_g}\sum_{i=1}^n (Y_{it} - Y_{i,t_0})\mathbf{1}\{G_i = g\} \\ &- \frac{1}{n_{g'}}\sum_{i=1}^n (Y_{it} - Y_{i,t_0})\mathbf{1}\{G_i = g'\}, \end{align} \] where \(n_{g'} = \sum_{i=1}^n \mathbf{1}\{G_i = g'\}\) is the number of units in cohort \(g'\), and \(t_0\) is any time period such that \(t_0 < g\).
For statistical inference, Callaway and Sant’Anna (2021) suggest using the multiplier bootstrap method to compute standard errors for \(\widehat{ATT}(g,t)\). We will use the did package to implement these estimation and inference procedures in the application below.
B.1.3 Aggregating group-time average treatment effects
In the staggered treatment design, for a cohort \(g \in \mathcal{G}\), we can have different group-time average treatment effects across time periods. For example, for cohort \(g\), we can have \(ATT(g,g)\), which is the average treatment effect for units in cohort \(g\) in the first period after they receive treatment; \(ATT(g,g+1)\), which is the average treatment effect for units in cohort \(g\) in the second period after they receive treatment; and so on. Across all cohorts and time periods, there may be a large number of group-time average treatment effects, which can be difficult to summarize and interpret. There can be several ways to aggregate the group-time average treatment effects into summary measures. See Table 1 in Callaway and Sant’Anna (2021) for a summary of the different aggregation methods proposed in the literature.
One popular summary measure is an event-study parameter showing the average effect of treatment by the length of exposure to the treatment: \[ \begin{align} \widehat{ATT}(e) &= \sum_{g \in \mathcal{G}} w_{g,e} \widehat{ATT}(g, g+ e), \end{align} \] where \(e\) is the length of exposure to the treatment, and \(w_{g,e}\) is a weight that depends on the number of units in cohort \(g\) and the number of time periods in which we can observe the effect of treatment for cohort \(g\) at length of exposure \(e\). It is given by \[ w_{g,e} = \mathbf{1}\{g + e \leq T\} P(G=g|G+e \leq T, G\leq T). \]
B.1.4 Application: The effect of Walmart entry on local labor markets
In this section, we study the effect of Walmart’s opening on local labor markets using the dataset collected by Brown and Butts (2025). The dataset contains information on the opening dates of Walmart stores across different counties in the United States, as well as information on local labor market conditions. The dataset is a balanced panel of 1277 counties over the period 1977–1999. The dataset is contained in the lw_walmart.dta file and includes the following variables:
-
fips: County FIPS code. -
year: Year of observation. -
cid: County identifier (numeric). -
first_year: Year of the first Walmart opening in the county. -
log_wholesale_emp: Log of wholesale employment level in the county. -
log_retail_emp: Log of retail employment level in the county. -
x1,x2,x3: Share of population above poverty line, share of employment in manufacturing, and share of population with high school degree or higher, respectively.
[1] "year" "fips" "cid"
[4] "first_year" "log_retail_emp" "log_wholesale_emp"
[7] "x3" "x1" "x2" In the following, we provide a table of summary statistics using the datasummary_skim function from the modelsummary package. We also add variable labels to the dataset using the set_variable_labels function from the labelled package, so that the summary statistics table has more informative variable names.
# Add variable labels
walmart_data <- set_variable_labels(
walmart_data,
year = "Year",
cid = "County ID",
first_year = "First opening",
log_wholesale_emp = "log(WholesaleEmp)",
log_retail_emp = "log(RetailEmp)",
x1 = "x1",
x2 = "x2",
x3 = "x3"
)
# Summary statistics
datasummary_skim(
walmart_data,
fmt = 2,
type = "numeric"
)| Unique | Missing Pct. | Mean | SD | Min | Median | Max | |
|---|---|---|---|---|---|---|---|
| Year | 23 | 0 | 1988.00 | 6.63 | 1977.00 | 1988.00 | 1999.00 |
| County ID | 1277 | 0 | 640.23 | 369.06 | 1.00 | 640.00 | 1280.00 |
| First opening | 15 | 0 | 1381.48 | 917.75 | 0.00 | 1989.00 | 1999.00 |
| log(RetailEmp) | 29161 | 0 | 7.76 | 1.28 | 4.58 | 7.50 | 12.92 |
| log(WholesaleEmp) | 29093 | 0 | 6.42 | 1.47 | 0.41 | 6.17 | 12.49 |
| x3 | 1277 | 0 | 0.09 | 0.03 | 0.02 | 0.09 | 0.19 |
| x1 | 1277 | 0 | 0.85 | 0.06 | 0.52 | 0.86 | 0.96 |
| x2 | 1277 | 0 | 0.10 | 0.05 | 0.01 | 0.10 | 0.29 |
Table B.1 shows that the first Walmart opening in the dataset occurred in 1986, and the last Walmart opening occurred in 1999. Thus, there are 14 cohorts of treated units, corresponding to the years 1989–1999, and a never-treated group consisting of counties that never had a Walmart opening during the sample period. Table B.2 shows the number of counties that had their first Walmart opening in each year from 1986 to 1999, as well as the number of counties that never had a Walmart opening (never-treated counties) during the sample period. The table shows that the never-treated group consists of 391 counties. The number of counties in the treated cohorts ranges from 21 to 118, with the largest number of counties having their first Walmart opening in 1990 (118 counties) and the smallest number of counties having their first Walmart opening in 1999 (21 counties). In Figure B.2, we show the distribution of the year of the first Walmart opening across counties.
| Year of first opening | Number of counties |
|---|---|
| 0 | 391 |
| 1986 | 69 |
| 1987 | 74 |
| 1988 | 60 |
| 1989 | 77 |
| 1990 | 118 |
| 1991 | 113 |
| 1992 | 88 |
| 1993 | 97 |
| 1994 | 46 |
| 1995 | 53 |
| 1996 | 22 |
| 1997 | 25 |
| 1998 | 23 |
| 1999 | 21 |
In the following, we generate the indicator D, which takes the value 1 if the county has a Walmart store in that year and in all subsequent periods, and 0 otherwise. We use this variable in the panelview function from the panelView package to visualize the treatment timing across counties and over time, as shown in Figure B.3. This figure shows the timing of Walmart openings across counties over time, with treated units shown in red and untreated units shown in gray. The height of the red bars indicates the number of treated units in each year, and the width of the red bars indicates the duration of treatment for each cohort. The figure shows that there is a staggered pattern of treatment timing, where the first Walmart openings occur in 1986 and the last Walmart openings occur in 1999. The figure also shows that there are some counties that never receive treatment (i.e., never have a Walmart opening during the sample period), which are shown in gray. Finally, in Figure B.4, we show the time series of the outcome variable log_retail_emp across counties and over time using the panelview function with type = "outcome".
# Generate D variable
walmart_data$D <- ifelse(walmart_data$first_year > 0 & walmart_data$year >= walmart_data$first_year, 1, 0)
We only consider the effects of Walmart openings on retail employment without including any covariates in the estimation. To estimate the group-time average treatment effects, we use the att_gt function from the did package. This function implements the estimation procedure described in the previous section, and it allows us to compute standard errors using the multiplier bootstrap method. In our estimation, we use the not-yet-treated group as the control group.
model_no_x <- att_gt(
yname = "log_retail_emp",
tname = "year",
idname = "cid",
gname = "first_year",
xformla = NULL,
data = walmart_data,
panel = TRUE,
control_group = "notyettreated",
bstrap = TRUE,
biters = 25000,
cband = TRUE,
base_period = "universal"
)All estimated group-time average treatment effects are contained in the model_no_x object. We can summarize these estimates using the summary function. However, since there are 14 cohorts and multiple time periods, there are many group-time average treatment effects, which can be difficult to summarize and interpret. Therefore, we will focus on summarizing these estimates using event-study parameters \(ATT(e)\) introduced in the previous section. To that end, we use the aggte function from the did package with type = "dynamic". This function takes the estimated \(ATT(g,t)\) from the model_no_x object and aggregates them into event-study parameters showing the average effect of treatment by the length of exposure to the treatment.
Call:
aggte(MP = model_no_x, type = "dynamic")
Reference: Callaway, Brantly and Pedro H.C. Sant'Anna. "Difference-in-Differences with Multiple Time Periods." Journal of Econometrics, Vol. 225, No. 2, pp. 200-230, 2021. <https://doi.org/10.1016/j.jeconom.2020.12.001>, <https://arxiv.org/abs/1803.09015>
Overall summary of ATT's based on event-study/dynamic aggregation:
ATT Std. Error [ 95% Conf. Int.]
0.0888 0.01 0.0693 0.1084 *
Dynamic Effects:
Event time Estimate Std. Error [95% Simult. Conf. Band]
-22 -0.1885 0.0867 -0.4430 0.0660
-21 -0.1765 0.0502 -0.3239 -0.0292 *
-20 -0.1770 0.0367 -0.2847 -0.0692 *
-19 -0.1707 0.0316 -0.2634 -0.0780 *
-18 -0.1224 0.0230 -0.1898 -0.0550 *
-17 -0.1175 0.0182 -0.1709 -0.0641 *
-16 -0.0928 0.0143 -0.1349 -0.0508 *
-15 -0.0778 0.0126 -0.1148 -0.0409 *
-14 -0.0652 0.0110 -0.0975 -0.0328 *
-13 -0.0504 0.0101 -0.0801 -0.0208 *
-12 -0.0434 0.0091 -0.0700 -0.0167 *
-11 -0.0389 0.0082 -0.0629 -0.0149 *
-10 -0.0294 0.0077 -0.0520 -0.0069 *
-9 -0.0255 0.0070 -0.0460 -0.0049 *
-8 -0.0182 0.0063 -0.0367 0.0004
-7 -0.0114 0.0059 -0.0287 0.0059
-6 -0.0089 0.0051 -0.0238 0.0061
-5 -0.0033 0.0045 -0.0166 0.0100
-4 0.0019 0.0039 -0.0094 0.0132
-3 0.0012 0.0029 -0.0074 0.0098
-2 0.0009 0.0021 -0.0052 0.0069
-1 0.0000 NA NA NA
0 0.0224 0.0026 0.0146 0.0301 *
1 0.0523 0.0037 0.0415 0.0631 *
2 0.0512 0.0044 0.0384 0.0640 *
3 0.0507 0.0053 0.0351 0.0663 *
4 0.0549 0.0063 0.0365 0.0734 *
5 0.0631 0.0074 0.0413 0.0850 *
6 0.0748 0.0086 0.0496 0.1001 *
7 0.0923 0.0098 0.0634 0.1211 *
8 0.1062 0.0110 0.0739 0.1385 *
9 0.1133 0.0127 0.0760 0.1506 *
10 0.1246 0.0152 0.0799 0.1694 *
11 0.1294 0.0189 0.0738 0.1850 *
12 0.1367 0.0242 0.0655 0.2079 *
13 0.1718 0.0349 0.0694 0.2743 *
---
Signif. codes: `*' confidence band does not cover 0
Control Group: Not Yet Treated, Anticipation Periods: 0
Estimation Method: Doubly RobustFinally, we plot the event-study parameters using the ggdid function from the did package. This function creates a plot of the estimated event-study parameters along with their confidence intervals. Figure B.5 shows the plot of the estimated event-study parameters. The figure indicates there are statistically significant pre-treatment effects, which suggests that the parallel trends assumption based on the not-yet-treated group may not hold for this application. The post-treatment effects show a positive and statistically significant effect of Walmart openings on retail employment, suggesting that Walmart openings may have led to an increase in retail employment in the treated counties. However, given the presence of pre-treatment effects, these post-treatment effects should be interpreted with caution, as they may be driven by factors other than the treatment effect of Walmart openings.
B.1.5 Identification and estimation with covariates
The event-study plot in Figure B.5 suggests that the parallel trends assumption based on the not-yet-treated group may not hold for this application. If we have covariates that are unbalanced across cohorts and if we beileve that these covariates are determinants of the unterated potential outcomes, then we should update our unconditional parallel trends assumptions to conditional parallel trends assumptions. In our application, we have three covariates: x1, x2, and x3. In the following table, we compare the average values of these covariates across treated cohorts and the not-yet-treated group at each time period.
covariate_balance <- walmart_data %>%
group_by(year) %>%
summarise(
meanx1_treated = mean(x1[D == 1], na.rm = TRUE),
stdx1_treated = sd(x1[D == 1], na.rm = TRUE),
meanx1_notyettreated = mean(x1[D == 0], na.rm = TRUE),
sdx1_notyettreated = sd(x1[D == 0], na.rm = TRUE),
normalized_diffx1 = (meanx1_treated - meanx1_notyettreated) / sqrt((stdx1_treated^2 + sdx1_notyettreated^2)/2),
meanx2_treated = mean(x2[D == 1], na.rm = TRUE),
stdx2_treated = sd(x2[D == 1], na.rm = TRUE),
meanx2_notyettreated = mean(x2[D == 0], na.rm = TRUE),
sdx2_notyettreated = sd(x2[D == 0], na.rm = TRUE),
normalized_diffx2 = (meanx2_treated - meanx2_notyettreated) / sqrt((stdx2_treated^2 + sdx2_notyettreated^2)/2),
meanx3_treated = mean(x3[D == 1], na.rm = TRUE),
stdx3_treated = sd(x3[D == 1], na.rm = TRUE),
meanx3_notyettreated = mean(x3[D == 0], na.rm = TRUE),
sdx3_notyettreated = sd(x3[D == 0], na.rm = TRUE),
normalized_diffx3 = (meanx3_treated - meanx3_notyettreated) / sqrt((stdx3_treated^2 + sdx3_notyettreated^2)/2)
) %>%
filter(year >= 1986) %>%
select(year, normalized_diffx1, normalized_diffx2, normalized_diffx3)
cnames <- c("Year", "x1 (Normalized Diff.)", "x2 (Normalized Diff.)", "x3 (Normalized Diff.)")
kable(covariate_balance, col.names = cnames, digits = 2)| Year | x1 (Normalized Diff.) | x2 (Normalized Diff.) | x3 (Normalized Diff.) |
|---|---|---|---|
| 1986 | -0.34 | -0.40 | -0.02 |
| 1987 | -0.40 | 0.03 | 0.33 |
| 1988 | -0.45 | 0.07 | 0.32 |
| 1989 | -0.37 | 0.10 | 0.32 |
| 1990 | -0.23 | 0.06 | 0.16 |
| 1991 | -0.17 | 0.06 | 0.08 |
| 1992 | -0.07 | 0.08 | 0.01 |
| 1993 | 0.03 | 0.10 | -0.04 |
| 1994 | 0.05 | 0.09 | -0.05 |
| 1995 | 0.11 | 0.13 | -0.03 |
| 1996 | 0.14 | 0.14 | -0.05 |
| 1997 | 0.17 | 0.12 | -0.08 |
| 1998 | 0.16 | 0.13 | -0.08 |
| 1999 | 0.18 | 0.14 | -0.08 |
Table B.3 shows that there are some normalized differences that are larger than 0.25 in absolute value, which suggests that there may be some imbalance in the covariates across treated cohorts and the not-yet-treated group at some time periods. Therefore, we consider the conditional parallel trends assumptions for the identification of the group-time average treatment effects. Let \(X\) be the vector of time invariant covariates. In our example on the effect of Walmart openings on retail employment, we can take X = (x1, x2, x3) to be the vector of covariates. As in the case of unconditional parallel trends assumptions, we can have different versions of conditional parallel trends assumptions depending on the choice of the comparison group. We state these assumptions below.
The next assumption is the conditional version of the strong overlap assumptionthat we adapted for the common treatment timing case.
In the next theorem, we show that under the no anticipation assumption, the conditional parallel trends assumptions, and the strong overlap assumption, we can identify \(ATT(g,t)\) for all \(g \in \mathcal{G}\) and \(t \geq g\).
Theorem B.2 (Identification of the group-time average treatment effects with covariates) Assume that the no anticipation and the strong overlap assumptions hold.
- Under the conditional parallel trends assumption based on the never-treated group, for all \(g \in \mathcal{G}\) and \(t \geq g\), we have \[ ATT(g,t) = \E[Y_{it} - Y_{i,g-1}|G_i = g] - \E[\E[Y_{it} - Y_{i,g-1}|G_i = \infty, X_i]|G_i = g]. \]
- Under the conditional parallel trends assumption based on the not-yet-treated groups, for all \(g \in \mathcal{G}\) and \(t \geq g\), we have \[ ATT(g,t) = \E[Y_{it} - Y_{i,g-1}|G_i = g] - \E[\E[Y_{it} - Y_{i,g-1}|G_i > \max\{g,t\}, X_i]|G_i = g]. \]
- Under the conditional parallel trends assumption for every period and group, for all \(g, g' \in \mathcal{G}\) and time periods \(t\) such that \(g'> t\) and \(t \geq g\), we have \[ ATT(g,t) = \E[Y_{it} - Y_{i,t_0}|G_i = g] - \E[\E[Y_{it} - Y_{i,t_0}|G_i = g', X_i]|G_i = g], \] where \(t_0\) is any time period such that \(t_0 < g\).
Proof (Proof of Theorem Theorem B.2). The proof of this theorem is similar to the proof of Theorem Theorem 22.3, by adding and subtracting the appropriate never-treated potential outcomes, and then applying the no anticipation and the appropriate conditional parallel trends assumption. We provide the proof of the first result, and the proofs of the second and third results are similar. \[ \begin{align} &ATT(g,t) = \E[Y_{it}(g) - Y_{it}(\infty)|G_i = g]\\ &= \E[Y_{it}(g) - Y_{i,g-1}(\infty)|G_i = g] - \E[Y_{it}(\infty) - Y_{i,g-1}(\infty)|G_i = g]\\ &= \E[Y_{it}(g) - Y_{i,g-1}(\infty)|G_i = g] - \E\left[\E[Y_{it}(\infty) - Y_{i,g-1}(\infty)|G_i = g, X_i]|G_i = g\right]\\ &= \E[Y_{it}(g) - Y_{i,g-1}(\infty)|G_i = g] - \E\left[\E[Y_{it}(\infty) - Y_{i,g-1}(\infty)|G_i = \infty, X_i]|G_i = g\right]\\ &= \E[Y_{it} - Y_{i,g-1}|G_i = g] - \E\left[\E[Y_{it} - Y_{i,g-1}|G_i = \infty, X_i]|G_i = g\right] \end{align} \] where the second equality follows from adding and subtracting \(Y_{i,g-1}(\infty)\), the third equality follows from the law of iterated expectations, the fourth equality follows from the conditional parallel trends assumption based on the never-treated group, and the last equality follows from the no anticipation assumption and the definition of the observed outcome.
As in the case of common treatment timing dicussed in Chapter 22, we can estimate \(ATT(g,t)\) using the sample analog of the identification results in Theorem Theorem B.2. The first term in these identification results \(\E[Y_{it} - Y_{i,g-1}|G_i = g]\) can be estimated using the sample average of \(Y_{it} - Y_{i,g-1}\) for units in cohort \(g\). The second term in these identification results can be estimated using the regression adjustment method, the inverse probability weighting method, or the doubly robust method discussed in Chapter 22. The att_gt function from the did package allows us to implement these estimation methods in the staggered treatment design with covariates.
In our application on the effect of Walmart openings on retail employment, we can estimate the group-time average treatment effects using the att_gt function with xformla = ~ x1 + x2 + x3 to include the covariates in the estimation. We can then summarize these estimates using event-study parameters and plot them as we did in the case without covariates.
model_with_x <- att_gt(
yname = "log_retail_emp",
tname = "year",
idname = "cid",
gname = "first_year",
xformla = ~ x1 + x2 + x3,
data = walmart_data,
panel = TRUE,
control_group = "notyettreated",
bstrap = TRUE,
biters = 25000,
cband = TRUE,
base_period = "universal"
)
event_study_with_x <- aggte(model_with_x, type = "dynamic")
summary(event_study_with_x)
Call:
aggte(MP = model_with_x, type = "dynamic")
Reference: Callaway, Brantly and Pedro H.C. Sant'Anna. "Difference-in-Differences with Multiple Time Periods." Journal of Econometrics, Vol. 225, No. 2, pp. 200-230, 2021. <https://doi.org/10.1016/j.jeconom.2020.12.001>, <https://arxiv.org/abs/1803.09015>
Overall summary of ATT's based on event-study/dynamic aggregation:
ATT Std. Error [ 95% Conf. Int.]
0.0913 0.0096 0.0725 0.1102 *
Dynamic Effects:
Event time Estimate Std. Error [95% Simult. Conf. Band]
-22 -0.1670 0.0803 -0.4036 0.0696
-21 -0.1639 0.0498 -0.3108 -0.0171 *
-20 -0.1661 0.0371 -0.2753 -0.0569 *
-19 -0.1571 0.0325 -0.2527 -0.0614 *
-18 -0.1071 0.0239 -0.1777 -0.0365 *
-17 -0.1048 0.0192 -0.1613 -0.0483 *
-16 -0.0799 0.0149 -0.1239 -0.0359 *
-15 -0.0648 0.0130 -0.1031 -0.0265 *
-14 -0.0543 0.0114 -0.0878 -0.0208 *
-13 -0.0414 0.0103 -0.0717 -0.0112 *
-12 -0.0361 0.0092 -0.0631 -0.0091 *
-11 -0.0323 0.0083 -0.0566 -0.0080 *
-10 -0.0241 0.0077 -0.0468 -0.0015 *
-9 -0.0205 0.0070 -0.0412 0.0002
-8 -0.0136 0.0064 -0.0323 0.0052
-7 -0.0074 0.0058 -0.0244 0.0097
-6 -0.0064 0.0051 -0.0214 0.0087
-5 -0.0006 0.0045 -0.0138 0.0125
-4 0.0037 0.0038 -0.0075 0.0149
-3 0.0030 0.0029 -0.0055 0.0116
-2 0.0015 0.0021 -0.0046 0.0076
-1 0.0000 NA NA NA
0 0.0219 0.0027 0.0141 0.0297 *
1 0.0518 0.0036 0.0411 0.0625 *
2 0.0506 0.0044 0.0374 0.0637 *
3 0.0509 0.0055 0.0348 0.0670 *
4 0.0545 0.0064 0.0357 0.0733 *
5 0.0613 0.0076 0.0390 0.0836 *
6 0.0707 0.0089 0.0444 0.0970 *
7 0.0892 0.0099 0.0601 0.1182 *
8 0.1038 0.0110 0.0713 0.1363 *
9 0.1139 0.0127 0.0765 0.1513 *
10 0.1314 0.0154 0.0860 0.1769 *
11 0.1407 0.0189 0.0850 0.1963 *
12 0.1497 0.0237 0.0799 0.2195 *
13 0.1882 0.0337 0.0888 0.2876 *
---
Signif. codes: `*' confidence band does not cover 0
Control Group: Not Yet Treated, Anticipation Periods: 0
Estimation Method: Doubly RobustThe event-study plot in Figure B.6 is similar to the event-study plot in Figure B.5. The figure indicates that there are still some statistically significant pre-treatment effects, suggesting that the conditional parallel trends assumption based on the not-yet-treated group may not hold in this application. Therefore, given the presence of these pre-treatment effects, the positive post-treatment effects should be interpreted with caution.
