Synthetic control methods hold the promise of identifying a credible counterfactual by closely reproducing pretreatment outcome paths.
Synthetic control methods hold the promise of identifying a credible counterfactual by closely reproducing pretreatment outcome paths. Their reliability, however, depends in part on how effectively they constrain researchers’ less-visible degrees of freedom. We explore these methods’ sensitivity by constructing 1440 plausibly defensible specifications to estimate the employment effect of California’s $20 fast-food minimum wage. While over 96 percent of the estimated employment effects are negative, the range is vast, illustrating that synthetic control methods are sensitive to specification choice. Estimates from simpler and far more transparent difference-in-differences methods lie near the centers of the comparable distributions of synthetic control estimates. The labor market effects of California's AB 1228 are also analyzed, with varied conclusions, in both the initial and revised version of NBER working paper34033and in working paper35171. Clemens thanks the Hoover Institution for its support in his capacity as an adjunct Senior Fellow. The authors used AI tools in attempting to replicate existing research, identify the source of discrepancies therein, and assist in creating replication code. AI tools were also used to create and edit LaTeX tables from output, create an initial outline, edit figures, identify typos, improve clarity, assist in describing technical material (including details of table and figure notes), and audit the draft and replication code package for consistency. The authors reviewed and edited the content as needed and take full responsibility for the content of the publication. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research. In addition toworking papers, the NBER disseminates affiliates’ latest findings through a range of free periodicals—theNBER Reporter, theNBER Digest, theBulletin on Health, theBulletin on the Economics of Alzheimer's, and theBulletin on Entrepreneurship—as well asresearch spotlights, lectures, and panel videos.