“Estimating the Value of Offsite Data to Advertisers on Meta” [Job Market Paper; first author, with Anna Tuchman, Brad Shapiro, and Rob Moakler]
Abstract: We study the extent to which advertisers benefit from data that are shared across applications. These types of data are viewed as highly valuable for digital advertisers today. Meanwhile, product changes and privacy regulation threaten the ability of advertisers to use such data. We focus on one of the most common ways advertisers use offsite data and run a large-scale study with hundreds of thousands of advertisers on Meta. Within campaigns, we experimentally estimate both the effectiveness of advertising under business as usual, which uses offsite data, as well as how that would change under a loss of offsite data. Using recently developed deconvolution techniques, we flexibly estimate the underlying distribution of treatment effects across our sample. We find a median cost per incremental customer using business as usual targeting techniques of \$43.88 that under the median loss in effectiveness would rise to \$60.19, a 37% increase. Similarly, analyzing purchasing behavior six months after our experiment, ads delivered with offsite data generate substantially more long-term customers per dollar, with a comparable delta in costs. Further, there is evidence that small scale advertisers and those in CPG, Retail, and E-commerce are especially affected. Taken together, our results suggest a substantial benefit of offsite data across a wide range of advertisers, an important input into policy in this space.
Alekseev G, Amer S, Gopal M, Kuchler T, Schneider JW, Stroebel J, & Wernerfelt N (2022). “The Effects of COVID-19 on US Small Businesses: Evidence from Owners, Managers, and Employees.” Management Science.
Chetty R, Jackson MO, Stroebel J, Kuchler T, Hendren N, Fluegge R, Gong S, Gonzalez F, Grondin A, Jacob M, Koenen M, Laguna-Muggenburg E, Mudekereza F, Rutter T, Thor N, Townsend W, Zhang R, Bailey M, Barbera P, Bhole M, & Wernerfelt N. (2022). “Social Capital in the United States I: Measurement and Associations with Economic Mobility.” Nature.
Chetty R, Jackson MO, Stroebel J, Kuchler T, Hendren N, Fluegge R, Gong S, Gonzalez F, Grondin A, Jacob M, Koenen M, Laguna-Muggenburg E, Mudekereza F, Rutter T, Thor N, Townsend W, Zhang R, Bailey M, Barbera P, Bhole M, & Wernerfelt N. (2022). “Social Capital in the United States II: Exposure, Friending Bias, and the Determinants of Economic Connectedness.” Nature.
“Effects of Digital Interventions on COVID-19 Attitudes and Beliefs” [with Susan Athey, Kristen Grabarz, and Mike Luca; revise and resubmit at Proceedings of the National Academy of Sciences]
Abstract: During the course of the COVID-19 pandemic, a common strategy for public health organizations around the world has been to launch interventions via advertising campaigns on social media. Despite this ubiquity, little has been known about their average effectiveness. We conduct a large-scale program evaluation of campaigns from 174 public health organizations on Facebook and Instagram that collectively reached 2.1 billion individuals and cost around \$40 million. We report the results of 819 randomized experiments that measured the impact of these campaigns across standardized, survey-based outcomes. We find on average these campaigns are effective at influencing self-reported beliefs, shifting opinions close to 1% at baseline with a cost per influenced person of about $3.41. There is further evidence that campaigns are especially effective at influencing users’ knowledge of how to get vaccines. Our results represent, to the best of our knowledge, the largest set of online public health interventions analyzed to date.
“Designing Experiments with Synthetic Controls” [with Nick Doudchenko, Dave Gilinson, and Sean Taylor]
Abstract: Synthetic controls have become a powerful and standard component of the applied re- searcher’s toolkit. Research to date often takes the treated unit as fixed and conducts post-hoc analyses of different interventions. A common problem that has become increasingly relevant in applied work, however, is given a set of possible test units, how can a researcher select the best one(s) to experiment on? This paper develops an approach for answering this question with synthetic controls, leveraging simulated interventions and permutation tests across candidate test units. We also discuss frequent implementation issues that may arise in practice and how they can be addressed. Finally, using historical data from Facebook, we demonstrate the design and analysis of a country-level experiment and show the substantial gains from utilizing this approach. Our methodology is implemented in the open source R package
Work in Progress
Measuring Valuations of Digital Goods around the World: Evidence from Large-Scale Survey Data on Meta [with Erik Brynjolfsson, Avinash Collis, Daniel Deisenroth, Haritz Garro, Daley Kutzman, JJ Lee, and Asad Liaqat] Data collection finished.
- Description: We analyze self-reported valuations of a wide range of digital goods from nearly 100,000 incentivized survey responses in 15 countries.
Do Accuracy Nudges Reduce Misinformation on Facebook? [with Daniel Deisenroth, Haritz Garro, Adam Hughes, Dave Rand, and Jesse Shore] Data collection finished.
- Description: We ran a large-scale experiment across 30 million Facebook users where we randomized exposure to ads that nudged them to think about accuracy and measured effects on sharing of misinformation and factual content.
Digital Advertising Effectiveness for Small and Large Businesses: Evidence from Meta [with Jean-Pierre Dubé, Asad Liaqat, and Sarah Moshary]
- Description: We use a B2B survey for a large cross-section of firms to distinguish between “small business” advertisers and larger businesses. We then conduct large-scale AB advertising experiments on Meta to test whether small businesses, who lack the resources to use traditional advertising media, generate larger advertising elasticities and ROMI than larger businesses.
Wernerfelt N, Slusky DJG, & Zeckhauser R. (2017). “Second Trimester Sunlight and Asthma: Evidence from Two Independent Studies.” American Journal of Health Economics. 3(2): 227-253.
Dreber A, Rand DG, Wernerfelt N, Garcia JR, Lum JK, & Zeckhauser R. (2011). “Dopamine and Risk Choices in Different Domains: Findings Among Serious Tournament Bridge Players.” Journal of Risk and Uncertainty. 43: 19-38.
Rand DG, Pfeiffer T, Dreber A, Sheketoff R, Wernerfelt N, & Benkler Y. (2009). “Dynamic Remodeling of In-group Bias During the 2008 Presidential Election.” Proceedings of the National Academy of Sciences. 106 (15): 6187-6191.
Amsden Z, et al. (2019) “The Libra Blockchain.” Technical White Paper.
Wernerfelt N, & Zeckhauser R. (2010). “Denying the Temptation to GRAB.” Chapter in The Natural Resources Trap. Ed. W Hogan and F Sturzenegger. Cambridge, MA: MIT Press.
Wernerfelt N, Tarnita C, Rand DG, & Nowak M. (2009) “A Modular Approach for Analyzing Evolutionary Games in Networked Populations.” Harvard College Mathematics Thesis.