The power of persuasion modeling

How do we know that an advertisement or promotion truly drives incremental revenue? Michelangelo D'Agostino and Bill Lattner share their experience developing machine-learning techniques for predicting treatment responsiveness from randomized controlled experiments and explore the use of these persuasion models at scale in politics, social good, and marketing.

Traditionally, the data science community has overlooked causal inference. Often, the ability to predict outcomes—who will purchase what book, who will click which ad, who will vote for a particular candidate—is good enough. But how do we avoid showing ads to people who would have purchased the book anyway? How do we allocate resources to get-out-the-vote campaigns to maximally mobilize people who would not have voted otherwise? These problems fall under the domain of randomized controlled experiments and causal inference, where we need techniques to model the impact of applying a treatment on an outcome of interest. The individual-level predictions that come out of such models tell us how a particular person will respond to a particular ad or intervention and can be used for optimally assigning treatments to individuals. As a concrete example, imagine we would like to promote a book. We can group our potential audience into three groups: people who will purchase the book regardless of any promotion, those who will purchase the book for a slight discount, and those who won’t purchase the book regardless of discounts or promotions. Approaching this as a traditional machine-learning problem, we might try to build a model predicting promotion redemption or ad clicks, which would have us spending resources on people from both of the first two groups. Ideally, since people in the first group would buy the book anyway, we would like to exclude them from promotional activities. Doing this requires predicting two things: the likelihood a person would buy the book and the likelihood a person would buy the book after exposure to a promotion. This sort of modeling is variously known as persuasion modeling, uplift modeling, or heterogeneous treatment effects modeling. While there is a rich literature on persuasion modeling in the social sciences and marketing, such techniques are often unknown and underutilized in the machine-learning and data science communities. Likewise, techniques from the machine-learning and data science communities often don’t make their way back to the social science and marketing realms. Michelangelo D’Agostino and Bill Lattner share their experience developing machine-learning techniques for predicting treatment responsiveness from randomized controlled experiments and explore the use of these Michelangelo and Bill start with a summary of randomized controlled experiments and the persuasion modeling problem, covering both baseline and cutting-edge techniques for building these models, before presenting ways to do evaluation and model selection. Along the way, they’ll discuss several successfully executed case studies from their work at Civis Analytics.