Testing ad content with survey experiments

Brands that test the content of ads before they are shown to an audience can avoid spending resources on the 11% of ads that cause backlash. Using a survey experiment to choose the best ad typically improves effectiveness of marketing campaigns by 13% on average, and up to 37% for particular demographics. Patrick Miller explores data collection and statistical methods for analysis and reporting.

Brands spend lots of money on ads but rely on intuition to choose the ad to show to an audience and avoid ads that cause backlash. Data scientists can help marketing departments optimize their decision making by testing the content of ads before they are shown to audiences. Testing ad content can identify ads that cause backlash and help a brand to choose optimal markets and channels to show ads in. Survey experiments are a valuable tool for empirically testing ad content. A survey experiment uses data from survey respondents and randomized control trial design to directly estimate the effect of an ad on brand favorability and purchase intent. The methodology is similar to an A/B test but can be run on a small scale before a major launch. The analysis can be used estimate the effect of each on purchase consideration and brand favorability, identify audiences for which an ad is effective, and catch ads that cause backlash. These results of a survey experiment make marketing budgets more efficient, and avoid public relations disasters. Patrick Miller illustrates the importance of empirically testing ad content before an ad airs through experiments that tested highly controversial ads from Nike and Dove. Out of the hundreds of ads Civis tests each year, the company found about 11% of ads cause backlash. Further, choosing the best ad from several candidates results in a ~13% improvement in key metrics, and up to a 37% improvement for demographic subgroups. Topics include: