Campaign Recommendations

01The stakes

2020

Delivery is the machine learning system deciding where, when, and to whom an ad shows.

Recommendations were held to one job: surface where a campaign was losing performance, never sell a feature. Each one traced to what the delivery model could predict, ROAS or conversions, not to what the platform wanted to promote.

Accuracy wasn't the problem. Trust was.

Mid-flight recommendations were meant to help advertisers fix results while a campaign was still running. Shipping a new recommendation type was slow. Relevance was uneven. And even accurate advice died in workflows too complex to act on.

Before · bespoke per case

Every recommendation designed one-off
No shared design or content framework
Relevance hit or miss for advertisers
Complex workflows blocking adoption
Approvals on every detail

After · unified framework

01One framework any team can apply

02Feedback from advertisers, in the product

03Simpler workflows to act on advice

Any team ships without starting from zero.

An ignored recommendation isn't neutral. It spends the trust the next one needs.

The core tension both workstreams answered

02The workstreams

2020 to 2021

Operated across Product Marketing, Content Strategy, UX Research, Engineering, Data Science, and the Interfaces Platform team.

Two workstreams ran at the same time. I was the designer on both.

One was a framework, so a new recommendation didn't have to be designed from scratch every time. The other was a feedback system, so the recommendations could learn from being ignored.

Workflow types

Named five interaction patterns, Flow A through E. The names gave five teams one vocabulary.

Research

Ran concept studies asking what advertisers expected feedback to change, and whether a dismissed recommendation should ever return.

Framework

Built the design and content framework with the platform team, replacing per-case approvals with patterns any team could apply. After that, adding a recommendation didn't mean adding another round of approvals.

Feedback

Mapped lightweight advertiser feedback to signal the ranking system could read.

Fig. 01 · Before the framework. Cost scaled with the number of cases.

Every delivery case, rebuilt from zero

Lookalike drop-off Budget pacing Schedule gap + every other case

One-off design One-off content Manual approval

Shipsone at a time

Fig. 02 · Concept studies. The question they surfaced, whether a dismissed recommendation should ever come back, shipped as snooze.

We built the concepts to get a reaction

Projected impact shown up front The reason a recommendation surfaced Dismiss, with a reason attached

“Should a dismissed recommendation ever come back?”the question nobody had asked

Shipped as snooze

03The framework

2021

A dismissed recommendation could return with updated evidence and a “returning with new information” indicator: the cry-wolf problem, addressed directly.

Five workflow types, one framework.

The framework set how a recommendation looks, reads, and behaves. Then it ran against a live case: Lookalike audiences, meaning new people who resemble existing customers.

Fig. 03 · The workflow framework. Developed with the platform team.

One recommendation, five ways to act on it

A recommendationone framework: how it looks, reads, and behaves

Flow ABottom sheet in Account Overview

Flow BBottom sheet in the campaign table

Flow CEdit-panel flow

Flow DBulk edit in the table

Flow EIn-line edit

Fig. 04 · The drop-off case, live. A 2020 walkthrough: an audience drop-off surfaced mid-flight, with a guided path to act on it.

0:00

Fig. 05 · Resolution, automated. The same case through the simplified workflow: resolving the issue moves from a multi-step edit to a guided action.

0:00

Fig. 06 · Feedback expressions. Snooze, mute, and dismiss: advertiser feedback mapped to signal the ranking system could read. Shipped in 2021.

The first closed loop between behavior and what surfaces next

Recommendation shown

Advertiser respondsact · snooze · mute · dismiss

Mapped to signalplus adoption speed, time to action

Ranking learnswhich surface, and when

↻ a better next recommendation

Fig. 07 · Intent-aware feedback, in motion. The expressions respond to why a person is dismissing, so every dismissal finally carries a different signal.

0:00

5

Workflow types, one framework

↑

Resolution rate up, first live test

2021

Feedback expressions shipped

04What it outlasted

2021 onward

The framework’s job was other teams shipping without us.

Partner teams started building their own recommendations on the framework. Resolution rate went up in the first live test.

Every dismissal used to read the same to the ranking model. Snooze, mute, and dismiss split them apart, and snooze covered the thing nobody had designed for: an advertiser saying not now. That work became an engineering workstream on recommendation ranking, and it was still running after I moved on.

The framework any team could ship with.