A few weeks ago I wrote about detecting whether people actually open Spotify after tapping through from your ad.
As promised, I put the idea to the test. My hypothesis was that enabling app detection would slow down the learning phase, but emerge cheaper on a cost-per-installed-app-user basis.
The short version:
So, based on these results, app detection is now enabled for every new SubmitHub Ad.
From October 12 to November 1, every new SubmitHub Ad was randomly assigned to one of two groups:
That's 660 ads and about $36,000 in ad spend, split down the middle.
The number I cared about is the cost per app open: how much you pay for each person who actually lands in a streaming app. That's what you want from a music ad. The rationale is that users with the app are way more likely to engage with your music.
Across all 660 ads, the ones with App detection paid about 10% less per app open. And more of the people tapping actually opened the app: 72% vs 62%.
The downside is that Meta takes longer to get out of the learning phase, because they hear about fewer conversions (only when the app opens). A typically ad needed roughly 30% more budget to get out of learning.
On day 1, App detection ads were actually about 3% more expensive. But once Meta figured out who to target, they pulled ahead, and the gap kept growing. By days 6-7 they were about 11% cheaper per app open.
Tier 3 countries are typically cheaper. They get lots of conversions, but often fail to actually convert on Spotify.
"Cheaper markets" = Mexico, Argentina, Chile, Colombia, Peru, Indonesia, India, the Philippines, South Africa and Turkey.
Meta also moved money around on its own. Ads with App detection spent about 22% less in Brazil and Mexico, and more in places like Japan and the US, where taps were turning into listeners.
Not much change in the US, UK, Canada, Germany and Japan. About 94% of the people tapping ads in those countries already open the app, so there's not much junk to filter out.
From what I can see the Brazil result is solid. The overall 10% is very likely real but not bulletproof - there were outliers on both sides. Either way, app detection came out cheaper on every one of the last two weeks of data, and the pattern makes sense: we're teaching Meta to find people who will really listen.
Just keep in mind that if you do enable this setting, your cost per conversion in Meta will look worse with App detection on. In the experiment it was about 40% higher in the first week. That's expected: we only report the app opens, so Meta sees fewer conversions. The taps that didn't open an app are still happening, we're just not counting them as wins for Meta.
Thanks to everyone who ran ads during the experiment. You helped make everyone's ads a little smarter 🤓