Go back

The Daily Carnage

may 11, 2025

Apples to Oranges

Meta A/B tests are skewed.

Be in The Know

🔍 Is “keywordless search” almost here?

💰 TikTok pledges $1M in ad credits for U.S. small businesses.

📈 Meta unveils “Trending” ad formats for Reels and Threads.

🎧 Spotify’s podcast “plays” are a new metric for a booming industry.

📊 Google Ads launches fancy new charts for reporting.

🧠 The marketing strategy behind the Internet’s favorite green menace.

🥤 Mountain Dew is showing off a new logo and an ’80s-inspired jingle.

🍫 Hershey’s and ADT launch anti-theft chocolate bars in Brazil.

Your Meta Ads A/B Tests Might Be Misleading. Here’s Why.

If you’re using A/B tests on Meta (Facebook, Instagram) to choose your winning ad creative, there’s a catch: you’re not running a true A/B test.

That’s the conclusion of a 2024 study by researchers at Southern Methodist University and the University of Michigan, which found that Meta’s algorithm skews delivery based on audience responsiveness, showing different ads to different types of people.

This means your test isn’t just comparing Creative A vs. Creative B. It’s comparing Creative A to one group of people and Creative B to another. And often, those audiences vary significantly—by gender, interests, or behavior—without your realizing it.

So while Meta is great at finding the most efficient combo of ad + audience, it’s not telling you which creative is objectively better. That’s a big deal if you’re trying to use those results to inform decisions beyond Meta… like what to run on TikTok, YouTube, or even in your email subject lines.

So, keep this in mind:

✅ Use Meta A/B tests only to optimize Meta campaigns.

❌ Don’t assume Meta results translate to other platforms or channels.

🧪 Run platform-specific tests—Facebook vs. TikTok vs. YouTube.

🗣️ Use surveys, interviews, or on-site tests to validate messaging or product ideas.

🔁 Use A/A testing to double-check the reliability of your results.

Test accordingly! Check out the research in the full post at Science Says.

Read moreSign up

Q for You

Are you interested in news and stories related to Upfronts and Newfronts events?

Psst! You need to be a Daily Carnage subscriber to participate in this poll.

Enter your email below to subscribe or verify your existing subscription

Subscribe to The Daily Carnage

Get a daily curation of marketing news, insights, tools, and culture—delivered straight to your inbox.

VERT

VERT is, you guessed it, a file converter.

No file size limit, no ads, all open source.

Image, audio, and document conversions happen right on your device, so your files never leave your hands.

Just drag and drop to convert images, audio, videos, and documents in nearly every major format—from PNG to MP3, DOCX to MKV.

Try ItSign up

On-Demand Sponsorships

Book a Classified Ad at your convenience.

For $99, you get 200 characters to share your stuff right in this spot.

Our on-demand process lets you select your date, drop in your creative, complete your payment, and get on our calendar ASAP.

Book nowSign up

Humanizing the Vision Pro

Apple A Gift for Mom

If you’re a late adopter who bristles at the sight of a Vision Pro headset, Apple is coming for you.

The brand’s short Mother’s Day film features a real couple, Sam and Khulan, who captured their son’s first year of life using iPhone’s spatial video feature.

While Mom watches a year of motherhood play back in 3D from her headset, we watch tears roll down her cheeks.

WatchSign up
Ads From the Past

Ads from the Past

Sony, 1975

Sony, 1975

Read more

Get the best daily marketing newsletter in your inbox