Solved: How to test new ads with inconsistent results?

Hi there,

Thanks for reaching out!

That's a really common and frustrating problem you've described. Honestly, it's one of the biggest challenges people face with Meta ads. You're trying to make smart, data-driven decisions, but the platform's performance feels like it's changing with the wind. It's easy to lose confidence and start second-guessing every move you make. You're right to be wary – turning off a potential winner just because it launched during a "bad week" is a massive waste of potential profit, and I've seen it happen countless times.

The good news is that there's a way to manage this chaos. The solution isn't about trying to predict Meta's daily mood swings, because you can't. It's about building a robust advertising framework that is resilient to this volatility and allows you to test new ads with confidence. It's less about finding a single 'magic' ad and more about creating a system that consistently surfaces winners over time, regardless of the platform's short-term fluctuations. I'm happy to give you some initial thoughts on how you might approach this, based on my experience managing these kinds of campaigns.

We'll need to look at why this inconsistency is happening...

First off, let's just accept an uncomfortable truth: Meta's ad platform is inherently unstable. It's not a machine where you put £1 in and get a predictable £5 out every single day. It's a massive, dynamic auction with millions of advertisers competing for the attention of billions of users whose behaviour changes constantly. Factors like seasonality (think Black Friday vs. a random Tuesday in February), increased competition in your niche, or even just changes in the algorithm can cause those performance dips you're seeing. It's not necessarily something you've done wrong; it's just the nature of the beast.

The core issue is that many advertisers react to this volatility by making rapid, panicked changes. They see a bad day, they tweak the audience. They see two bad days, they kill the ad. Every time you do this, you're potentially resetting the 'learning phase' for that ad set, where the algorithm is trying to figure out who best to show your ads to. You end up in a constant state of flux, never giving anything enough time to find its footing. The goal is to move from being reactive to being systematic.

The problem you've described, where your 10x ROAS ad had bad weeks, is the perfect illustration. Your instinct to kill it would have been wrong, but it's an understandable one. The key is to create a structure that allows you to ride out those bad weeks and make decisions based on meaningful data, not short-term noise. It's about controling what you can (your testing method, your campaign structure, your metrics) to mitigate the impact of what you can't (the algorithm's daily whims). We need to build a system that tells you if an ad is failing because it's genuinely a poor performer, or because the whole account is just having a wierd week.

I'd say you need a proper campaign structure...

Before we even get to testing individual ads, the foundation of your entire account needs to be solid. A disorganised account with ads and audiences all over the place is like trying to conduct a scientific experiment in the middle of a hurricane. You can't isolate any variables. The most resilient structure I've found, and the one we implement for our clients, is based on the marketing funnel: Top of Funnel (ToFu), Middle of Funnel (MoFu), and Bottom of Funnel (BoFu).

This isn't just marketing jargon; it's a practical way to seperate your audiences and objectives, which creates stability. You treat cold audiences differently to warm audiences.

Top of Funnel (ToFu) - Your Prospecting Engine

This is where you find new customers. It's your cold outreach. Because these people have never heard of you, this is naturally the most volatile part of your funnel, and it's where the majority of your budget and testing should happen. The goal here is to use one single, long-running Campaign Budget Optimisation (CBO) campaign. CBO is important because it lets Meta's algorithm automatically allocate your budget to the best-performing ad set within that campaign. You're letting the machine do the heavy lifting of finding the cheapest conversions on any given day.

Inside this ToFu CBO campaign, you create seperate ad sets for your different audience tests. Based on what I've seen work, you should prioritise them like this:

-> 1. Detailed Targeting: Start here. This is your bread and butter. Group related interests, behaviours, and demographics into thematic ad sets. For instance, if you sell high-end coffee beans, one ad set might target interests like 'James Hoffmann', 'Specialty Coffee Association', and 'Aeropress'. Another might target people who like premium kitchenware brands like 'Sage' or 'Fellow'. The key is to pick interests that are specific to your target audience. Targeting something broad like 'Coffee' is a waste of money because it includes everyone from instant coffee drinkers to connoisseurs.
-> 2. Lookalike Audiences: Once you have enough data (you need at least 100 conversions, but honestly, you want more like 500-1000 for it to be potent), you can create lookalikes. The quality of your source audience is everything. A lookalike of your 'highest value previous customers' will almost always outperform a lookalike of 'all website visitors'. You should test these in seperate ad sets, starting with a 1% lookalike and then expanding if it works well.

By keeping these in one CBO campaign, you can clearly see which audience "theme" is performing best over time, seperate from the performance of the ads inside them.

MoFu/BoFu - Your Retargeting Safety Net

This is where you recapture people who have shown interest but haven't bought yet. These are your warmest audiences, and performance here should be much more stable and profitable. This is your safety net that often props up the account's overall ROAS during prospecting volatility.

Again, you'd use a single, seperate CBO campaign for all your retargeting. Inside it, you can have ad sets targeting different points of the funnel:

-> BoFu (Bottom of Funnel): People who are very close to converting. This is your highest-intent audience. You'd group audiences like 'Added to Cart (7 Days)', 'Initiated Checkout (14 Days)', but exclude 'Purchased (180 Days)'.
-> MoFu (Middle of Funnel): People who have shown some interest. This could be an ad set grouping 'Viewed Content/Product Page (30 Days)', 'Instagram Engagers (60 Days)', and '75% Video Viewers (90 Days)'.

If your budget is smaller, you can definately combine all these retargeting audiences into a single ad set to ensure it gets enough budget and exits the learning phase. The ads here should be different, maybe focused on overcoming objections, showing customer reviews, or offering a small incentive to complete the purchase.

Here’s a simplified view of what that structure looks like:

This structure alone will bring a huge amount of clarity. When performance dips, you can now ask better questions. Is it just my ToFu campaign that's struggling while retargeting holds strong? Or is the whole account down? This context is everything.

You probably should rethink your creative testing process...

With a stable structure in place, we can now address your main question: how to test new ads without being fooled by randomness. The common advice to launch a new ad and kill it if it's not performing within 48-72 hours is, frankly, terrible advice for most advertisers. It completely ignores the learning phase and platform volatility.

My approach is different. The goal is to introduce new creatives into a controlled environment and give them a fair chance to succeed. Here’s how I would do it:

1. Test Within a Winning Ad Set: Take your best-performing ToFu CBO ad set—the one that consistently delivers the best results over time. This is your "control" environment. It already has winning ads running in it (your "control" ads).

2. Introduce the Challenger: Simply add your new creative (your "challenger" ad) into this existing, winning ad set. Don't create a new campaign or a new ad set just for one test. Adding it to an existing ad set that is already out of the learning phase gives it the best possible start. Meta's algorithm will start feeding it a small amount of impressions to see how it performs against your existing winners.

3. Be Patient. Really Patient: This is the hardest part. You need to let it run. Don't look at it for at least 3-4 days. Then, check in, but don't make any decisions yet. You need to let that new ad accumulate meaningful data—I'm talking thousands of impressions and hundreds of clicks. For an ad to be judged fairly, I'd want it to spend at least 1x, ideally 2-3x your target Cost Per Acquisition (CPA), before making a call. If your target CPA is £50, the ad needs to spend £50-£150 before you can say with any confidence whether it's working or not.

4. Judge on Trends, Not Dailies: When you do analyse it, ignore the daily fluctuations. Look at the performance over the entire testing period (e.g., the last 7 or 14 days). Compare its primary metric (e.g., Cost Per Purchase) directly against the other ads in that same ad set. If it's performing on par with, or better than, your existing winners, you keep it. If it's significantly worse after spending 2-3x your target CPA, then you can confidently turn it off, knowing you gave it a fair shot. You now have data, not a guess.

This systematic approach removes the guesswork. You're no longer asking "is this ad bad or is it a bad week?". You're asking "how does this new ad perform relative to my proven winners, within the same audience, over a meaningful period of time?". It’s a much better question to ask.

I remember one e-commerce client selling subscription boxes. They were constantly churning through creatives, killing them after two days. We implemented this exact structure. Their first new ad looked like a failure for 4 days, but we held our nerve. By day 10, the algorithm had found its pocket and it became their best-ever ad, eventually helping them to a 1000% Return On Ad Spend. Patience pays off.

You'll need to focus on the right metrics to diagnose the real problem...

Your focus on ROAS is correct – it's the ultimate measure of success. But when you're testing and diagnosing problems, ROAS can be a 'lagging indicator'. It's the final outcome of a long chain of events. If your ROAS is low, you need to look further up the funnel to understand why. Staring at your ROAS number going up and down won't tell you what to fix. You need to become a detective and follow the data.

Here's how I'd break it down, using the metrics available in Ads Manager:

-> Problem: Low CTR (Click-Through Rate) / High CPC (Cost Per Click). This means people are seeing your ad but not clicking. The ad is not grabbing their attention.
Diagnosis: The problem is with your creative (the image/video) or your hook (the first line of copy). It’s failing the "scroll-stop" test.
Solution: Test new images or videos. Test completely different opening lines in your copy. The ad itself is the issue.

-> Problem: High CTR, but low Landing Page Views. People are clicking, but they're not even making it to your website.
Diagnosis: Your website is probably too slow to load. People are impatient; if the page doesn't load in a couple of seconds, they're gone.
Solution: Work on your site speed. Optimise images, use a faster hosting service. This is a technical issue, not an ad issue.

-> Problem: Lots of Landing Page Views, but very few 'Adds to Cart'. You're getting relevant traffic to your product page, but they aren't taking the next step.
Diagnosis: The problem is on your product page. It's failing to convince them. It could be poor product photos, a weak product description, unclear pricing, or a lack of trust signals like reviews.
Solution: Improve your product page. Get professional photography. Write more persuasive copy that focuses on benefits, not just features. Add customer testimonials. Test different price points or special offers.

-> Problem: Lots of 'Adds to Cart', but very few Purchases. People are almost there, but something is stopping them at the final hurdle.
Diagnosis: The problem is in your checkout process. It might be unexpected shipping costs (the number one reason for cart abandonment), a complicated checkout form, or not enough payment options.
Solution: Be transparent about shipping costs upfront. Simplify your checkout to as few steps as possible. Offer multiple payment options like PayPal, Apple Pay, etc.

By analysing the funnel this way, you stop guessing. The data tells you exactly where the hole in your bucket is. You could have the best ad in the world, but if your product page is unconvincing, your ROAS will always be terrible. The ad did its job (got the click), but the website failed to do its job (get the sale).

This is the main advice I have for you:

Running ads successfully isn't just a tactical exercise of launching and killing creatives. It requires a strategic foundation. Getting that foundation right is what separates the advertisers who get consistent results from those who are perpetually frustrated by inconsistency. Here's what I believe your focus should be:

I know this is a lot to take in, and it's a very different way of thinking about ad management than most people are used to. It requires discipline and a shift in mindset from 'quick wins' to 'long-term system building'. But this is precisely the kind of strategic oversight that can transform an account from being a source of frustration into a predictable engine for growth.

Putting these systems in place, correctly interpreting the data, and consistently coming up with new creative angles to test is a full-time job in itself. This is where expert help can make a significant difference. We've seen this exact framework turn around accounts that were on the brink of being shut down. We helped one medical SaaS client, for example, reduce their cost per user from a crippling £100 down to just £7 by implementing this exact structured approach. For another B2B software client, it helped us get 4,622 registrations at just $2.38 each.

If you'd like an expert pair of eyes to look over your account and discuss how these principles could be applied directly to your business, we offer a free, no-obligation strategy session. We can walk through your current setup and give you some actionable advice on the spot. It might be helpful to get that outside perspective.

Hope this helps!

Regards,

Team @ Lukas Holschuh