Picture a hiker in dense fog. They can feel the ground beneath their feet and tell whether they are going uphill or downhill, but they cannot see the surrounding landscape. Using a simple strategy of always moving uphill, they will eventually reach the top of whatever hill they happen to be on. They will stand at the peak, satisfied that every direction leads down, and declare they have found the highest point.
But they might be standing on a foothill while the true summit towers above them a mile away. They reached a local maximum, the highest point in their immediate vicinity, but not the global maximum, the highest point overall. To reach the true summit, they would need to walk downhill first, crossing a valley before climbing the much taller peak.
This is the central dilemma of optimization. And it applies directly to A/B testing and product experimentation.
The Local Maximum Trap in Product Optimization
Most A/B testing programs focus on incremental improvement: change a headline, adjust a button color, reword a call-to-action, rearrange elements on the page. Each test makes a small modification and measures whether it performs better than the current version.
This approach works. Incremental tests produce reliable, measurable results. They are easy to implement, carry low risk, and generate steady compounding gains. Over months and years, a disciplined incremental testing program can meaningfully improve conversion rates, revenue, and user experience.
But incremental optimization has a ceiling. Each improvement makes the next one harder to find. The gains shrink as you approach the peak of your current hill. Eventually, you reach a point where no small change produces a detectable improvement. You have exhausted the local maximum.
At this point, teams often conclude that they have "optimized everything" and experimentation has reached its natural end. This conclusion is almost always wrong. They have optimized the current approach, the current design paradigm, the current user flow. But radically different approaches might perform dramatically better. They just cannot be reached through incremental changes.
Why Small Changes Cannot Cross Valleys
To understand why incremental testing gets stuck, consider the nature of the changes it produces. An incremental test modifies one element while keeping everything else constant. This means the variant is always very similar to the control. In the landscape metaphor, each step is very short.
If the path from your current position to a higher peak requires crossing a valley (a temporary performance decrease), no single small step will get you there. Every small step that moves toward the valley looks worse than staying put. Your testing framework, correctly, tells you the change is a loser. So you stay on your current hill.
Consider a concrete example. Your e-commerce checkout is a traditional multi-step process: cart review, shipping, payment, confirmation. You have optimized every step through dozens of incremental tests. Each step is polished. But a fundamentally different approach, a single-page checkout that combines all steps, might perform significantly better. You cannot reach the single-page design through small modifications to the multi-step flow. Each intermediate state (where you have removed one step but not the others) creates a confusing, broken experience that tests terribly.
The valley between the local maximum (optimized multi-step checkout) and the potential global maximum (optimized single-page checkout) is deep. Only a bold leap can cross it.
Exploitation vs. Exploration: The Fundamental Tension
This tension has a formal name in decision theory: the exploitation-exploration tradeoff. Exploitation means making the best use of what you already know. Exploration means seeking new knowledge that might lead to better outcomes.
Incremental A/B testing is exploitation. You are refining your current approach based on accumulated knowledge. Each test builds on what you have already learned about your users, your product, and your market. The expected value of each test is relatively predictable, and the risk is low.
Bold, transformative testing is exploration. You are venturing into unknown territory, testing approaches that might fail dramatically or succeed spectacularly. The expected value is uncertain, and the risk is high. But the potential upside is much larger than anything incremental testing can deliver.
Organizations that over-index on exploitation (only incremental tests) converge on their local maximum efficiently but miss potentially transformative improvements. Organizations that over-index on exploration (only radical tests) generate a lot of noise but fail to capitalize on their learnings. The optimal strategy blends both.
Signs You Are Trapped at a Local Maximum
How do you know if your optimization program is stuck? Several signals suggest you have reached the top of your current hill:
Declining win rates. If the percentage of tests producing statistically significant positive results is falling over time, you may be running out of incremental improvements to find.
Shrinking effect sizes. Even when tests do win, the lifts are getting smaller. Early in an optimization program, 10-20% lifts are common. At a local maximum, you are fighting for 1-2% improvements.
Test ideas feel trivial. Your backlog is dominated by minor variations: this shade of blue versus that shade of blue, this font size versus that font size. The strategic significance of each test has diminished.
Stakeholder fatigue. Leadership is losing interest in the experimentation program because the results feel marginal. The excitement of early wins has faded.
Competitor disruption. A competitor launches a fundamentally different approach and captures market share. They found a higher peak while you were polishing your current one.
What Bold Tests Look Like
Bold tests are not reckless. They are strategic experiments that test fundamentally different approaches rather than minor variations of the current approach. Here are characteristics that distinguish bold tests from incremental ones:
They change the paradigm, not the parameter. Instead of testing which headline converts better, test whether having a headline at all is the right approach. Instead of optimizing the pricing page layout, test a completely different pricing model.
They test user journeys, not page elements. Instead of modifying a single page, redesign the entire flow. Test a three-step onboarding versus a one-step onboarding, not just the copy on step two.
They challenge assumptions. Every product has implicit assumptions baked into its design: users need social proof, navigation should be at the top, forms should be short. Bold tests question these assumptions directly.
They accept short-term risk for long-term learning. A bold test might lose. It might lose badly. But even a dramatic loss generates valuable information about user behavior and preferences that incremental tests cannot reveal.
A Framework for Balancing Incremental and Transformative Tests
The optimal balance depends on your program's maturity, your traffic levels, and your competitive environment. But a practical starting framework is the 70-20-10 allocation:
70% incremental tests. These are your bread and butter: well-hypothesized, focused tests that refine the current experience. They produce reliable, predictable results and steady compounding improvement. They keep the program delivering value and maintaining stakeholder support.
20% moderate innovations. These tests make meaningful changes to structure, flow, or approach without completely reinventing the experience. They might test a new section on the homepage, a different email nurture strategy, or a revised information architecture. Higher risk than incremental tests, but also higher potential reward.
10% bold experiments. These are the moonshots: fundamentally different approaches that test your core assumptions. They have the highest failure rate but also the highest potential to discover a new peak. One successful bold experiment can deliver more impact than a year of incremental testing.
This allocation ensures you are continuously exploiting your current position while maintaining a pipeline of exploratory tests that might discover higher peaks.
Managing the Psychology of Bold Tests
Bold tests face organizational resistance for psychological reasons that are deeply human. Loss aversion makes stakeholders focus on the downside risk (what if it tanks?) rather than the upside potential. Status quo bias creates comfort with the current approach. Accountability structures punish visible failures more than they reward invisible missed opportunities.
Several strategies help manage these dynamics:
Frame bold tests as learning investments. The primary goal is not to win; it is to learn. Position the test as answering a strategic question rather than trying to beat the current design. This reframes a potential loss as valuable information rather than failure.
Limit exposure. You do not need to test a radical design change on 50% of your traffic. Run bold tests on smaller segments (10-20%) to limit the downside while still gathering enough data for statistical validity.
Build a portfolio narrative. Present your testing program as a portfolio, not a series of individual bets. Just as an investment portfolio includes some high-risk, high-reward positions alongside stable holdings, your testing program should include bold experiments alongside reliable incremental tests.
Celebrate the learning. When a bold test loses, invest time in understanding why. What did the results reveal about user behavior? What assumptions were proven wrong? These insights often become the foundation for future successful tests.
The Evolutionary Analogy
Nature solved the exploitation-exploration problem billions of years ago. Evolution operates through two mechanisms: selection (exploitation) which preserves and refines what works, and mutation (exploration) which introduces random variation that occasionally produces breakthrough adaptations.
Most mutations are harmful. They are the equivalent of bold tests that lose. But occasionally, a mutation produces a radical improvement that defines an entirely new species. Without mutation, evolution would stop at the first local maximum. Life would have never progressed beyond single-celled organisms.
Your product optimization program faces the same evolutionary pressure. Incremental testing refines what exists. Bold testing introduces the variation that can lead to breakthrough improvement. Both are necessary. Neither is sufficient alone.
Practical Steps to Escape Your Local Maximum
If you suspect your program is trapped at a local maximum, here are actionable steps:
Audit your test history. Plot win rates and average effect sizes over time. If both are declining, you are approaching your local peak.
List your assumptions. Write down every assumption embedded in your current design. Users prefer multi-step forms. Social proof drives conversion. Navigation must be visible at all times. Each assumption is a candidate for a bold test.
Study adjacent industries. Look at how companies in different verticals solve similar problems. A design pattern that works in travel booking might revolutionize your SaaS onboarding.
Talk to users who left. Your current design is optimized for people who tolerate it enough to stay. The people who bounced might have valuable perspectives on what a fundamentally better experience would look like.
Prototype before you test. Bold tests require more upfront investment. Use rapid prototyping and qualitative user testing to filter ideas before committing to a full experiment. This reduces the resource cost of exploration.
The most successful optimization programs in the world share a common trait: they combine the discipline of rigorous incremental testing with the courage to periodically question everything. The incremental tests pay the bills. The bold tests discover new summits. Together, they build products that continuously improve in ways their competitors cannot predict or replicate.