10 High-Impact A/B Tests You Can Launch in Optimizely This Week
Not all A/B tests are equal. Here are 10 experiments with tight behavioral hypotheses, realistic lift expectations, and the exact failure modes to watch out…
Read Article→
Free Learning · Better Decisions Make Growth Decisions
Make Growth Decisions
You Can Defend
Know what to test, when to trust the result, and what to do next. Practical guides for analysts, growth teams, and founders.
A/B vs Multivariate vs Multi-Page Testing: Which Should You Run?
The wrong test type is one of the most common ways CRO programs waste months.
Read Article→
Why You Can Never Change a Running Experiment (And What To Do Instead)
Someone changed your live A/B test. Maybe it was you. Here's exactly what that broke, why the data is compromised, and the step-by-step rescue workflow to…
Read Article→
How Long Should You Run an A/B Test? (The Real Answer)
Seven years running 100+ experiments taught me that test duration is the most violated rule in CRO.
Read Article→
Minimum Detectable Effect (MDE): The Most Important Number You're Not Setting Correctly
MDE isn't a calculator input — it's the foundation of your entire experiment design.
Read Article→
Bayesian vs Frequentist Testing in Optimizely: Which Should You Choose?
Optimizely now offers three statistical engines: Sequential (Stats Engine), Frequentist Fixed Horizon, and Bayesian.
Read Article→
Statistical Significance in Optimizely: What It Really Means
How Optimizely calculates statistical significance, what 95% actually tells you, and the common misreadings that cost teams real money.
Read Article→
Why Your Optimizely Results Keep Changing (And When to Worry)
Tuesday your experiment shows 94% confidence. Friday it's 71%. Nothing changed — so what's happening?
Read Article→
False Discovery Rate in Optimizely: Why Running Many Tests Simultaneously Is Riskier Than You Think
Running 20 tests at 95% confidence means you expect at least one false positive by chance.
Read Article→
Why Your Optimizely Test Isn't Statistically Significant
Five reasons Optimizely experiments stall below statistical significance — sample size, MDE, traffic allocation — and the fix for each one.
Read Article→
Homepage Testing Is Overrated: Evidence from 16 Homepage Experiments
16 homepage A/B tests exposed a 69% inconclusive rate — worse than any other page type. Data shows downstream pages win at 2x the rate.
Read Article→
Why Your Pricing Page A/B Test Failed: Lessons from 13 Pricing Experiments
We ran 13 pricing page A/B tests with a 15% win rate. Here are the counterintuitive lessons about why pricing psychology fails in practice.
Read Article→
Why 61% of A/B Tests Are Inconclusive — And Why That’s Exactly Right
Most A/B tests don't produce winners. Our data from 97 experiments reveals why a 61% inconclusive rate signals a rigorous program, not a broken one.
Read Article→
The Social Proof Experiment That Failed: Why Evidence Beats Intuition
We ran 3 social proof A/B tests and got 0 winners. Here is why the most recommended conversion tactic failed and what actually works instead.
Read Article→
Mobile vs. Desktop: 13 Experiments That Challenge Responsive Design Assumptions
Analysis of 13 mobile A/B tests reveals a 38% win rate, beating desktop. Learn why device-specific testing matters more than responsive design alone.
Read Article→
Behavioral Science Claims vs. Experiment Reality: What Actually Moves the Needle
97 real A/B experiments tested behavioral science principles in the field.
Read Article→
The Hidden Cost of Running Zero Experiments: A Business Economics Analysis
Most companies quantify testing costs but never calculate what NOT testing costs. The Experiment P&L framework reveals the true economics.
Read Article→
Product Comparison Pages: 19 Experiments Reveal What Actually Converts
19 A/B tests on product comparison pages reveal layout structure beats content additions, attribute curation drives decisions, and interactive builders…
Read Article→
CUPED Variance Reduction in A/B Testing, Explained
How CUPED uses pre-experiment data to cut A/B test duration by 20–50%, where it works (and where it doesn't), and how to start using it.
Read Article→
A/B Testing in E-Commerce: Funnel Optimization That Actually Moves Revenue
A practical guide to A/B testing across the e-commerce funnel — from category pages to checkout.
Read Article→
A/B Testing Segmentation: Why One-Size-Fits-All Results Are Lying to You
Learn why aggregate A/B test results hide the truth. Master segmentation analysis, understand heterogeneous treatment effects, and avoid the segment fishing trap.
Read Article→
Canary Deployment vs A/B Testing: What's the Difference?
Canary releases, feature flags, and A/B tests solve different problems. When to use each — and why a 10% rollout is not an experiment.
Read Article→
Political A/B Testing: What Campaign Optimization Teaches Product Teams
From Obama's $60M fundraising lift to modern campaign optimization, political A/B testing is a masterclass in high-stakes, time-constrained experimentation…
Read Article→
The Tradeoffs of A/B Testing: When Testing Costs More Than It Teaches
A/B testing isn't free. Learn the real costs — opportunity cost, engineering resources, decision delay — and develop the judgment to know when shipping fast…
Read Article→
Better Decisions
Make Growth Decisions
You Can Defend
Know what to test, when to trust the result, and what to do next. Practical decision guides for analysts, growth teams, and founders. Free. Weekly.
Opens Substack to confirm your subscription · Free · Unsubscribe anytime