Skip to main content

Reference · Glossary

Behavioral Science
& Experimentation
Glossary

Expert-level definitions from a practitioner who's run the experiments. Not textbook theory — applied behavioral science with real A/B test context.

66 Terms Defined

Browse by Category

A/B Testing

2 terms

Conversion Rate Optimization

17 terms

Statistics & Methodology

14 terms

Experimentation Strategy

9 terms

A–Z Index

A

A/B Testing A randomized experiment comparing two or more versions of a page or feature to determine which perfo... A/B Testing Above the Fold The portion of a webpage visible without scrolling — the most valuable real estate for capturing att... Conversion Rate Optimization Anchoring Bias A cognitive bias where people rely too heavily on the first piece of information they encounter when... Behavioral Economics Average Order Value The average dollar amount spent per transaction — total revenue divided by number of orders — a crit... Conversion Rate Optimization

B

Bayesian A/B Testing An alternative to frequentist A/B testing that uses probability distributions to express uncertainty... Statistics & Methodology Bounce Rate The percentage of visitors who leave a webpage without taking any action — no clicks, no scrolls, no... Conversion Rate Optimization

C

Choice Overload The phenomenon where having too many options leads to decision paralysis, reduced satisfaction, and ... Behavioral Economics Click-Through Rate The percentage of users who click on a specific link, button, or call-to-action out of the total num... Conversion Rate Optimization Cognitive Fluency The ease with which information is processed — fluent (easy) experiences feel more trustworthy, fami... Behavioral Economics Cognitive Load The total amount of mental effort required to process information and make decisions — the invisible... Behavioral Economics Completion Bias The innate human drive to finish what we've started — incomplete tasks create cognitive tension that... Behavioral Economics Confidence Interval A range of values that likely contains the true effect of your A/B test variation — more informative... Statistics & Methodology Conversion Rate Optimization The systematic practice of increasing the percentage of visitors who complete desired actions — usin... Conversion Rate Optimization Cost Per Acquisition The total cost required to acquire one converting customer, calculated by dividing total campaign sp... Conversion Rate Optimization Customer Lifetime Value The predicted total revenue a customer will generate over their entire relationship with a business ... Conversion Rate Optimization

D

Decoy Effect A phenomenon where adding a third, inferior option changes preferences between the original two opti... Behavioral Economics Default Effect The powerful tendency for people to accept pre-selected options — making default settings one of the... Behavioral Economics

E

Endowment Effect The tendency for people to overvalue things they already own or feel ownership over, compared to ide... Behavioral Economics Exit Rate The percentage of sessions that end on a specific page, calculated as page exits divided by total pa... Conversion Rate Optimization Experiment Documentation The practice of recording experiment hypotheses, designs, results, and learnings in a structured for... Experimentation Strategy Experiment Velocity The rate at which an organization runs experiments — typically measured in tests per month or quarte... Experimentation Strategy Experimentation Culture An organizational mindset where decisions are driven by evidence, hypotheses are tested before shipp... Experimentation Strategy

F

Feature Flagging A software development practice that allows teams to enable or disable features for specific user se... Experimentation Strategy Form Optimization The practice of improving web forms — in design, copy, field order, and validation logic — to increa... Conversion Rate Optimization Framing Effect A cognitive bias where people react differently to the same information depending on how it is prese... Behavioral Economics Funnel Analysis The process of mapping and measuring user progression through a series of steps toward a conversion ... Conversion Rate Optimization

G

Goal Gradient Effect The tendency for people to increase effort as they get closer to completing a goal — acceleration to... Behavioral Economics Guardrail Metrics Protective metrics monitored during experiments to ensure that improving the target metric doesn't c... Experimentation Strategy

H

Heatmap Analysis A visual analytics method that uses color-coded overlays to show where users click, scroll, move the... Conversion Rate Optimization Hick's Law The time it takes to make a decision increases logarithmically with the number and complexity of cho... Behavioral Economics Holdout Testing A method for measuring the cumulative impact of all shipped experiments by withholding changes from ... Experimentation Strategy Hyperbolic Discounting The tendency to prefer smaller, immediate rewards over larger, delayed rewards — with the preference... Behavioral Economics Hypothesis Testing The practice of formulating a specific, falsifiable prediction before running an experiment — the fo... Experimentation Strategy

I

ICE Scoring Framework A prioritization framework that scores experiment ideas by Impact, Confidence, and Ease — each on a ... Experimentation Strategy IKEA Effect The cognitive bias where people place disproportionately high value on things they helped create, ev... Behavioral Economics

L

Landing Page Optimization The systematic process of testing and improving landing page elements — copy, layout, CTAs, trust si... Conversion Rate Optimization Loss Aversion The psychological tendency for people to prefer avoiding losses over acquiring equivalent gains — lo... Behavioral Economics

M

Mere Exposure Effect The psychological phenomenon where people develop a preference for things simply because they've bee... Behavioral Economics Minimum Detectable Effect The smallest difference between control and variation that your test is designed to reliably detect ... Statistics & Methodology Multivariate Testing An experiment that tests multiple variables simultaneously to find the optimal combination, rather t... A/B Testing

N

North Star Metric The single metric that best captures the core value your product delivers to customers — used to ali... Experimentation Strategy Novelty Effect The temporary increase in engagement or conversions that occurs simply because something is new and ... Statistics & Methodology Nudge Theory A concept in behavioral science that proposes designing choice environments to influence decisions w... Behavioral Economics

P

p-value The probability of observing results at least as extreme as the actual results, assuming there is no... Statistics & Methodology Paradox of Choice The counterintuitive finding that having too many options leads to decision paralysis, lower satisfa... Behavioral Economics Peak-End Rule People judge experiences based on the most intense moment (peak) and the final moment (end), not the... Behavioral Economics Personalization The practice of tailoring website content, offers, and experiences to individual visitors based on t... Conversion Rate Optimization Practical Significance Whether an A/B test result is large enough to matter for the business — distinct from statistical si... Statistics & Methodology Progressive Profiling A lead capture strategy that collects user information incrementally across multiple interactions ra... Conversion Rate Optimization

R

Reciprocity Principle The social norm that compels people to return favors — when you give something of value first, peopl... Behavioral Economics Regression to the Mean The statistical phenomenon where extreme measurements tend to be followed by measurements closer to ... Statistics & Methodology Return on Ad Spend The ratio of revenue attributed to advertising spend to the cost of that advertising, expressed as a... Conversion Rate Optimization Revenue Per Session Total revenue divided by total sessions — a more comprehensive optimization metric than conversion r... Conversion Rate Optimization

S

Sample Size The number of visitors or users needed in each variation of an A/B test to detect a meaningful diffe... Statistics & Methodology Satisficing A decision-making strategy where people choose the first option that meets their minimum acceptable ... Behavioral Economics Scarcity Bias The tendency to place higher value on things that are scarce or perceived as limited in availability... Behavioral Economics Selection Bias A systematic error that occurs when the sample of users in an experiment is not representative of th... Statistics & Methodology Sequential Testing A statistical approach that allows experimenters to check results continuously as data accumulates, ... Statistics & Methodology Session Recording A qualitative analytics tool that captures and replays individual user sessions, showing exactly how... Conversion Rate Optimization Social Proof The psychological tendency to look to others' actions and opinions when deciding what to do — especi... Behavioral Economics Statistical Power The probability that a test will correctly detect a real effect when one exists — the complement of ... Statistics & Methodology Statistical Significance The probability that an observed difference between test variations is not due to random chance — ty... Statistics & Methodology Status Quo Bias The preference for the current state of affairs, where any change from the baseline is perceived as ... Behavioral Economics Sunk Cost Fallacy The tendency to continue investing in something because of previously invested resources (time, mone... Behavioral Economics

T

Type I Error A false positive — concluding that a test variation is a winner when the observed difference is actu... Statistics & Methodology Type II Error A false negative — concluding that a test variation has no effect when it actually does have a real,... Statistics & Methodology