🥊 Quick Strike
Lift retention without quietly costing revenue
You move an onboarding step, a reward, a difficulty curve or an ad placement. Sessions go up, so you ship it. Two weeks later revenue per user is down and players are leaving earlier in the lifecycle. Three lines written before launch would have caught that.
THE HIT
Name the one metric that settles it. Firebase calls this the goal metric, the single number that decides the winning variant. Write it down with its cohort and its measurement window before launch. Amplitude separates N-day retention, where a user returns on exactly day N, from unbounded retention, where they return on or after day N. Picking which one you mean up front stops the test being graded twice.
Source: Amplitude, Cohort retention analysis
Add the guardrail that would show the damage. Firebase lets you track up to five non-goal metrics alongside the goal, and the list includes crash-free users, purchase revenue and estimated ad revenue. Pick the one your change is most likely to hurt. A retention lift that arrives with a revenue drop is worth catching while the test is still running.
Source: Firebase, Create a Remote Config experiment
Write the stop rule before you see the chart. Firebase gives you a Stop experiment control, and recommends two weeks as the minimum runtime for a typical Remote Config experiment. Deciding in advance what number halts the test keeps a scary first day from killing something that needed more time, and keeps a bad result from being argued into a rollout.
Source: Firebase, A/B testing concepts
DO THIS TODAY (10 minutes)
Open the brief for the experiment you are running right now.
Add three lines to it: Win, Guardrail, Stop rule.
On the Win line, write one metric with its cohort and its measurement window.
On the Guardrail line, write the revenue or stability metric your change could damage, and the number that makes you halt.
A better test is usually a simpler one that knows when to stop.
See you Tuesday,ChurnJitsu by AppQ