The read-out meeting is on a Thursday, because it is always on a Thursday.
The experiment ran eight weeks — a generous window by the standards of a live-service calendar, long enough that two content drops fell inside it and had to be argued about in the analysis. The intervention was reasonable: a gentler on-ramp for returning players, someone’s conviction, six months of a designer’s life. The retention delta lands on the screen at four decimal places, hugging zero, confidence interval straddling it like a man standing over a crack in the pavement. Someone says the words no detectable effect. The feature is shelved by the end of the meeting, the deck is filed, and the team moves to the next candidate on the roadmap — which will get its own eight weeks, its own Thursday, its own crack in the pavement. The process is rigorous. Everyone in the room is doing their job well. And the process has just, quietly, possibly, thrown away a thing that worked — because it worked at a speed the window cannot see.
Here is the shape of the problem, and we can state it almost without numbers. An experiment window is a filter. It passes effects that develop quickly and attenuates effects that develop slowly, the way a fast shutter freezes the sprinter and erases the glacier. Now put the two churns side by side — the two ways a player leaves that we have written about before. Friction is fast: the sour update, the degraded match, the queue that grew — cause on Tuesday, exit by Friday, fingerprints everywhere. Satiation is slow: the loop delivering, and delivering, and meaning a little less each month, until an ordinary evening when the marginal session stops being worth the thumb. One of these fits inside a quarter. The other one is the quarter, and the quarter after that, accruing at a rate the dashboard reads as ordinary decay.
Run every decision through the same eight-week shutter and you do not get a neutral sample of your own causal structure. You get a portrait of the fast causes, sharpened, and the slow causes, gone — a systematic bias, baked into the evidence base, that no individual analysis can see because it lives in the design of all of them. The experiments are each sound. The archive they accumulate into is skewed. Ask that archive what drives churn and it will answer friction, in good faith, with p-values — because friction is what the shutter could catch. The archive has never once seen satiation arrive, develop, and take a player, start to finish, inside a single test window. So satiation appears in the org chart’s shared model of the game as a rumor, a theorist’s worry, a thing the numbers never quite confirm — while it works through the base at its own pace, unphotographed.
The cruelty runs in both directions, and the shelved on-ramp is the other half. An intervention aimed at the slow churn — pacing, generosity, room to breathe — spends its effect over months. Test it for eight weeks and you have measured the first centimeter of a long curve and called it flat; the null result is a fact about the window wearing the costume of a fact about the feature. I think of the archive of dead experiments every studio keeps, the features that “didn’t move retention,” and I wonder how many were killed at the exact moment a longer instrument would have watched them begin to work. The roadmap is steered by these verdicts. Quarter by quarter, verdict by verdict, the steering compounds: bets on the fast and visible, divestment from the slow and real. Nobody chose this. The cadence chose it, and the cadence was chosen for reasons — planning cycles, ship dates, the reasonable human need to decide things and move on — that have nothing to do with the timescales on which players actually change.
There is a way through, and what it costs is patience. The window is an instrument, and instruments can be built to the phenomenon — which has never heard of the fiscal quarter. Long-horizon holdouts that persist across quarters. Cohorts followed for the months the drift actually takes, with the drift itself — the slow reshaping of a player’s behavior under the system’s steady pull — promoted from noise floor to measured object. Staggered designs that let a slow effect mature in one arm while the business keeps its cadence everywhere else. None of this is exotic; it is the ordinary discipline of matching the measurement to the process, applied to processes a quarterly rhythm has trained us not to look at. The hard part is organizational, and it is honest to say so — a longer window means telling a room full of people with a roadmap that the answer arrives in March.
But the players are already running the long experiment, whether we instrument it or not. The base you have next year is the readout. The only question is whether anything was measuring while it happened — or whether the Thursday meetings will go on returning their crisp, well-powered verdicts on everything fast, while the slowest and most expensive change in your game files itself, one ordinary evening at a time, under noise.
The Shutter is this argument as an interactive demonstration — set the window, watch which fix ships.