The self-experiment problem: why an n-of-one result rarely settles anything

What a single-subject design can and cannot do

A self-experiment can answer a question about one person, which is very often exactly the question that person actually cares about. What it cannot do on its own is separate the effect of the intervention from everything else that changed over the same period. Formal single-subject designs handle this by alternating conditions repeatedly and randomising the order, which is rarely what a self-tracker actually does.

Without that structure the comparison is between a period before and a period after, and those two periods differ in many ways at once. The design is not worthless; it is simply answering a narrower question than the person running it usually believes it is answering.

Regression to the mean does most of the work

People typically begin an intervention when they feel worse than usual, because feeling worse is what prompts the search for a change. Measurements taken at an unusually bad moment tend to be followed by better ones regardless of what happens in between them. That statistical tendency alone produces the shape of a successful intervention, complete with a convincing contrast between before and after.

The effect is strongest for measures that fluctuate a great deal day to day, which describes energy, mood, sleep quality and digestive comfort. Anyone reasoning from personal experience is reasoning against this headwind, and being aware of it does not remove its influence.

Blinding is hard outside a laboratory

Expectation changes reported symptoms substantially, and it also changes behaviour in ways the person doing the reporting does not notice. Someone who has started a regimen tends to sleep, eat and move slightly differently, and those changes are hopelessly confounded with the intervention. Self-blinding is possible for a capsule using identical placebos prepared in advance, and it is impossible for anything with a felt sensation.

Cold exposure, light devices and dietary changes all announce themselves loudly, which puts a whole class of interventions beyond home experiment. Objective measurements help a little, though they too respond to changed behaviour rather than only to the intervention under test.

Order effects and carryover

Many biological changes persist after an intervention stops, so a period following treatment is not a clean control period for comparison. Adaptation runs the other way as well, with an initial response fading as the system adjusts, which makes early results unrepresentative of later ones. Seasonal variation, illness, work stress and travel all move the same measures being tracked, and not one of them has been randomised.

A design with a single crossover cannot distinguish any of that from the intervention, whereas several alternations gradually begin to. The number of alternations required rises sharply as the expected effect shrinks, which is why detecting subtle effects at home is largely hopeless.

Where the approach genuinely works

Single-subject designs are strong when an effect is large, fast, reversible and objectively measured, which describes a narrow but real set of questions. They are also useful for detecting an individual response to something already known to work on average across a population. Used that way, a self-experiment refines an existing evidence base rather than attempting to substitute for one that does not exist.

Treated as a source of general knowledge, the same design produces confident conclusions that fail to replicate in anybody else. For a symptom that persists, or anything involving prescribed treatment, the question belongs with a clinician rather than in a spreadsheet.