By this point the pipeline was working. A public temporal U-Net for detection, a global ILP for linking, my own graph calibration on top: 0.881.
And then everything stopped working.
Failure one: the improvement worth exactly zero
Line-fit smoothing was the obvious next step. Fit a line through each track's positions and snap the points onto it, so cells travel along clean trajectories instead of jittering. Visually it is a clear improvement — the tracks stop wobbling.
My local scorer agreed: +0.0064. Small, but well outside anything I'd have called noise.
The leaderboard moved by 0.000. Not a little. Not within error. The score before was 0.881 and the score after was 0.881.
In hindsight the reason is sitting in the metric definition from Part 1. A predicted node matches a ground-truth node if its centroid falls within about 7 µm. Inside that radius, closer is not better. Matching is a threshold, not a gradient.
My smoothing moved points by fractions of a micron. Every point it moved was already inside the radius and stayed inside it. Not one match flipped, so not one edge changed, so the score could not move. I had spent a day polishing positions — and the score is about connections. I was polishing cells.
One honesty note, because I want the claim to be exactly as strong as the evidence. The measured facts are +0.0064 local and +0.000 on the leaderboard. The explanation — that sub-threshold polish is invisible to a radius-matched metric — is a well-supported reading of the metric definition, not something I isolated in a separate experiment. It fits, and I believe it. It is reasoning, not a measurement.
Failure two: the improvement too small to see
The second one is quieter and, I think, more instructive.
I swept the detection confidence threshold across its full range, looking for a better operating point. The local curve was flat — every value within about ±0.001 of every other. A reasonable reading is "this parameter doesn't matter."
Instead I shipped the value that looked best, 0.98, at +0.0005 local.
It lost 0.002 on the board, taking 0.881 down to 0.879. I reverted it.
Nothing here was a modelling mistake. The mistake was treating the highest point on a flat curve as a signal. A +0.0005 difference was smaller than my scorer's ability to resolve, so "best" was indistinguishable from "arbitrary" — and I picked an arbitrary value and paid for it.
Why these are not the same failure
It is tempting to file both under "the local score didn't transfer." That reading is wrong, and it throws away the useful part.
| Local | Leaderboard | What it says about the scorer | |
|---|---|---|---|
| Line-fit smoothing | +0.0064 | +0.000 | Biased — measured a real change the true metric ignores |
| Detection threshold 0.98 | +0.0005 | −0.002 | Noisy — the difference was below its resolution |
The smoothing result was big enough to measure — +0.0064 sits well outside a ±0.002 band. It measured the wrong thing. That is bias: my scorer was rewarding position accuracy that the real metric, with its match radius, is structurally incapable of caring about.
The threshold result was too small to measure at all. That is noise: nothing wrong with what the scorer values, just no ability to resolve a difference that fine.
Different diagnoses, different fixes. Bias means your metric disagrees with the real one about what counts, and no amount of extra data helps — you have to fix the metric. Noise means your metric agrees but cannot see finely enough, and more volumes or more repeats would help.
A number you compute for yourself can be wrong in both of those ways. Mine was wrong in each, exactly once.
The discipline I'd actually state
Part 2 ended on a local scorer that predicted the leaderboard to within 0.008, which felt like the project's foundation. It was — but I drew the wrong conclusion from it.
Being accurate to 0.008 in absolute terms says nothing about being able to rank two options that differ by 0.002. I treated those as the same property for about a week.
So the rule is not "measure locally." It is:
Know your measurement's noise floor, and refuse to ship inside it.
Had I plotted my own scorer's variance before trusting its third decimal place, both of these would have been caught before they cost submissions. The threshold sweep would have been visibly flat rather than deceptively peaked, and the smoothing gain would have been visibly the wrong kind of gain.
Other things that did not work
For completeness, since negative results are most of the actual work:
- Motion-gated gap recovery. Only close a gap when the motion is consistent — clearly smarter than closing every gap. Measured worse at every setting, 0.9571 down to 0.9544. It stayed in the codebase, defaulted off. Testing it prevented a regression, which is a real return on a negative result.
- The division term is effectively dead. Worth 10% of the score, and division Jaccard came in at ≤0.009 on the one volume with enough labelled divisions to measure, and zero elsewhere. I put real effort here for no return. Judging by public scores, so did everyone else — which makes it one of the two places genuine headroom might still be hiding.
- Longer temporal windows and longer training — both flat, both covered in Part 3.
Where the ceiling is
0.881. I am fairly confident that is the ceiling for this approach — a public detector plus my own graph calibration — because every cheap lever I could find is exhausted and the remaining ideas all measured flat or negative.
Beating it means training a detector from scratch, in-domain, at full scale. Days of compute, on a hunch, with no positive signal that it would help. That is a pure appetite bet, and I have not taken it.
Two caveats keep the number honest. Public notebooks in this competition reach 0.902, so 0.881 sits at the public floor, not ahead of it. And nearly everything was tuned on a single volume from a single embryo, so cross-embryo generalisation is untested — the risk I flagged in Part 5 and have not retired.
The competition is still open, so there is no ranking to report and I am not claiming one.
What I will claim is the smaller, more portable thing. I built a measurement, trusted it further than it deserved, and got caught twice — once by bias and once by noise. Finding the edge of your own instrument is not a detour from the work. On this project it was the work.
Knowing where the ceiling is, is also a result.