Simulation first · real data second · every number from a canonical result file
When a correction makes the map look cleaner, what did it take with it?
Surveys remove observational systematics — depth, dust, stars, seeing, scan stripes — by regressing the measured map against maps of observing conditions and subtracting the fit. Increasingly the regressor is a flexible machine-learning model. DarkSignal asks what that flexibility costs the weak signal the measurement is for.
On real data this question has no answer, because the true signal is unknown. So the benchmark is built on simulated worlds where the true signal, the contamination and the noise are all kept separately, and every correction is decomposed against them.
The question
An observed map is d = s + c + n: signal, contamination, noise. A correction sees d and
the nuisance templates, and returns an estimate of the signal. If part of the true signal happens to look like the
templates, removing the templates removes that part too. If the corrector is flexible, it can remove more than that.
A map that has lost signal also has less variance, so it looks cleaner — which is exactly the problem.
Simulation canonical world, ρ = 0.3 · corrected by …
What the benchmark found
1. The degeneracy has a closed form, and it holds.
By construction exactly ρ² of the signal variance lies in the span of the templates. Any correction that projects
the templates out must lose that fraction, so linear regression's signal transfer is 1 − ρ². Across the
ρ sweep the measured value matches it to within …. On real DESI
templates, the injection test reproduces it too.
2. Flexible correctors erase signal that nothing forced them to.
At ρ = 0.3, gradient boosting keeps … of the signal no template can mimic, the CNN …, the autoencoder …. Linear regression keeps …. This loss does not grow with ρ: it is not confounding.
3. The zero-contamination control catches all of them.
Given data with nothing to remove, every flexible method still removes signal: gradient boosting passes in … of worlds, the linear family in ….
4. Block cross-validation ranks them correctly — leaky CV does not.
With contiguous spatial folds, CV prefers the linear methods. With random pixel folds it picks models that keep only … (random forest) of the clean signal, against … under block CV.