Multi-head retention-time calibration by default - #81
Conversation
Leaving `rt_im_train.multihead_calibration` unset now calibrates the DeepLC base model over 80 of its best-correlating LC-setup heads against each run's own confident seed PSMs, in place of the single head `deeplc.predict` returns. Measured on two acquisitions, six pooled runs each, at an unchanged empirical decoy fraction of 0.0100 in all four arms (docs/08 section 4d): AIF 80,842 -> 84,725 peptides (+4.8%), Astral 102,942 -> 117,652 (+14.3%), precursors +4.8% and +15.5%, protein groups +2.4% and +7.1%. Fewer candidates reach rescore and more of them are real, which is interference removed rather than a threshold traded. It costs 1.4x to 1.7x wall clock, because the calibration is fitted against each run's own anchors and cannot be shared across an experiment; `multihead_calibration: 0` restores the previous behaviour exactly. The field becomes `Option<usize>`, because a bare number could not express this default without breaking two things: - a native, Python-free run is a supported configuration, and a default it cannot satisfy would have turned "no interpreter" into a startup error. Unset means automatic, and automatic means nothing when no interpreter resolved; - `finetune_deeplc` occupies the same slot, and validation refuses the pair. A configuration that enables the fine-tune would have been rejected for a conflict it never wrote. The fine-tune now keeps its slot; asking for both explicitly is still an error. The default is also SCOPED, which three existing tests caught the absence of: it applies only where the run's retention times are DeepLC's already -- an imported library under `library_irt = auto` or `deeplc`, or FASTA with `rt_predictor = deeplc`. Without that scope a `rt_predictor: native` run would have had its RT source silently replaced by DeepLC because an unrelated interpreter happened to be discoverable. An explicit count ignores the scope and is a hard requirement. `config.rs::multihead_heads` is the single resolver, so the two orchestrators and the `library_irt` predicate cannot disagree about whether it ran. Two consequences: the separate base-model re-prediction no longer runs under the defaults, because multi-head re-predicts the library itself (27 minutes on a 10.9M-row library, producing a column it would overwrite); and `model_identities.rt_predictor` now reads `multihead-80` rather than `deeplc-4.4.0-base`, which is how to tell which one a run used. Also fixed, because this default made it reachable: an optional numeric setting could not be changed from the desktop settings screen. `Option<usize>` was classified as kind "other", so the form sent the STRING "0" and the engine rejected it -- the off-switch for the very default this commit ships. The generator now unwraps `Option<T>` for numeric T (`extract.windows_in_flight` too) and a blank optional number means unset rather than an error. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CLAUDE.md gates a sensitivity default on entrapment plus two acquisition contexts. The two acquisitions were already recorded; this is the empirical null. E. coli AIF file against an E. coli + 1:1 human entrapment library, one binary, two runs differing only in rt_im_train.multihead_calibration: real peptides at 1% 7,879 -> 8,216 (+4.28%) spike-in peptides 138 -> 144 empirical FDP 0.995% -> 0.995% Identical to three decimal places, so the gain is real discoveries rather than a loosened threshold. At 138 and 144 accepted spike-ins this is about five times more sensitive than the entrapment arms in docs/28, which rest on 22-30 and cannot resolve below roughly 0.1 percentage points. Two things the write-up is careful not to over-read. Decoys accepted at the entrapment threshold moved 91 to 113; under classifier=entrapment the q column is the entrapment FDP rather than target-decoy, so that is a side diagnostic and not the threshold in use. And the 1.06x wall clock here is not the cost of the default: a single run whose baseline already pays a DeepLC re-prediction displaces that work rather than adding to it, so the 1.4x-1.7x from the pooled experiments is what to plan around. The archived entrapment library could not be used. Its report reads `native-rt-v1; native-frag-v1`, and a run against it returns 333 confident seeds and 6,401 candidates surviving extract -- the degenerate regime the HCDch2 finding describes, in which nothing can be measured. It was rebuilt over the same 5,828,348 peptidoforms with MS2PIP 4.2.0 HCDch2 and DeepLC 4.4.0 (`deeplc-4.4.0-base; ms2pip-4.2.0-HCDch2`, 69,564,520 fragments). Anyone repeating this benchmark should check the library's model_identity first, which is why that is written down. `bench/entrapment_fdp.py` is the analysis, mirroring `rescore.rs::classify_entrapment` and recomputing `(ratio * entrapment + 1) / real` rather than trusting a single column. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Entrapment arm: done, gate closedThe empirical null CLAUDE.md requires. E. coli AIF file against an E. coli + 1:1 human entrapment library, one binary, two runs differing only in
FDP is At 138 and 144 accepted spike-ins this is roughly five times more sensitive than the entrapment arms in docs/28, which rest on 22-30 and cannot resolve FDP differences below about 0.1 percentage points. Two things deliberately not over-read:
The library had to be rebuilt firstThe archived entrapment library reads
Gate status: both halves now satisfied. Two acquisitions (AIF, Astral) and an empirical null. Out of draft. |
# Conflicts: # configs/config-schema.json # docs/24_config_reference.md # rust/mumdia/crates/mumdia/src/python.rs
Leaving
rt_im_train.multihead_calibrationunset now calibrates the DeepLC base model over 80 of its best-correlating LC-setup heads against each run's own confident seed PSMs, instead of using the single headdeeplc.predictreturns.Measured
Two acquisitions, six pooled runs each, configurations differing only in this setting (docs/08 §4d):
Identical decoy fraction in all four arms, and fewer candidates reaching rescore with more identifications out of them: interference removed, not a threshold traded.
Cost: 1.4×–1.7× wall clock. The calibration is fitted against each run's own anchors, so it cannot be shared across an experiment.
multihead_calibration: 0restores the previous behaviour exactly.Why the field became
Option<usize>A bare number could not express this default without breaking two working configurations:
finetune_deeplcoccupies the same slot, and validation refuses the pair — so a config enabling the fine-tune would have been rejected for a conflict it never wrote. The fine-tune now keeps its slot; asking for both explicitly is still an error.The default is scoped, and three tests caught the absence of that
It applies only where the run's retention times are DeepLC's already: imported library under
library_irt = auto/deeplc, or FASTA withrt_predictor = deeplc. My first cut lacked this and would have let art_predictor: nativerun have its RT source silently replaced by DeepLC because an unrelated interpreter happened to be discoverable. An explicit count ignores the scope and is a hard requirement.config.rs::multihead_headsis the single resolver, so the two orchestrators and thelibrary_irtpredicate cannot disagree about whether it ran.Two consequences worth knowing:
library_irtbase-model re-prediction no longer runs under the defaults, because multi-head re-predicts the library itself — about 27 minutes saved on a 10.9M-row library, producing a column it would have overwritten;model_identities.rt_predictornow readsmultihead-80rather thandeeplc-4.4.0-base. That is how to tell which one a given run used.Also fixed, because this default made it reachable
An optional numeric setting could not be changed from the desktop settings screen at all.
Option<usize>was classified as schema kindother, so the form sent the string"0"and the engine rejected it — the off-switch for the very default this PR ships. The generator now unwrapsOption<T>for numericT(this also fixesextract.windows_in_flight), and a blank optional number means unset rather than an error.Gate status
CLAUDE.md requires entrapment plus two acquisition contexts before a sensitivity default moves.
native-rt-v1; native-frag-v1, and a run against it produces 333 confident seeds and 6,401 candidates surviving extract — the degenerate regime, which measures nothing. It is being rebuilt with MS2PIPHCDch2+ DeepLC over the same 5.83M peptidoforms, after which both arms run against it and the FDP goes in this PR and in docs/08.I will not take this out of draft until that number exists.
Validation
cargo fmt --check,clippy --all-targets -D warnings,cargo test --workspace283 passedcargo test --lib87 passed;check_desktop_ui.py;node --checkgen_config_reference.py --check,check_doc_refs.pycleanTwo pre-existing tests changed meaning rather than breaking, and are updated to assert the new semantics: the base-model re-prediction is now suppressed under the shipped defaults because multi-head is the re-prediction.
🤖 Generated with Claude Code