Skip to content

fix(error_models): catch mis-stated binned-qual anchors (#263) - #266

Merged
cjfields merged 1 commit into
mainfrom
fix/binned-anchors-263
Oct 4, 2026
Merged

cjfields merged 1 commit into
mainfrom
fix/binned-anchors-263

Conversation

@cjfields

@cjfields cjfields commented Oct 4, 2026

Copy link
Copy Markdown
Member

Closes #263.

The problem, measured on real data

trans is indexed by each unique's mean quality, so binned reads do not produce a cleanly binned matrix. They leave mass spikes at the true bins and a thin smear of averaged qualities between them. On 5 NovaSeq 16S samples: Q37 70%, Q36 11.7%, Q35 5.5%, Q25 3.4%, Q11 1.7%, and every other column under 1%.

An anchor that misses the true bins lands in the smear. It still fits, but from a handful of transitions: anchor Q12 rested on 37 transitions, against 1.66% of all transitions at Q11 next to it. With 2,12,23,37 on that data, 241 of 456 off-diagonal rates sat at the 1e-7 floor and 21 at the 0.25 clamp, up to 6 orders of magnitude from the correct model. R builds the identical model (1.9e-15), with no error.

Changes (src/error_models.rs, check_binned_anchors)

Case Before After
data outside the anchors error (as R) unchanged
observed min/max not an anchor warning (as R), repeated every iteration same warning, printed once per run
anchor inside the observed range with no observations rates across the gap silently 1e-7; the whole model when every segment is affected error, naming the empty anchors and the qualities present (R returns NA, or stops)
anchor holding less mass than a non-anchor quality within ±2 silent stern warning, once per run, with both masses: "THE ERROR MODEL IS LIKELY WRONG"
anchor below or above the observed range (NovaSeq's Q2 after trimming) silent, flat-filled unchanged, and now pinned by a test

Alternative bin schemes still fit; the warning only says why they are suspect. --errfun external remains the path for deliberately custom models. docs/commands/learn-errors.md explains the mean-quality smear and the new error and warning.

Verification

Real data, NovaSeq 16S (5 samples, filtered once, and the same files given to R and to dada2-rs):

Anchors dada2-rs warnings Model
2,11,25,37 (correct, unobserved Q2) none byte-identical to before the change, and to R within 2e-15 (F and R reads, identical sum(trans))
2,12,23,37 3, once each: min-not-anchor, Q12 vs Q11, Q23 vs Q25 builds, same as R
11,24,37 (extremes on anchors, so R's warning stays silent) 1: Q24 (0.36%) vs Q25 (3.40%) builds

No false alarm on MiSeq i100 (bins 12,24,38, 3 samples).

Unit tests:

  • an empty interior anchor errors, both in the full collapse and in a one-sided gap;
  • an unobserved outer anchor is silent and gives the same model as without it;
  • correct anchors on smeared data are silent;
  • shifted anchors are named.

Mutation-checked: with the empty-anchor error disabled, both error tests fail.

Full suite: cargo test --profile test-opt passes on default and --features wfa (465 passed); clippy is clean on both; mkdocs build --strict passes.

A correction to the issue

#263 was filed from a synthetic case where the bin columns hold all the mass, and it says the model "collapses to min_error_rate". On real data, wrong anchors corrupt the model rather than zeroing it: about half the rates at the floor and some at the clamp. The issue text and PR #265's comments are corrected to say so.

🤖 Generated with Claude Code

trans is indexed by each unique's mean quality, so binned reads leave
mass spikes at the true bins and a thin smear between them. A wrong
anchor lands in the smear and fits from a handful of transitions: on
NovaSeq 16S, anchors 2,12,23,37 put 241 of 456 rates at the 1e-7
floor. R builds the same model. Three changes:

- an anchor inside the observed range with no observations at all is
  an error (R returns NA there, or stops);
- a stern warning when an anchor holds less mass than a non-anchor
  quality within two of it, naming both masses;
- binned-qual warnings print once per run, not once per iteration.

Correct-bin models are unchanged and match R to 2e-15 (NovaSeq 16S
F and R).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

binned-qual errfun silently collapses to min_error_rate when interior anchors have no observations

1 participant