Per-file progress in a multi-file experiment - #85
Merged
Conversation
Reported: the steps in the progress tab work for the first file but not for the ones after it. `scan_stages` walked the whole output tree and aggregated by stage NAME, and `render` then took "the furthest expected stage that has any report" as the current one. In a pooled experiment the engine runs the per-file chain -- convert through compete -- once per input file in that file's own subdirectory, then does a single rescore/quant/report at the experiment root. So once file 1 reached the end, file 2's `convert` could not move the ladder backwards, and five of six files ran with the display frozen on the first file's finish. The counts were wrong in the same way: summed across files, they described no file in particular. On the six-file Astral experiment that is `extract` reading 87,859,456 rows from 12 artifacts. The snapshot now carries `runs` -- one entry per output subdirectory that holds an artifact report, with that file's own stages -- and `root_stages` for the pooled tail. A subdirectory counts as a run when it has a report, so this needs no coordination with the engine's naming, and `r10` sorts after `r2` rather than between `r1` and `r2`. The deep aggregate stays as `stages`, unchanged, for the single-run case. The display uses the last file with any stage, unions its stages with the root's, and adds "file k of n" to the subtitle. During the very first file `runs` already has one entry, so it reads "file 1 of n" rather than "0". Verified against the reported experiment's own output: six run directories found, seven stages each, `rescore` alone at the root. A test covers the per-file split, the numeric ordering, the root/file separation, and that the aggregate still sums. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
clippy::unnecessary_sort_by, which CI caught and I did not: I ran the desktop tests on this branch but not `cargo clippy` on that crate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reported: "the steps in the progress tab work for the first file, but not for following files".
Cause
scan_stageswalked the whole output tree and aggregated by stage name;renderthen took "the furthest expected stage that has any report" as the current one.In a pooled experiment the engine runs the per-file chain — convert through compete — once per input file in that file's own subdirectory (
r0,r1, …), then does a single rescore/quant/report at the experiment root. So once file 1 reached the end, file 2'sconvertcould not move the ladder backwards, and five of six files ran with the display frozen on the first file's finish.The counts were wrong the same way. Summed across files, they described no file in particular — on your six-file Astral experiment:
Fix
The run snapshot now carries:
runs— one entry per output subdirectory holding an artifact report, with that file's own stages;root_stages— the pooled tail that follows every per-file chain.A subdirectory counts as a run when it has a report, so this needs no coordination with the engine's naming (
--run-namesworks as well as the defaultr0…), andr10sorts afterr2rather than betweenr1andr2. The deep aggregate stays asstages, unchanged, for the single-run case.The display uses the last file with any stage, unions its stages with the root's, and adds "file k of n" to the subtitle. During the first file
runsalready has one entry, so it reads "file 1 of n" rather than "0".Verified against the reported experiment
Run directly over
results2: six run directories discovered in order, seven stages each,rescorealone at the root — which is exactly what the ladder will now follow.88 desktop tests (up from 87). The new one covers the per-file split, the numeric ordering, the root/file separation, and that the aggregate still sums.
🤖 Generated with Claude Code