Repository navigation
feat(spec-review): collate-evidence --comment writes spec-review-verdict.json itself (FAFF-1467) - #1183
Merged
alechill merged 6 commits intoNov 5, 2026
Conversation
…c-review-verdict.json itself, so a skipped build step cannot drop the evidence Signed-off-by: Alec Hill <alec@shftwst.dev>
…ict.json itself (FAFF-1467) The command that renders the spec-review evidence comment now also writes the anchored spec-review-verdict.json from that exact text, through the existing writer. A new placeSpecReviewEvidence check places the file from the scratch dir: <run-dir>/<ISSUE> in run mode (through the unchanged guardWriterPaths), or .faff/spec-review/<ISSUE> for interactive prep. A failed write warns on stderr and never changes stdout or the exit code. A new spec-review write-evidence verb covers prep's paths-only carry. faff-prep/SKILL.md carries the two spec-mandated edits; three unrelated restatements were trimmed so the file stays under its prose-bytes baseline, which is lowered to the new size. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Alec Hill <alec@shftwst.dev>
…ent names an unresolvable root (FAFF-1467) From the adversarial review of FAFF-1467: `write-evidence --file -` waited forever on a terminal stdin, and placement reported an unresolvable repository root as ".faff/spec-review resolves outside the repository". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Alec Hill <alec@shftwst.dev>
Signed-off-by: Alec Hill <alec@shftwst.dev>
Contributor
|
Ready to land. Run this locally in a real terminal: |
…rose edits The rule-inventory check reached main while this branch was open. It needs the spec to map each deleted faff-prep/SKILL.md line. The amended spec is re-posted on the issue, and this commits its builder view. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Alec Hill <alec@shftwst.dev>
alechill
deleted the
FAFF-1467-spec-review-collate-evidence-writes-verdict
branch
November 5, 2026 06:18
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
faff spec-review collate-evidence --commentnow writes the anchoredspec-review-verdict.jsonitself, from the exact comment text it prints. A build that skips the prose step that pipes the comment to the writer can no longer drop the evidence.Placement. A new
placeSpecReviewEvidencecheck inevidence.jsplaces the file from the scratch dir alone:<run-dir>/<ISSUE>, through the unchangedguardWriterPaths;.faff/spec-review/<ISSUE>.It refuses a symlinked
.faff,.faff/spec-reviewor scratch dir.One writer.
writeSpecReviewVerdictTois split out ofwriteSpecReviewVerdict, so both paths go through the same parse, redact, anchor and validate steps. The written file is byte-identical to whatfaff evidence write spec-review-verdictproduces from the posted comment.No lost comment. A failed write prints a stderr warning and never changes stdout or the exit code. After placement, a refusal removes any earlier file. A fault moves it to
spec-review-verdict.stale.json.Carry path. A new
faff spec-review write-evidence --dir --fileverb covers prep's paths-only carry.Prose and docs.
faff-prep/SKILL.mdgains the two edits the spec specified.docs/guide/cli.mddocuments the side effect and the new verb.Tracker access. No faff command reads or writes the tracker.
Deviations from the spec
Extra prose trims.
faff-prep/SKILL.mdalso trims three unrelated restatements, without changing their meaning:The spec says "no other prose changes". On the operator's instruction the trims went in anyway, so the spec's own edits fit under the zero-headroom prose-bytes baseline instead of raising it. The baseline is lowered from 104391 to 104335.
Subdirectory gap.
collate-evidencerun from a subdirectory with a relative--dirstill exits 2: round reading is cwd-relative, which the spec leaves unchanged. Placement itself resolves against the repo root, and that is tested throughwrite-evidence. This is recorded as possible follow-up work.Verification
faff gates runpasses all 9 rungs.test/spec-review-evidence-write.test.mjshas 27 tests, all passing. The collate-evidence cases moved there out of thespec-review.jsselftest.test/evidence-write.test.mjsis unchanged and passes.mainand this branch print a byte-identicalcollate-evidence --commentfor the same rounds.Acceptance Criteria
faff-1467#done:1collate-evidence --comment, run against a run-mode scratch, leaves<runDir>/<ISSUE>/spec-review-verdict.jsonon disk with no further command.Verified: test passes: run-mode --comment leaves //spec-review-verdict.json
Method: test
test/spec-review-evidence-write.test.mjs::run mode: --comment prints the comment, exits 0 and leaves the file, byte-equal to the evidence-write replayfaff-1467#done:2The diff adds norequireofnode:http,node:https,node:netor any tracker adapter module tospec-review.jsorevidence.js.Verified: test passes: no node:http/https/net or tracker require in either module; git diff adds only require("./evidence")
Method: test
test/spec-review-evidence-write.test.mjs::the diff adds no network or tracker require to the two modulesMethod: command
git diff origin/main -- plugin/skills/faff/bin/lib/spec-review.js plugin/skills/faff/bin/lib/evidence.js | grep -E '^\+.*require\('faff-1467#done:3guardWriterPathsis byte-unchanged, andfaff evidence write spec-review-verdictwith a--run-diroutside.faff/runsstill exits 2.Verified: guardWriterPaths body identical to origin/main; outside-runs replay exits 2 (test passes)
Method: command
diff <(git show origin/main:plugin/skills/faff/bin/lib/evidence.js | awk '/^function guardWriterPaths/,/^}/') <(awk '/^function guardWriterPaths/,/^}/' plugin/skills/faff/bin/lib/evidence.js)Method: test
test/spec-review-evidence-write.test.mjs::faff evidence write spec-review-verdict still refuses a --run-dir outside .faff/runs with exit 2faff-1467#done:4placeSpecReviewEvidencereturnsmode: interactivefor a direct child of a real.faff/spec-review/andmode: runfor<runDir>/<ISSUE>/spec-reviewwithrunDirunder.faff/runs/. It throwsWriterFaultfor any other directory, for a symlinked child that resolves outside, for a.faffor.faff/spec-reviewthat is itself a symlink, and for an invalid issue id.Verified: tests pass: both modes placed; other dirs, symlinked child, symlinked .faff and .faff/spec-review, bad issue id throw WriterFault
Method: test
test/spec-review-evidence-write.test.mjs::placeSpecReviewEvidence throws WriterFault for every other directoryMethod: test
test/spec-review-evidence-write.test.mjs::placeSpecReviewEvidence refuses a symlinked child, a symlinked .faff/spec-review and a symlinked .fafffaff-1467#done:5A test intest/asserts that, for both modes,placeSpecReviewEvidence(specReviewDir(issue, runDir))returns the expectedtarget_dirandmode, pinning the duplicated scratch layout.Verified: test passes: placeSpecReviewEvidence(specReviewDir(issue, runDir)) pinned for both modes
Method: test
test/spec-review-evidence-write.test.mjs::placeSpecReviewEvidence pins specReviewDir for both modesfaff-1467#done:6writeSpecReviewVerdict(runDir, issue, …)delegates towriteSpecReviewVerdictTo, and every existing case intest/evidence-write.test.mjspasses unchanged.Verified: writeSpecReviewVerdict now returns writeSpecReviewVerdictTo(guardWriterPaths(...)); test/evidence-write.test.mjs unchanged, 14/14 pass
Method: command
node --test test/evidence-write.test.mjsfaff-1467#done:7placeSpecReviewEvidenceandwriteSpecReviewVerdictToare exported fromevidence.js.Verified: both exported: typeof is function for each
Method: command
node -e 'const e=require("./plugin/skills/faff/bin/lib/evidence.js");console.log(typeof e.placeSpecReviewEvidence, typeof e.writeSpecReviewVerdictTo)'faff-1467#done:8SPEC_REVIEW_USAGElistswrite-evidence --dir <scratch> --file <file|->.Verified: usage error output contains write-evidence --dir --file <file|->
Method: command
node plugin/skills/faff/bin/faff spec-review --bogusfaff-1467#done:9collate-evidence --commentstdout is byte-identical before and after this change for the same inputs, and on success stderr includes exactly one linefaff spec-review collate-evidence: wrote <path>(existingskipped …lines may also appear).Verified: origin/main and branch binaries print byte-identical stdout for the FAFF‑1467 rounds (cmp clean); exactly one wrote line asserted by test
Method: command
node plugin/skills/faff/bin/faff spec-review collate-evidence --dir <copy of .faff/spec-review/FAFF‑1467 rounds> --dispositions dispositions.json --comment, run from main and from the branch, then cmpMethod: test
test/spec-review-evidence-write.test.mjs::stdout of --comment is the rendered comment alone, and the file digest matches its blockfaff-1467#done:10On a placement fault or a writer refusal,collate-evidence --commentexits 0, prints the comment, and stderr includes exactly one line beginningfaff spec-review collate-evidence: warning: spec-review-verdict.json not written:.Verified: tests pass: exit 0, comment printed, exactly one warning line on placement fault and refusal
Method: test
test/spec-review-evidence-write.test.mjs::a symlinked .faff/spec-review warns, exits 0, prints the comment and writes nothing in the link targetMethod: test
test/spec-review-evidence-write.test.mjs::a run dir outside .faff/runs warns and still prints the commentfaff-1467#done:11After a placed-but-refused write from either command, nospec-review-verdict.jsonremains in the target. After a placed write that faults, an earlier file is atspec-review-verdict.stale.json, not the canonical path.Verified: tests pass: refusal removes the earlier file; a write fault moves it to spec-review-verdict.stale.json
Method: test
test/spec-review-evidence-write.test.mjs::write-evidence: a digest mismatch exits 1 with refused: and removes an earlier fileMethod: test
test/spec-review-evidence-write.test.mjs::write-evidence: a refusal into an interactive scratch removes the earlier fileMethod: test
test/spec-review-evidence-write.test.mjs::a write fault moves an earlier file to spec-review-verdict.stale.json and the warning is printedfaff-1467#done:12A relative--dirof.faff/spec-review/<ISSUE>places correctly when the command runs from a subdirectory of the repository.Verified: test passes: relative .faff/spec-review/ places from a subdirectory (exercised via write-evidence; collate's round read stays cwd-relative, unchanged)
Method: test
test/spec-review-evidence-write.test.mjs::a relative --dir places correctly when the command runs from a subdirectoryfaff-1467#done:13collate-evidencewithout--commentwrites no file.Verified: test passes: plain mode writes no file in either layout
Method: test
test/spec-review-evidence-write.test.mjs::plain mode (no --comment) writes no file in either layoutfaff-1467#done:14The--from-judge --spec … --commentpath writes the same file as the--dispositionspath when both resolve to the same dispositions.Verified: test passes: --from-judge and --dispositions files are byte-equal
Method: test
test/spec-review-evidence-write.test.mjs::the --from-judge path writes the same file as the --dispositions pathfaff-1467#done:15write-evidenceexits 0 and writes the file for a carry comment. It exits 1 withrefused:for a digest mismatch or a missing section, and exits 2 for missing flags, an unreadable--file, or a placement fault.Verified: tests pass: exit 0 carry (file and stdin), exit 1 refused (digest, missing section), exit 2 (flags, unreadable file, placement fault)
Method: test
test/spec-review-evidence-write.test.mjs::write-evidence: exit 0 and the path on stdout for a carry comment, from a file and from stdinMethod: test
test/spec-review-evidence-write.test.mjs::write-evidence: a refusal with no earlier file writes nothing; a missing section exits 1Method: test
test/spec-review-evidence-write.test.mjs::write-evidence: missing flags, an unreadable --file and a placement fault exit 2faff-1467#done:16The filecollate-evidencewrites is byte-identical to the filefaff evidence write spec-review-verdictwrites from its stdout.Verified: tests pass: written file byte-equal to the evidence-write replay of stdout
Method: test
test/spec-review-evidence-write.test.mjs::run mode: --comment prints the comment, exits 0 and leaves the file, byte-equal to the evidence-write replayMethod: test
test/spec-review-evidence-write.test.mjs::the integration smoke: a replay of the printed comment leaves the written file unchangedfaff-1467#done:17faff-prep/SKILL.md:263 carries the two in-place replacements, and :553 carries thewrite-evidencecall. No other prose changes.Verified: git diff shows both :263 replacements and the :553 write-evidence call; three restatements elsewhere were also trimmed at the operator's direction to hold the prose-bytes baseline
Method: command
git diff origin/main -- plugin/skills/faff-prep/SKILL.mdfaff-1467#done:18faff-prep/SKILL.mdstays at or under 611 lines, its :263 line stays at or under 200 words, andfaff validate-adapterspasses withSKILL_LINE_BASELINEunchanged.Verified: 611 lines; :263 is 198 words; validate-adapters PASS; SKILL_LINE_BASELINE unchanged; prose-bytes baseline lowered 104391 -> 104335
Method: command
node plugin/skills/faff/bin/faff validate-adaptersfaff-1467#done:19docs/guide/cli.mddocuments thecollate-evidenceside effect and thewrite-evidencerow.Verified: collate-evidence row extended and write-evidence row added; lint-cli-doc rung passed in the gate ladder
Method: command
git diff origin/main -- docs/guide/cli.mdfaff-1467#done:20Running the terminal seam twice leaves one file whose bytes match the latest comment.Verified: test passes: second seam run leaves one file matching the latest comment
Method: test
test/spec-review-evidence-write.test.mjs::the terminal seam run twice leaves one file matching the latest commentfaff-1467#done:21Round readers ignorespec-review-verdict.jsonin an interactive scratch. A test runscollateSpecReviewRoundson a dir that holds the file and reports no skip for it.Verified: test passes: collateSpecReviewRounds over a scratch holding the file and its temp sibling reports no skip
Method: test
test/spec-review-evidence-write.test.mjs::round readers ignore spec-review-verdict.json and its temp sibling in an interactive scratchfaff-1467#done:22New tests live intest/, for exampletest/spec-review-evidence-write.test.mjs, run undernode --test, and cover every item above using a temp git root with.faff/runsand.faff/spec-review, astest/evidence-write.test.mjsdoes.Verified: 26 tests in test/spec-review-evidence-write.test.mjs pass under node --test, using a temp git root with .faff/runs and .faff/spec-review
Method: command
node --test test/spec-review-evidence-write.test.mjsfaff-1467#done:23The collate-evidence cases in thespec-review.jsselftest (:240-259) move intotest/and are deleted from the selftest block. No new selftest cases are added to either module.Verified: collate cases deleted from specReviewSelftest (0 collate references remain in it) and moved into test/
Method: test
test/spec-review-evidence-write.test.mjs::collation: an objection-bearing round before a clean approve is kept, a malformed round is skipped, the plain CLI exits 0 (moved from the selftest)faff-1467#done:24node --testovertest/andfaff regions checkpass.Verified: faff gates run UNIT rungs pass; faff regions check exits 0
Method: command
faff gates run --json --resumeMethod: command
node plugin/skills/faff/bin/faff regions checkCloses FAFF-1467