Skip to content

feat(cli): opt-in --partialGraph for generate-hashes, deriving from --baseHashes without a bazel query - #589

Merged
tinder-maxwellelliott merged 3 commits into
masterfrom
claude/agitated-bun-14d281
Oct 31, 2026
Merged

tinder-maxwellelliott merged 3 commits into
masterfrom
claude/agitated-bun-14d281

Conversation

@tinder-maxwellelliott

Copy link
Copy Markdown
Collaborator

What

Adopts serve --partialGraph (#586) in the one-shot CLI. generate-hashes --partialGraph hashes the working tree's own git tree object, records every rule's hashing recipe and the tree id in the file, and with --baseHashes <file> re-evaluates that base's recipes against the workspace's tree when git diff between the two trees only modifies regular file content: no bazel query, no Bazel at all. Anything else (graph-defining files, adds/deletes/renames, symlinks, repo-rule inputs, seed files, --partialGraphRebuildPattern, a base tree the clone lacks) falls back to a full generation with the reason logged. The file a derivation writes is byte-identical to the one a full generation in the mode writes, so derived files are bases for later runs.

Phases 1-2 of a four-phase plan; phases 3 (a CLI cache: --cacheDir, the shared S3/Nexus tiers, ancestor base search, --revision) and 4 (a CLI scenario in the nightly partial-graph bench) are not in this PR.

Why

The query is the whole cost of generate-hashes, and a source-only change cannot alter the graph. In the traditional two-file workflow this lets a PR job derive its hashes from the merge base's file without Bazel.

Decisions a reviewer should know about

  • The revision is the working tree's tree object, built from a temporary copy of the index (add --all, write-tree), so the repo's own index is never touched. A clean tree yields HEAD^{tree}, so CLI and serve entries for the same commit share an id; a dirty tree yields a real tree object git diff/ls-tree accept. The workspace must be the repository root.
  • The run's own output, --baseHashes and --depEdgesFile files are left out of the snapshot when they lie inside the workspace. Without this, a hashes file written into the repo shows up as an added path and forces a fallback on every run.
  • Partial-graph files always carry target types, and the dependency edges when --depEdgesFile is given: recipes cannot be evaluated without kinds, and a derivation cannot invent edges (a base without them falls back when edges are requested). --targetType is rejected under the flag; filter in get-impacted-targets instead.
  • get-impacted-targets and explain refuse a pair of files of which only one carries recipes, since the two formulas are not comparable.
  • An unreadable --baseHashes is an error (a misconfiguration); a base without recipes or tree, or a classification failure, is a logged fallback.
  • metadata.tree is written only in partial mode, so default-mode files and serve's default-mode cache entries are byte-for-byte unchanged.

Refactor

Git plumbing (revision/tree resolution, ls-tree indexing, the working-tree snapshot, checkout, classification) moves from server.rs to src/git.rs behind a Git runner that serve binds to the query's cancellation token; the configuration fingerprint and cache key move to src/cache.rs so a later CLI cache shares serve's <tree>.<fingerprint> key space. server.rs keeps one-line wrappers, so its tests did not move. The S3/Nexus tiers stay in server.rs until the CLI uses them.

Tests

  • src/git.rs: resolution, snapshot (clean = HEAD tree, dirty edits, untracked/ignored, deletions, excluded files, no index file, non-root workspace), checkout with a stale index.lock, classification.
  • src/cache.rs: fingerprint sensitivity, key shape.
  • src/main.rs: flag parsing, every rejected combination, derive / same-tree / fallback paths against a real git repo with a never-called generator, the mixed-mode guard for both subcommands.
  • e2e core::generate_hashes_partial_graph_derives_from_base_hashes: derived file byte-identical to a full generation, dep-edges files equal, same impacted set as the default mode, uncommitted and committed edits yield the same file, mixed modes refused, BUILD edit falls back.
  • Verified locally: //src:rust_tests, //src:cli_tests, the new e2e case plus both serve partial-graph cases, //tools/e2e:regen_check, //:rust_format_check, clippy aspect. The coverage gate was not run locally.

README gains a "Partial Graph Updates" section (generated from tools/readme_template.md).

🤖 Generated with Claude Code

tinder-maxwellelliott and others added 3 commits October 30, 2026 11:53
…-baseHashes without a bazel query

Adopts serve's partial-graph derivation in the one-shot CLI. Under the
flag, generate-hashes hashes the working tree's own git tree object
(committed or not; a clean tree is HEAD's), records every rule's recipe
and the tree id in the file, and with --baseHashes re-evaluates the
base's recipes against that tree when `git diff` between the two trees
only modifies regular file content. Anything else falls back to a full
generation with the reason logged; either way the file is byte-identical
to a full generation's, so derived files are bases for later runs.

Details that fell out of making it usable:
- the run's own output, --baseHashes and --depEdgesFile files are left
  out of the snapshot when they lie inside the workspace, otherwise a
  hashes file written into the repo forces a fallback every time;
- partial-graph files always carry target types, and the dependency
  edges when --depEdgesFile is given, since recipes cannot be evaluated
  without kinds and a derivation cannot invent edges; --targetType is
  rejected under the flag;
- get-impacted-targets and explain refuse a pair of files of which only
  one carries recipes, as the two formulas are not comparable.

Refactor: git plumbing (resolution, ls-tree indexing, working-tree
snapshot, checkout, classification) moves from server.rs to git.rs, and
the configuration fingerprint and cache key to cache.rs, so a later CLI
cache shares serve's key space. TreeIndex carries its tree id and hash
files record it as metadata.tree in partial mode only.

Tests: git.rs and cache.rs unit tests, CLI tests for the flags and the
derive/fallback paths, and e2e
core::generate_hashes_partial_graph_derives_from_base_hashes asserting
byte-identical output and impacted-set equality with the default mode.
README gains a "Partial Graph Updates" section.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ry copy

The working-tree snapshot copies .git/index to a temporary index before
`git add --all`. On Linux the copy gets a fresh mtime, which defeats git's
racy-index rule: an edit made in the second the index was last written,
keeping the file's size, is trusted as unchanged and left out of the
snapshot. macOS's fs::copy preserves the mtime, which is why the CLI unit
test only flaked on the Linux runner (Rust candidate, Bazel 9.x).

Stamp the copy with the original's mtime so the snapshot re-reads exactly
the entries git itself would, and add a deterministic regression test that
models the race with core.checkStat=minimal and core.trustctime=false.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@tinder-maxwellelliott
tinder-maxwellelliott marked this pull request as ready for review October 31, 2026 12:16
@tinder-maxwellelliott
tinder-maxwellelliott merged commit b5790f1 into master Oct 31, 2026
19 checks passed
@tinder-maxwellelliott
tinder-maxwellelliott deleted the claude/agitated-bun-14d281 branch October 31, 2026 12:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant