Skip to content

feat(cache): cache tasks that rewrite files they read - #913

Merged
wan9chi merged 2 commits into
test/e2e-step-redactfrom
feat/cache-rewritten-inputs
Oct 30, 2026
Merged

wan9chi merged 2 commits into
test/e2e-step-redactfrom
feat/cache-rewritten-inputs

Conversation

@wan9chi

@wan9chi wan9chi commented Oct 30, 2026

Copy link
Copy Markdown
Member

Motivation

A task that reads a file and writes it back is never cached today ("not cached because it modified its inputs"). That covers tsc --incremental rewriting tsconfig.tsbuildinfo, tools updating their cache files, formatters, and most build tools on a built tree, which read their previous outputs.

Inputs are fingerprinted after the run, so such a file can simply be compared with how the task left it. And because the task wrote it, it may also be missing, as on a clean checkout, where a hit restores it.

Changes

  • Each inferred input records may_be_missing, true for a path the task wrote. Validation accepts a missing path when it's set; a path that's there must still match.
  • What the task wrote comes from every write fspy recorded, before output exclusions and Vite's ignoreOutput are applied. Those still decide only what is archived.
  • The refusal is removed with its summary notice, its --last-details text and the suggested exclusions, along with the input_read_write_not_cached fixture and the docs section about it. The cache schema version is bumped.
  • node_compile_cache_outside_workspace caught a misplaced compile cache only through the refusal, so it now lists the workspace root after its runs.

E2E cases: rewrite_own_file (synthetic, vtt fs read/write) and typescript_incremental (TypeScript 7 tsc --incremental --noEmit). The first commit records today's refusals; the second shows hits, including after tsconfig.tsbuildinfo is removed.

Part of a stack that lets automatic tracking cache tasks which read their own outputs and list the folders holding them, so they also hit on a clean checkout.

🤖 Generated with Claude Code

@wan9chi
wan9chi added this pull request to stack #920 October 30, 2026 03:17
@wan9chi
wan9chi force-pushed the feat/cache-rewritten-inputs branch from 7b723f5 to fcf994a Compare October 30, 2026 03:19
@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

fspy benchmark

linux

dynamic/launch             change  -0.21%  [ -2.86% ..  +3.54%]  overhead   +53.34%
dynamic/access             change  +0.87%  [ -2.00% ..  +3.82%]  overhead   +22.52%
dynamic/access-relative    change  +0.00%  [ -4.78% ..  +8.97%]  overhead   +70.31%
dynamic/access-contended   change  +2.32%  [ -6.54% .. +12.11%]  overhead   +34.08%
static/launch              change  +0.09%  [-10.44% .. +12.24%]  overhead  +186.34%
static/access              change  -0.16%  [ -1.02% ..  +0.91%]  overhead +1446.72%
static/access-relative     change  +0.18%  [ -0.58% ..  +1.04%]  overhead +1976.34%
static/access-contended    change  +0.04%  [ -1.04% ..  +1.45%]  overhead +1441.58%

macos

dynamic/launch             change  +0.01%  [ -5.16% ..  +6.58%]  overhead  +231.96%
dynamic/access             change  -0.34%  [ -8.17% ..  +8.74%]  overhead    +3.97%
dynamic/access-relative    change  +0.55%  [ -8.00% ..  +7.28%]  overhead  +245.43%
dynamic/access-contended   change  +1.47%  [ -3.64% .. +18.37%]  overhead    +1.46%

windows

dynamic/launch             change  +0.48%  [ -3.96% ..  +3.48%]  overhead   +26.89%
dynamic/access             change  +0.19%  [ -0.56% ..  +0.75%]  overhead    +1.14%
dynamic/access-relative    change  +0.00%  [ -0.73% ..  +0.54%]  overhead    +0.94%
dynamic/access-contended   change  +0.00%  [ -0.70% ..  +0.36%]  overhead    +1.45%

wan9chi and others added 2 commits October 30, 2026 13:06
Two cases for caching a task that rewrites a file it read. In
`rewrite_own_file`, the task reads `state.txt` and writes it back, like a
tool that keeps a cache file, through `vtt fs`, a new test helper that
makes a sequence of file system calls in one process. In
`typescript_incremental`, `tsc --incremental --noEmit` (TypeScript 7)
probes for and writes `tsconfig.tsbuildinfo` on a clean run, rewrites it
after an edit, and only reads it when nothing changed.

Today every run that writes the file is refused ("not cached because it
modified its inputs"), and removing `tsconfig.tsbuildinfo`, as on a clean
checkout, is a cache miss.

On Windows, `tsc` also lists the package root, so `typescript_incremental`
runs on Unix only until listings leave out the files a task wrote.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A task that reads a file and writes it back, like `tsc --incremental`
rewriting `tsconfig.tsbuildinfo` or a tool updating its cache file, was
refused ("not cached because it modified its inputs") and ran every
time. Inputs are fingerprinted after the run, so such a file is compared
with how the task left it. A file the task wrote may also be missing, as
on a clean checkout, where a hit restores it: each inferred input now
records `may_be_missing`, and validation accepts a missing file when it
is set. What may be missing is decided in the new `rebuildable` module
from every write fspy recorded, before output exclusions and
`ignoreOutput` are applied; those still decide only what is archived.

The refusal, its run summary notice, its `--last-details` text and the
suggested exclusions are gone, with the `input_read_write_not_cached`
fixture and the docs section about them. The cache schema version is
bumped for the new inferred input format.

`node_compile_cache_outside_workspace` caught Node's compile cache
landing in the workspace only through the refusal; a cache there would
now just hit, so the fixture lists the workspace root after its runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@wan9chi
wan9chi force-pushed the feat/cache-rewritten-inputs branch from fcf994a to 53150e0 Compare October 30, 2026 04:07
@wan9chi
wan9chi merged commit d3a897c into main Oct 30, 2026
21 checks passed
@wan9chi
wan9chi deleted the feat/cache-rewritten-inputs branch October 30, 2026 06:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant