Skip to content

Blob request and response bodies for plugin routes (#1720) - #1764

Closed
michielbdejong wants to merge 1 commit into
claude/plugin-delivery-queuefrom
claude/plugin-route-blobs
Closed

michielbdejong wants to merge 1 commit into
claude/plugin-delivery-queuefrom
claude/plugin-route-blobs

Conversation

@michielbdejong

Copy link
Copy Markdown
Contributor

Closes #1720. Stacked on #1760 (delivery queue), which is on #1759, #1754, #1752, #1751, #1749, #1732 and #1726.

Design: atomic-plugins docs/design/server-plugin-routes.md (ontola/atomic-plugins#108), sections 2.6 and 2.8.

What it does

Everything is behind #[cfg(feature = "plugin-routes")]. Storing a body only happens at --plugin-routes read-write. The 501 route-blob-unavailable from #1751 is gone. Code: new server/src/plugins/route_blobs.rs, called from route_exec.rs and route_writes.rs.

Request bodies (body: "blob")

The checks run in this order, and nothing is stored if one fails:

  1. Level: below read-write the answer is 403 route-blob-refused. A release with a blob route can't install there anyway.
  2. Grant: the installation needs a route grant for one of the route's write targets, checked before a byte is read. Without one the answer is 403 route-write-not-granted. Storing a body is a write.
  3. Size and quota, before storage:
    • The limit is the route's maxBodyBytes, default 16 MiB, capped by the operator's new --plugin-route-max-blob-bytes / ATOMIC_PLUGIN_ROUTE_MAX_BLOB_BYTES, default 16 MiB.
    • Content-Length, then each chunk while it streams in, is checked against the limit (413 route-body-too-large) and against the bytes the installation has left today (429 route-quota-exceeded with Retry-After).
  4. Signature: auth: http-signature is verified over the received bytes.
  5. Storage: after a pool slot is taken, the bytes are booked in Route writes: the route grant, quotas and provenance (#1717) #1754's bytes-per-day ledger and stored content-addressed (BLAKE3). Tree::PluginMeta records that this installation stored them.
  6. Handler: it gets request.blob = { hash, size, type, subject }, and request.body is null. subject is atomic:blob:<hash>, what a File's blob property takes. The bytes never enter the sandbox.

Blob responses

  • Answering with a blob. A handler can return response.blob (a hex hash or an atomic:blob: / did:ad:blob: subject) instead of body. Setting both is 502 route-handler-failed.
  • Rights. A route may only serve a blob its installation stored, or one a resource directly under an approved write target's parent holds. Otherwise the answer is 502 route-blob-refused. The same rule covers blob references in route writes: a route can't write a foreign hash into its folder and then serve it from there (502 route-write-refused).
  • Headers:
    • Content-Type is the handler's, else the type the blob was uploaded with, else application/octet-stream. Run plugin routes: the http trigger and route execution (AS-05) #1751's rules still apply: no HTML, XHTML or SVG on drive-prefix, the sandbox CSP there, and nosniff always.
    • ETag: "<hash>" is set by the host and replaces any etag the handler set.
    • For HEAD, only the size is read, and Content-Length is the blob's.
  • Missing bytes. A blob that is allowed but not on this node is 502 route-blob-missing. That is checked before any intents are applied.
  • Size cap. Blob responses are not held to the inline 1 MiB response cap.

Conditional requests (host-handled)

  • If-Match (strong comparison) and If-None-Match (weak comparison, * supported) are compared with the served blob, or with response.current. That field is the blob (or null) the handler says the target held before a write.
  • GET/HEAD: 304 with ETag, or 412.
  • Other methods: 412 route-precondition-failed.
  • In both cases this is decided before any intent or enqueue is applied, so nothing is stored.
  • If the handler gives neither a blob nor current, the host doesn't evaluate the preconditions. The if-* headers still reach the handler, as before.
  • After a blob upload, a 2xx response without an etag gets the new blob's ETag.

Lifecycle

When an installation is revoked or destroyed, its stored-blob records are erased along with its keys and tokens. The bytes stay: they are content-addressed, and a File may hold them.

Limits

Value
Blob body without maxBodyBytes 16 MiB (design 2.8)
Operator maximum (--plugin-route-max-blob-bytes) 16 MiB default (my choice; the design says "operator-configured")
Quota blob bytes count in full toward --plugin-route-bytes-per-day (64 MiB default from #1754)

Fixture

testdata/plugin-routes/files/ is a remoteStorage-like drive-prefix plugin. It has one write target, files (parent config:folder, class File), and it is exposed as test_fixture::files_release().

  • PUT /files/{*path} (body: blob) creates a File, or points the existing one at the new blob. It returns current and echoes what it saw.
  • GET|HEAD /files/{*path} answers with the File's blob.
  • PUT /small/{*path} does the same as /files with maxBodyBytes: 1024.
  • GET /raw/{hash}?type= answers with any hash. POST /link/{*path} writes any hash into a File. Both exist for the refusal tests.

Tests

  • route_blobs_test.rs (through the app, read-write):
    • a 2 MiB upload is stored, and the handler sees only the reference;
    • a blob response has the right content type, ETag, nosniff, CSP, Cache-Control and length, also for HEAD;
    • conditional GETs answer 304 and 412;
    • PUT with If-None-Match: * on an existing file, or with a stale If-Match, answers 412 and nothing changes; a matching If-Match updates;
    • both size caps (the operator's and the route's maxBodyBytes) answer 413, and the blob isn't in the store;
    • the byte quota: a second upload gets 429 + Retry-After, and isn't stored;
    • a foreign blob is refused as a response and as a write value, and is served once the owner puts a File that holds it under the folder;
    • HTML and SVG are refused on drive-prefix;
    • without a route grant: 403, nothing stored;
    • at read-only the blob release doesn't install, while a GET-only release still serves a blob its installation stored.
  • route_blobs.rs unit tests: hash parsing, the limit cap, precondition rules, and finding blob references.
  • tests/it/plugin_routes.rs (real server): PUT 2 MiB, then GET it back byte for byte with ETag, content type and Content-Length. Also HEAD, 304, 412, and 413 for 16 MiB + 1. install_inbox is generalised into install_writer.

Runs (ATOMICSERVER_SKIP_JS_BUILD=true):

  • cargo test -p atomic-server --lib: 397 passed with --features plugin-routes, 299 without.
  • cargo clippy -p atomic-server --all-targets -- -D warnings: clean both ways.
  • cargo test -p atomic-server --features plugin-routes --test it plugin_routes: 5 passed.

Interpretations

  • "Streamed": the blob backend (put_blob) takes whole values, so the body is buffered in memory while it is hashed, bounded by the limit, then stored. Limits are enforced per chunk as it arrives, so an oversized body is refused before storage. A streaming backend API would be a separate change.
  • Storing requires a route grant. The issue doesn't say. I treated storing a body as a route write, so it needs an approved write target.
  • Quota counts every accepted body in full, even when the same bytes were already stored. It is booked at storage time and not refunded if the handler later fails: the bytes are stored by then.
  • Host-handled preconditions on writes need the handler's help. The host can't know what a path held before, so a handler reports it as response.current. Without it, preconditions stay with the handler.
  • "Its write targets hold" means a resource directly under an approved target's parent references the blob, whoever created that resource.
  • No Range support: the issue doesn't ask for it.
  • Status codes the design doesn't fix: 403 route-blob-refused below read-write; 502 route-blob-refused / route-blob-missing; 412 route-precondition-failed; 500 route-blob-store-failed.

No UI changes.

🤖 Generated with Claude Code

@michielbdejong

Copy link
Copy Markdown
Contributor Author

Merge-risk triage (2026-09-28). This is a read-only review of this PR's own diff. The rule it applies: nothing that works today may break for existing users or clients, and new behaviour is added alongside the old. The stack is being rebased onto develop as one chain before merging, so line references are against the current stacked base.

Verdict: SAFE (conflict is keep-both-sides)

Blob responses get nosniff and a sandboxing CSP (default-src 'none'; sandbox), and HTML/SVG are refused on drive-prefix, so there is no stored-XSS path on the API origin. A route can only reference blobs its installation stored or its write targets hold.

Note: The only conflict with its base (and on top of #1871) is server/src/config.rs: two new Opts fields and env-var test rows next to each other. Keep both sides.

🤖 Generated with Claude Code

A `body: blob` route no longer answers 501. The host reads the body
(413 past the route's maxBodyBytes, default 16 MiB, capped by the new
--plugin-route-max-blob-bytes; 429 past the bytes-per-day quota), hashes
it with BLAKE3 and stores it in the blob store before the sandbox runs.
The handler gets request.blob = { hash, size, type, subject }, never the
bytes. Only at read-write, and only under a route grant for one of the
route's write targets.

A handler may answer with response.blob. The host sends the bytes with
the handler's (or the uploaded) content type, ETag "<hash>", nosniff and
the drive-prefix rules (no HTML/SVG, sandbox CSP); HEAD gets the length.
A route may only serve, or reference in a route write, a blob its
installation stored or one its approved write targets hold.

The host answers If-Match / If-None-Match against the served blob, or
against response.current (what the target held before a write): 304 or
412, and nothing the verdict asks for is stored.

Fixture testdata/plugin-routes/files/ (remoteStorage-like PUT/GET), unit
tests in route_blobs_test.rs, and a 2 MiB PUT/GET round trip in
tests/it/plugin_routes.rs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@michielbdejong

Copy link
Copy Markdown
Contributor Author

Closing as part of the plugin-host stack cleanup (Q-101): not needed by atomic-plugins now. See the inventory at #1726 (comment). The branch is kept, and pin candidates are unaffected. Closed by Claude (VPS coordinator) on behalf of Michiel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant