Skip to content

MCP server (local and hosted), and connecting apps and CLIs with their own key - #1870

Closed
joepio wants to merge 13 commits into
developfrom
claude/atomic-mcp-r4jdjj
Closed

joepio wants to merge 13 commits into
developfrom
claude/atomic-mcp-r4jdjj

Conversation

@joepio

@joepio joepio commented Sep 28, 2026 •

Copy link
Copy Markdown
Member

Requested by Joep · project thread

Before: only the in-app assistant could work with Atomic Data, the CLIs needed you to paste your agent secret, and no MCP client could reach a node.

After: a node serves POST /mcp itself, and that is the one implementation of the MCP tools. A client that cannot run a process (claude.ai) connects to it directly. A local client (Claude Code, Cursor) runs npx @tomic/mcp connect, which signs in to the same endpoint with OAuth and keeps a token in a file only you can read; the @tomic/mcp process is then only a stdio bridge to /mcp. In both cases the person approves at /app/authorize-mcp: which drives, and read only or read and edit. Account settings lists the connections under Connected apps, each with Revoke. atomic-cli connect and ad-generate connect keep using a key of their own (/app/connect-agent).

How, server (server/src/mcp/):

  • The node is the OAuth 2.1 authorization server (dynamic client registration, PKCE S256, RFC 9728 and 8414 metadata, resource checked) and /mcp the resource server (Streamable HTTP, JSON-RPC). Client ids, codes and tokens are signed with a key derived from the node key, so nothing is stored.
  • On Allow the app asks the node for an issued agent (/oauth/agent), gives it read (and write if allowed) on the drives picked, as the person, and completes with /oauth/approve. What a token may reach is what the ACLs give that agent.
  • Tools: list_drives, get_resource (documents and meetings include their text), search, query, get_user_classes, get_schema, and, only when editing was allowed, create_resource, edit_resource and delete_resource (never a whole drive). Documents take Markdown as _documentText (lib/src/document_markdown.rs).
  • A write is a commit the node signs as the issued agent (its key is derived again from the token's claims, never stored) and sends through the normal commit pipeline: signature, rights, subject ownership, broadcast. So it can only do what the ACLs give that agent, history shows the app by name, the node never signs as the person, and no bearer token is accepted on /commit or the WebSocket.

How, consent page: it only accepts a server that is the app's own origin or the node the person is signed in to, and only redirects back to exactly the registered redirect_uri plus code, state and iss. The host the client returns to is in the heading. Tests in hostedMcp.test.ts.

Not in this PR: semantic_search on /mcp (needs the embeddings index), format=compact short refs, refresh-token rotation, a one-time code store that survives a restart. The in-app assistant still has its own tool list.

Related Issues

Part of #1049. Follows planning/mcp-endpoint.md (the plan from #1347).

Checklist

  • Add changelog entry linking to issue, describe API changes (CHANGELOG.md, browser/CHANGELOG.md)
  • Add or update tests if needed (Rust unit tests for Markdown to document body, value coercion, redirect URIs, PKCE; browser/lib/tests/hosted-mcp.integration.test.ts runs registration, approval, PKCE, reads, writes, revoking and the local bridge through real OAuth against a real server; browser/mcp tests the connect flow and the bridge; hostedMcp.test.ts the consent page checks)
  • Update docs if needed (browser/mcp/README.md, planning/mcp-endpoint.md)

`@tomic/mcp` (browser/mcp) is a stdio MCP server for Claude Code, Claude
Desktop, Cursor and other clients. It runs locally and signs every edit
with the user's own Agent key, so writes are ordinary signed commits.
Tools: list_drives, get_resource (documents and meetings include their
text), search, semantic_search, query, get_user_classes, get_schema,
create_resource, edit_resource and delete_resource (never a whole drive).

The data verbs behind those tools now live in @tomic/lib
(assistant-tools.ts), and the in-app assistant calls the same functions,
so there is one implementation behind both surfaces. The compact JSON-AD
dialect, short subject refs and class helpers move from the data-browser
to @tomic/lib for the same reason. Reads no longer include the genesis
certificate, which is long and tells a model nothing.

This is steps 1 and 2 of planning/mcp-endpoint.md; a hosted endpoint
with OAuth (for claude.ai connectors) is not part of this change.
@joepio joepio self-assigned this Sep 28, 2026
atomic-mcp now makes a key on the machine it runs on and prints a link.
In the app (/app/connect-agent) the person picks which drives it may
reach and whether it may edit, then clicks Allow. Account settings lists
connected apps with a Revoke button.

Only an agent may edit its own Agent resource, so the grant is not
recorded there: the ACLs on the shared drives are the record. The app
adds the key to read (and write), the MCP finds its access by searching
for resources that list it, and revoking removes it everywhere, including
from what it created. The app remembers which apps were connected on the
private drive, so settings can list them.

ATOMIC_AGENT_SECRET still works for scripts and CI.
Any app that makes its own key (a CLI, a script, the MCP server) can now
ask for access the same way: connectAgentUrl builds the approval link,
optionally asking for edit rights or specific resources, waitForGrant
polls until the person clicks Allow, and publishAgentName sets the name
shown under Connected apps. The approval page honours the write and
target hints as preselections; the person still decides.

The MCP server now uses these instead of its own copies.
atomic-cli connect makes a key on this machine, prints the approval link
(/app/connect-agent), waits for Allow and writes the config. Without a
config the CLI now points there instead of asking for the secret.

ad-generate connect does the same for the codegen CLI, so private
ontologies no longer need an agent secret in atomic.config.json, which
often ends up in a repository. The key file helper moves from the MCP
server into @tomic/lib/node (loadOrCreateLocalAgent), shared by both.

Also adds the new screens' strings to the translation catalogs.
@joepio joepio changed the title Add an MCP server so LLM clients can read and edit Atomic Data MCP server, and connecting apps and CLIs with their own key Sep 28, 2026
POST /mcp (Streamable HTTP) for clients that cannot run a local process. The
node is the OAuth 2.1 authorization server (dynamic client registration,
PKCE S256, RFC 9728 and 8414 metadata) with stateless signed tokens derived
from the node key. The person approves in the app at /app/authorize-mcp, which
gives a fresh issued agent read rights on the drives they pick, so revoking is
removing it under Connected apps. The node never signs as the person. Tools:
list_drives, get_resource (with document text) and search.
@joepio joepio changed the title MCP server, and connecting apps and CLIs with their own key MCP server (local and hosted), and connecting apps and CLIs with their own key Sep 29, 2026
@joepio
joepio marked this pull request as ready for review October 5, 2026 09:25

joepio commented Oct 5, 2026

Copy link
Copy Markdown
Member Author

Review of the hosted MCP part (oauth.rs, tokens.rs, endpoint.rs, AuthorizeMcpRoute, hostedMcp.ts). The token design is sound (stateless MACs per kind, PKCE S256, redirect URIs checked against the registered list, single-use codes, read-only tools). One blocker, plus smaller points.

Blocker: the consent page trusts server from the URL (AuthorizeMcpRoute.tsx, hostedMcp.ts). /app/authorize-mcp?server=https://evil.example&client_id=...&redirect_uri=https://x&code_challenge=x makes the app, once the person clicks Allow:

  1. send two requests signed by their agent to evil.example (/oauth/agent, /oauth/approve),
  2. grantAgent(store, issued.agent, ...) with whatever agent subject evil.example returns, i.e. read rights on the picked drives for an attacker-held key,
  3. window.location.assign(redirect) with whatever redirect_url evil.example returns, including a javascript: URL, which runs in the app's origin and can read the stored agent secret.

Fix: only accept a server that is the node the person is signed in to or an allow-listed host (compare origins, refuse otherwise), and show it on the page; parse redirect_url and refuse anything that is not exactly the redirect_uri the page was opened with plus code/iss/state params (and https/localhost/app scheme only). A test for both.

Smaller:

  • client_name is attacker chosen (open registration) and shown first on the consent page. The redirect host is shown, which helps; put the host in the heading, not just the body, and show the node host too.
  • redeem_once is per process. After a restart or on a second node a code can be redeemed again inside its 120 s TTL. Fine for now; note it in the plan.
  • Refresh tokens last 90 days, are not rotated and cannot be revoked except by removing the agent from the ACLs. Say so under Connected apps, or rotate on use.
  • /oauth/register has no rate limit (stateless, so only noise). /mcp does not check Origin, which the Streamable HTTP spec asks for; the bearer token covers it today.
  • The resource indicator (RFC 8707 resource) is not checked at the token endpoint.

Everything else I read looks right. The local stdio server and the CLI connect flow have a smaller surface; I did not find problems there beyond the grant model above. Held back from merge until the blocker is fixed.


Generated by Claude Code

joepio and others added 2 commits October 5, 2026 11:34
…writes signed as the issued agent

The hosted /mcp endpoint now has every tool, including create, edit and delete
when the person allowed editing; the node signs those commits as the issued
agent behind the token and sends them through the normal commit pipeline. The
local atomic-mcp is a stdio bridge to it with a token from an OAuth connect.

The consent page only accepts the node the person is signed in to, and only
redirects to exactly the registered redirect URI.

joepio commented Oct 5, 2026 •

Copy link
Copy Markdown
Member Author

Pushed b2f4176 (develop merged in), which answers the review above and unifies the two MCP implementations.

Blocker fixed. The consent page now refuses a server that is not the app's own origin, the node the app talks to, or the node the person's drive lives on, and says so instead of signing anything. redirect_url from the node is parsed and must equal the registered redirect_uri (scheme, host, path, no credentials) plus only code, state, iss or the original query; javascript: and friends are refused. Tests for both in hostedMcp.test.ts.

Smaller points. The redirect host is in the heading, and the node host is in the text. resource is now checked at the token endpoint (RFC 8707). Not done: Origin check on /mcp (the bearer token covers it), code reuse across a restart, refresh rotation, rate limit on /oauth/register; all noted in planning/mcp-endpoint.md.

One implementation. /mcp now has every tool the local server had, except semantic_search: query, get_user_classes, get_schema, and, only if the person allowed editing on the consent page, create_resource, edit_resource and delete_resource. A write is a commit the node signs as the issued agent (key derived again from the token's claims, never stored), through the normal commit pipeline, so the ACLs stay the limit and the person is never the signer. Markdown to document body moved to Rust (lib/src/document_markdown.rs). @tomic/mcp is now a stdio bridge to /mcp with a token from an OAuth connect (loopback redirect, PKCE), so the TS tool code, loro-crdt, zod and the MCP SDK are gone from it.

Checked: hosted-mcp.integration.test.ts against a real server covers read-only refusal, create (folder, document with text), edit, query, delete, writes outside the shared drive refused, revoke, and the bridge through the real OAuth flow. browser/mcp (8) and hostedMcp (7) unit tests, Rust unit tests, clippy, lint and typecheck pass. No CI run on this head yet.

joepio added 2 commits October 5, 2026 13:31
The consent page needs the identity the desktop app holds, which a browser
on the same machine does not. connect --desktop opens atomic://authorize-mcp,
the app routes it to the consent page, and after Allow it tells the loopback
listener and stays in the app instead of navigating away.

joepio commented Oct 8, 2026

Copy link
Copy Markdown
Member Author

Complexity review (about 8.4k changed lines; read the added files, not every test).

1. The compact JSON-AD layer now exists twice. The TS side (browser/lib/src/json-ad-compact.ts 487 lines, assistant-tools.ts 303) and the Rust side (server/src/mcp/compact.rs 343, tools.rs 561, write.rs 362) implement the same shortname/tag/date resolution and the same read, query and write verbs. coerce_value in compact.rs:152 is the coerceValueIn twin, and the tool descriptions in tools.rs repeat the TS ones. Every new verb or datatype is now two changes that can drift. Pick one home. If the node is meant to be "the one implementation" (as the bridge.ts header says), make the in-app assistant call /mcp too and delete assistant-tools.ts and useAtomicTools.ts's verbs, rather than keeping both.

2. Document body read and write live in two different crates. server/src/mcp/document_text.rs (149) reads the loro-prosemirror tree and lib/src/document_markdown.rs (558) writes it, each documented as the "mirror" of the other. They belong together in atomic_lib next to each other, so the Markdown round trip can be one test instead of a comment. The reader is also a Rust copy of documentText in browser/mcp (named in its header), so check that the TS one is actually gone.

3. Three auth/consent surfaces for one grant. server/src/mcp/oauth.rs (590), browser/mcp/src/oauth.ts (342), AuthorizeMcpRoute.tsx (282), ConnectAgentRoute.tsx (232), cli/src/connect.rs (144) and agent-grants.ts (212) all handle "approve this connection". If ConnectAgentRoute and AuthorizeMcpRoute differ only in where the token goes, merge them into one route with a target parameter.

4. Smaller: tokens.rs (191) plus agent-grants.ts hold the same grant shape twice, so generate the TS type from the Rust one or the reverse. hosted-mcp.integration.test.ts (529) has the bulk of the coverage but no Rust unit tests next to compact.rs, which is where item 1 will bite first.

Biggest win is item 1; items 2 and 3 are cheap if done in this PR and expensive after it lands.


Generated by Claude Code

joepio added a commit that referenced this pull request Oct 8, 2026
Combines #2123, #2127, #2133, #2134, #2135, #2136, #2138, #2140, #2141, #2142, #2143, #2147, #2149, #2132, #2121, #2129, #2146 and #1870. Includes fixes for the combination: Dagger mount for the form validation cases, drive mapping list without the flush sentinel, web_time in search, sidebar resize test at the touch bar height.

joepio commented Oct 8, 2026

Copy link
Copy Markdown
Member Author

Landed on develop through the batch #2153 (d102056), with this PR's head unchanged. Closing as merged.


Generated by Claude Code

@joepio joepio closed this Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants