Skip to content

bug(web): readMemory() drops everything outside the co-web-notes block, so hand-written modes/_profile.md guardrails never reach the model #4003

Description

@rubicon

Code of Conduct

  • I agree to follow the Code of Conduct

What happened?

The anti-fabrication guardrails a user writes in modes/_profile.md never reach the model on a web-triggered run. Not truncated, not deprioritised: absent from the prompt entirely, with nothing anywhere saying so.

modes/_profile.md is the file AGENTS.md tells users to put their personalization in, and in practice it is where they write the rules that keep generated CVs, cover letters and form answers honest: what may not be claimed, which metrics are actually supported, which gaps are real. readMemory() supplies that content to buildPrompt(), buildAnswerPrompt() and the assistant/explore preambles. When it returns the empty string, buildPrompt() drops the whole "Durable notes about the user" section rather than emitting an empty one, so the resulting prompt carries no trace that anything was meant to be there.

The consequence is a candidate-safety one. A run that should have been constrained produces output that looks exactly like a correctly constrained run, and the only way to notice is to diff the prompt against a file nobody thinks to check. On the instance where this surfaced it went unnoticed through a full day of operation, and two separate output problems were misdiagnosed as the model ignoring its instructions when the instructions had never been sent.

The cause

web/src/lib/career-ops.ts:394 (unchanged on today's main):

export function readMemory(): string {
  try {
    const md = fs.readFileSync(profilePath(), "utf8");
    const i = md.indexOf(NOTES_START);
    const j = md.indexOf(NOTES_END);
    if (i !== -1 && j !== -1 && j > i) return md.slice(i + NOTES_START.length, j).trim();
  } catch {
    /* no _profile.md yet */
  }
  try {
    return fs.readFileSync(path.join(careerOpsRoot(), ".career-ops-web", "memory.md"), "utf8").trim();
  } catch {
    return "";
  }
}

with, at lines 388 and 389:

const NOTES_START = "<!-- co-web-notes:start -->";
const NOTES_END = "<!-- co-web-notes:end -->";

The reader only ever returns the marker-delimited block. Everything the user hand-wrote in the same file is discarded, and when the markers are absent the function does not fall back to the file it just read successfully: it falls through to .career-ops-web/memory.md, and returns "" when that does not exist.

That produces two failure states, and both are ordinary installs rather than edge cases.

No markers, which is the default. modes/_profile.template.md ships with no co-web-notes markers, and doctor.mjs auto-copies it to modes/_profile.md on first run. rememberFact() is the only thing that ever creates the block. So until a user happens to trigger the web assistant's remember path, a fully written modes/_profile.md returns "", permanently.

Markers present. Once rememberFact() has created the block, the read returns only the bullets the assistant wrote about the user, and still silently drops everything the user wrote. On the instance this was found on: a 180-line modes/_profile.md, 175 of those lines hand-written outside the block, and GET /api/memory returned only the four bullets inside it.

Nothing about modes/_profile.md signals that only a delimited block is read. It is documented and edited as a normal Markdown file, doctor.mjs warns when its content is unpersonalized, and the evaluate prompt tells the agent to go read it in full.

Blast radius

Five call sites take readMemory():

Call site Does the agent read modes/_profile.md itself?
/api/run kind evaluate Yes. The prompt says "read cv.md, config/profile.yml and modes/_profile.md", so this path degrades rather than losing the content
/api/run kind pdf No. Step 1 names modes/pdf.md, cv.md, config/profile.yml and the report
/api/apply/prefill No. buildAnswerPrompt() names cv.md and config/profile.yml
/api/assistant No. The preamble names cv.md, config/profile.yml, reports/, data/applications.md
/api/explore/ai No

For everything except evaluate, the injected memory slice is the only path by which that file can reach a model. CV tailoring and application-form prefill are on that list, which is to say the two surfaces whose output goes to a real employer.

Steps to reproduce

Either of these, against web 0.10.0 at afc5275d:

Default install. Start from a checkout where modes/_profile.md is doctor.mjs's auto-copy of the template, add real personalization content to it, and confirm .career-ops-web/memory.md does not exist. Start the web UI and curl localhost:3000/api/memory. Actual: {"memory":""}. The same value is what /api/run passes as memory, so the built prompt contains no profile content at all.

Managed block present. Trigger the assistant's remember path once so rememberFact() writes the block, then hand-write anything above it. curl localhost:3000/api/memory returns only the block's bullets; the hand-written content is not in the response and is not in any prompt.

Expected behavior

modes/_profile.md is a user-authored, in-scope file under AGENTS.md's Source-of-Truth Boundary. Its contents should reach the model, whether or not a machine-managed block happens to exist inside it.

Possible directions

Worth weighing, and I do not assume which one you want:

  1. Read the whole file. The markers stay a write-time construct for rememberFact(), which genuinely needs them to know where to append and how to dedupe, and stop being a read-time filter. The legacy .career-ops-web/memory.md fallback still applies when modes/_profile.md is missing or empty. This matches what a user writing that file plainly expects, and it is what AGENTS.md already promises about the file.

  2. Keep the strict slice and make the miss loud. doctor.mjs reports "profile notes are not being sent to the model" as an actionable finding, and the web Config surface shows it.

I lean towards (1), for a reason that is specific rather than aesthetic. A narrower version of (1), falling back to the whole file only when the markers are absent, reintroduces the same failure with a delay: the user's guardrails would be injected until the assistant remembers its first fact, at which point the block appears and the hand-written content silently disappears again. That cliff is the original bug wearing a different hat.

I also do not think (2) survives (1). Once the whole file is read, having no managed block is a perfectly ordinary state rather than a fault, so a doctor warning about it would be a false alarm on most installs. doctor.mjs already flags an unpersonalized modes/_profile.md, which covers the case where the file has no useful content in it at all.

Happy to be overruled on either point. Related but distinct: #3810 covers onboarding never filling modes/_profile.md in the first place.

Version / channel

web 0.10.0 · afc5275d (main, post-career-ops-v1.32.0)

Screen / route

/api/memory, /api/run, /api/apply/prefill, /api/assistant, /api/explore/ai

CLI tool

Claude Code

OS / Browser

macOS 26 / Node 24

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:webWeb/UI surface — owned by the career-ops-ui agent (first-party web)bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions