[kovtcharov]

After finishing what the user asked, the flagship could start an unrelated job and run until cancelled. In one TUI session a one-line bugfix was done and verified in 40 s, followed by 16 minutes of "extract every source into an inventory". The trigger was the coding skill: its "find every call site" tip switched on document-inventory mode, so every coding session was pushed toward it. Now only a skill built on the extraction tool turns that mode on, and two general stop conditions end work the request didn't ask for.

Closes #4448

• Root cause: a skill enables inventorying only by listing extract_document_items in tools_required. None of the starter skills do, while real "list every item in this transcript" requests still get full, source-quoted extraction (fix(agents): verify completion and preserve exhaustive inventories #4374's guarantees are kept).
• Work after the answer: once the turn has answered, a tool call that touches nothing the request touched isn't run, and the model is told to finish and offer the work instead. A second such call ends the turn.
• A tool failing with new arguments each time is stopped like an identical repeat. The repeat guard used to miss it because the arguments varied.

Test plan

• pytest tests/unit/agents tests/unit/eval: 3001 passed, 78 skipped, 1 failed. The failure, test_symlinked_directory_is_the_same_output, also fails on untouched main on Windows (symlink privileges).

• New tests: turn scope (drift after the answer, failing-tool loop, what counts as related); the coding skill's wording no longer enables extraction while an extraction skill still does; code read along the way isn't an inventory source; bench-harness hooks.

• Before/after on DeepSeek V4.1 Flash via Fireworks, same machine, gaia eval tasks run --suite everyday --tasks 02-bugfix,01-refactor,b6-config-env --no-judge:

  	Before (main)	After
  Passed / verified by tests	3/3 · 3/3	3/3 · 3/3
  Steps	45	29
  Tokens	832k	544k
  Time	323 s	127 s

  n=1, but the deltas are far larger than run-to-run noise.

• python util/lint.py --black --isort.

• Judged quality pass (gaia eval tasks judge) once the judge CLI is signed in on the eval box.

[kovtcharov]

Blocking regression found in end-to-end testing: with this PR, the agent can finish a task, get sent back to verify it, and then be forbidden from doing so. It ends with a boilerplate "task is incomplete" message instead of its answer.

Measured on a combined build of the open fix PRs, DeepSeek V4.1 Flash, everyday suite, judged. 11-codereview passed on main this morning and fails with this PR:

• The model reviewed dates.py, found real bugs with a probe, then tried pytest, which was refused because the coding skill wasn't loaded.
• It called load_skill("coding"), and the turn scope refused it: "You already answered this request. load_skill is outside what the request touched."
• The completion checker then replaced the whole answer with "I couldn't verify completion… The task is incomplete…". The user got no review.

b3-triage shows the same pattern: after one gh issue list, browse_directory was refused as "already answered", and the reply became an "Incomplete extraction" notice.

Suggested fix:

1. When the loop itself re-prompts after an answer (verify-after-change, completion/extraction gaps), clear answered, or treat the re-prompt's required work as related to the request.
2. Treat load_skill, list_skills and skill_status as always related: they change what the agent can do, not what it touches.
3. Add a test for "answer → framework re-prompt → the work the re-prompt asks for is allowed".

🔍 Technical details

• agent.py (~9124): mark_answered() runs on the first answer_candidate, before _verify_after_change_prompt() and the completion checks decide to continue the turn.
• turn_scope.py: _ALWAYS_RELATED = {read_tool_output, request_user_input, session_findings}.
• Suite numbers on the combined build: 11/14 passed, quality 4.44, 109 steps, 584 s. On main: 10/14, 3.62, 223 steps, 2051 s. The step and time gains are real; these two tasks are the cost.

[kovtcharov]

Fixed the regression above: when the agent is sent back to run a check after answering, it can now load the skill that check needs and look inside the folder the prompt named. A working folder named in the prompt also no longer switches on document extraction, which is what sent b3-triage back to "inventory" a directory.

• New regression tests from both real transcripts (11-codereview, b3-triage); tests/unit/agents and tests/unit/eval pass (3004 passed; the one failure is a symlink test that needs Windows admin rights and is unrelated).
• Live rerun of 11-codereview and b3-triage on DeepSeek, queued behind the scenario run on the test box.

[kovtcharov-amd]

Found a real regression, not CI noise: any agent built without running Agent.__init__ crashes the instant a turn starts.

🔍 Technical details

_process_query_impl unconditionally calls self._turn_scope.begin_turn(...) (src/gaia/agents/base/agent.py:6608), but _turn_scope is only ever created in Agent.__init__ (self._turn_scope = TurnScopeGuard(max_consecutive_repeats)). Every other per-turn object with the same "only set in __init__" shape gets a defensive getattr(self, "_extraction_ledger", None) read elsewhere in this file; _turn_scope doesn't, and unlike _extraction_ledger it isn't recreated inside _process_query_impl itself, so there's no fallback when __init__ didn't run.

This reproduces deterministically in CI, not intermittently: Test Chat Agent → test_skill_body_filter_runs_before_the_turn_tool_filter →

AttributeError: 'ChatAgent' object has no attribute '_turn_scope'. Did you mean: '_read_scope'?

tests/unit/test_chat_dynamic_tools.py builds ChatAgent via __new__ on purpose (documented at the top of the file) so it can drive _process_query_impl without a Lemonade backend — that pattern is now broken by this PR.

Root-caused the rest of the red X's while I was in there, both unrelated to this diff:

• Linux / Windows-smoke / macOS-smoke are all the same test_client_disconnect_cancels_the_agent race (test(agent): Windows smoke - test_client_disconnect_cancels_the_agent races the LLM-call step #4420) — already fixed on main via fix(test): stop the disconnect race in test_client_disconnect_cancels_the_agent #4421, whose commit landed ~3s after this branch's tip. A rebase picks it up.
• pr-review never ran a model — its own log says You've hit your weekly limit, the known Anthropic quota outage (ci(eval): every eval gate dies in preflight — the judge org is out of monthly spend #4080).

The turn-scope-guard logic itself — the drift limit, the failure-streak stop, related()'s path/command matching — looks correct and is well covered by the new tests/unit/agents/test_turn_scope.py. Give _turn_scope a safe default (matching the _extraction_ledger pattern) and rebase onto current main, and this should be mergeable.

[kovtcharov]

Live recheck on DeepSeek V4.1 Flash, rebased onto today's main: both tasks that regressed now pass. 11-codereview passes at quality 5 (8 steps, 56 s), and b3-triage at quality 5 (6 steps, 38 s). Ready for review.

[github-actions]

Verdict: Approve with suggestions. Removing the coding skill's accidental trigger for inventory mode fixes the root cause, and the before/after bench numbers show it. One edge case can stop the agent partway through a fix it was asked to make.

🟡 A fix the agent is sent back to finish can be blocked halfway. The agent is marked "answered" just before it is asked to run the tests. If those tests fail in a file it hasn't touched yet, reading or editing that file counts as off-request work. The first attempt is refused and the second ends the turn, so the user gets a closing summary while the tests are still failing. A folder named in the prompt doesn't help: only that exact folder is in scope, not the files inside it. So "the tests in toybox/ fail, fix them" can't reach toybox/cli.py after the answer. The bench scoring replays the same rule and would mark such a fix as off-request.

Suggested fix: treat anything under a folder the request named as in scope. Also consider not blocking file work until the test re-run the agent was sent back for has finished. A test for the prompt → answer → failing pytest → edit a new file path would lock this down.

🟢 When the prompt names the working folder itself, its relative form is ., which appears in almost every shell command. That lets any command through once the turn has answered, so the drift guard is off in the "You are working in <dir>" prompts the bench uses.

🔍 Technical details

• Ordering: mark_answered() runs at src/gaia/agents/base/agent.py:9059, and the verify-after-change reprompt follows at :9068. Every call made to act on the re-run's result goes through TurnScopeGuard.check.
• Exact-match scope: TurnScopeGuard.related (src/gaia/agents/base/turn_scope.py:543) checks self._call_paths(args) & self.paths, an exact set intersection. With begin_turn("The tests in toybox/ fail. Fix them.", root), paths == {root/toybox}, so read_file("toybox/cli.py") is refused and a following edit_file sets turn_should_end. Tool results (for example, a failing path in pytest output) never widen paths. A prefix check would cover the named-folder case:

  call = self._call_paths(args)
  if any(c == p or c.startswith(p + os.sep) for c in call for p in self.paths):
      return True

  Also exempt the root itself (p == self.root); otherwise every path in the workdir matches.
• The . case: in the command-text branch, _relative(root, root) returns ".". "." in normalized is then true for nearly every command (cat x.txt, python -m ...). Skip rel in (".", "").
• calls_after_answer in src/gaia/eval/flagship_tasks.py reuses related(), so the bench inherits both behaviours.