skills: wait-for-notification build/test instructions for hosts with wakeup

[gontzess]

Why

Hosts such as c1aw can resume the connector-authoring specialist when a build or draft test completes. Polling instructions currently override that host contract and spend turns unnecessarily.

What

The orchestrator and build-and-test stage wait for completion notifications when the host provides wakeup. Direct API/BYO callers without wakeup still poll get_run or get_test_run_evidence with a bounded wait. Activation remains a human-only boundary and is unchanged.

Validation

• npm ci
• npm run typecheck:decl (passed)
• npm run typecheck:parity (passed)

Companion to CXF-353 PR 1 in ductone/c1; this is a new skills-only change after connector-authoring#20. Do not merge automatically.

[linear-code]

CXF-353

[github-actions]

Superseded — see the current review report for commit 83840c146951

General PR Review: skills: wait-for-notification build/test instructions for hosts with wakeup

Blocking Issues: 0 | Suggestions: 3 | Threads Resolved: 0
Criteria: Criteria status: none loaded - .claude/skills/ci-review.md was not found at trusted base 047cbde673c4.
Review mode: full
View review run

Review Summary

A docs-only change to skills/author-in-app-connector/SKILL.md (S5 build result, S10 evidence) and skills/build-and-test/SKILL.md (steps 3–4 and the frontmatter description). Agents now wait for a host completion notification when the host provides wakeup. Callers without wakeup still poll get_run / get_test_run_evidence, now with "a bounded wait" and "report pending, don't redispatch". The activation boundary is untouched, so the change matches the PR description. I scanned the full diff for security and correctness. I also checked that the locked literals in evals/runner/skills_bundle.test.ts (RUN_STATE_SUCCEEDED, test_run_id, Do not redeem the approval token, etc.), the section markers, the ASCII-only rule and the 200-line bound still hold. No repo-local criteria loaded, so only the base criteria applied. I did not run the eval tests (this is a read-only review).

Security Issues

None found.

Correctness Issues

None found.

Suggestions

• New (medium confidence): skills/build-and-test/SKILL.md:29-44 and skills/author-in-app-connector/SKILL.md:42-45,56-59 — "do not poll while waiting" has no fallback if the wakeup host never delivers the notification, so the agent can stall indefinitely. The edit also replaces the concrete bound ("every 5-10s", "~10 polls") with an undefined "bounded wait", which makes direct-caller behavior less deterministic.
• New (low-medium confidence): skills/build-and-test/SKILL.md:3, skills/author-in-app-connector/SKILL.md:4, evals/skills-bundle/bundle.json — the skill bodies changed but their versions (0.2.0 / 0.2.1 / bundle 0.4.0) were not bumped. Eval records and consumers can't tell the polling contract apart from the notification contract.
• New (low confidence): skills/diagnose-authoring-failure/SKILL.md:42-45 — this skill still tells agents to poll get_test_run_evidence unconditionally, which contradicts the new wakeup-host contract on that path.

Prompt for AI agents

Verify each finding against the current code and only fix it if needed.

## Suggestions

In `skills/build-and-test/SKILL.md`:
- Around line 29-44: For hosts with wakeup, add a fallback: on resume, or if no completion notification arrives within a stated timeout, call `c1_connector_authoring_get_run` / `c1_connector_authoring_get_test_run_evidence` once to confirm state before continuing or reporting pending. For direct API callers, restore a concrete bound (e.g. backoff every 5-10s, report pending after ~10 polls) instead of the undefined "bounded wait".

In `skills/author-in-app-connector/SKILL.md`:
- Around line 42-45 and 56-59: Mirror the same fallback and concrete polling bound in S5 and S10.

In `skills/build-and-test/SKILL.md`, `skills/author-in-app-connector/SKILL.md`, and `evals/skills-bundle/bundle.json`:
- Around line 3-4 / bundle entries: Bump the frontmatter versions of both changed skills, update the matching bundle.json entries, and consider bumping the bundle version (and the version pinned in evals/scenarios/tier1-directory-full.json plus the VERSION constant and assertion in evals/runner/skills_bundle.test.ts), so the behavior change can be traced.

In `skills/diagnose-authoring-failure/SKILL.md`:
- Around line 42-45: Qualify the evidence-polling instruction the same way: wait for the host completion notification if one is provided, and poll only when the host has no wakeup.

[github-actions]

🟡 Suggestion (medium confidence): If the host has wakeup but the notification never arrives (dropped event, host restart), "do not poll while waiting" leaves the agent with no way out. Add a one-time get_run / get_test_run_evidence check on resume or after a timeout. This edit also drops the concrete bound ("every 5-10s", "~10 polls") in favor of an undefined "bounded wait". Keeping the numbers would keep direct-caller behavior deterministic. The same applies to step 4 and to S5/S10 in author-in-app-connector.

[github-actions]

🟡 Suggestion (low-medium confidence): The skill body's behavior changed, but version: 0.2.0 here, author-in-app-connector 0.2.1, and evals/skills-bundle/bundle.json (0.4.0) were not bumped. skills_bundle.test.ts treats bundle.json as the version source of truth for skill bodies. Without a bump, eval baselines and consumers can't tell the polling and notification variants apart. Consider bumping both skills, their bundle entries, and the bundle/scenario version.

[github-actions]

Superseded — see the current review report for commit 78796aeb645d

General PR Review: skills: wait-for-notification build/test instructions for hosts with wakeup

Blocking Issues: 0 | Suggestions: 3 | Threads Resolved: 0
Criteria: Criteria status: none loaded - .claude/skills/ci-review.md was not found at trusted base 047cbde673c4.
Review mode: full
View review run

Review Summary

A docs-only change to skills/author-in-app-connector/SKILL.md (S5 build result, S10 evidence) and skills/build-and-test/SKILL.md (steps 3–4 and the frontmatter description). Agents now wait for a host completion notification when the host provides wakeup. Callers without wakeup still poll get_run / get_test_run_evidence, now with "a bounded wait" and "report pending, don't redispatch". The activation boundary is untouched, so the change matches the PR description. I scanned the full diff for security and correctness. I also checked that the locked literals in evals/runner/skills_bundle.test.ts (RUN_STATE_SUCCEEDED, test_run_id, Do not redeem the approval token, etc.), the section markers, the ASCII-only rule and the 200-line bound still hold. No repo-local criteria loaded, so only the base criteria applied. I did not run the eval tests (this is a read-only review).

Security Issues

None found.

Correctness Issues

None found.

Suggestions

• New (medium confidence): skills/build-and-test/SKILL.md:29-44 and skills/author-in-app-connector/SKILL.md:42-45,56-59 — "do not poll while waiting" has no fallback if the wakeup host never delivers the notification, so the agent can stall indefinitely. The edit also replaces the concrete bound ("every 5-10s", "~10 polls") with an undefined "bounded wait", which makes direct-caller behavior less deterministic.
• New (low-medium confidence): skills/build-and-test/SKILL.md:3, skills/author-in-app-connector/SKILL.md:4, evals/skills-bundle/bundle.json — the skill bodies changed but their versions (0.2.0 / 0.2.1 / bundle 0.4.0) were not bumped. Eval records and consumers can't tell the polling contract apart from the notification contract.
• New (low confidence): skills/diagnose-authoring-failure/SKILL.md:42-45 — this skill still tells agents to poll get_test_run_evidence unconditionally, which contradicts the new wakeup-host contract on that path.

Prompt for AI agents

Verify each finding against the current code and only fix it if needed.

## Suggestions

In `skills/build-and-test/SKILL.md`:
- Around line 29-44: For hosts with wakeup, add a fallback: on resume, or if no completion notification arrives within a stated timeout, call `c1_connector_authoring_get_run` / `c1_connector_authoring_get_test_run_evidence` once to confirm state before continuing or reporting pending. For direct API callers, restore a concrete bound (e.g. backoff every 5-10s, report pending after ~10 polls) instead of the undefined "bounded wait".

In `skills/author-in-app-connector/SKILL.md`:
- Around line 42-45 and 56-59: Mirror the same fallback and concrete polling bound in S5 and S10.

In `skills/build-and-test/SKILL.md`, `skills/author-in-app-connector/SKILL.md`, and `evals/skills-bundle/bundle.json`:
- Around line 3-4 / bundle entries: Bump the frontmatter versions of both changed skills, update the matching bundle.json entries, and consider bumping the bundle version (and the version pinned in evals/scenarios/tier1-directory-full.json plus the VERSION constant and assertion in evals/runner/skills_bundle.test.ts), so the behavior change can be traced.

In `skills/diagnose-authoring-failure/SKILL.md`:
- Around line 42-45: Qualify the evidence-polling instruction the same way: wait for the host completion notification if one is provided, and poll only when the host has no wakeup.

---

Reviewed commit: 83840c146951

[github-actions]

No blocking issues found — see the full review report

[github-actions]

General PR Review:

⏳ Provisional — deeper review still in progress.

Review in progress; this comment will be replaced with the full summary.

[github-actions]

Superseded — see the current review report for commit 78796aeb645d

General PR Review: skills: wait-for-notification build/test instructions for hosts with wakeup

Blocking Issues: 0 | Suggestions: 3 | Threads Resolved: 0
Criteria: Criteria status: none loaded - .claude/skills/ci-review.md was not found at trusted base 047cbde673c4.
Review mode: incremental since 83840c1
View review run

Review Summary

This PR changes when the agent waits for build and draft-test results. In the build-and-test skill (steps 3/4) and the author-in-app-connector skill (S5/S10), agents on hosts with wakeup now wait for a completion notification instead of polling. Direct API callers still poll get_run / get_test_run_evidence.

The new commits since 83840c1 do three things:

• Add a fallback for the wakeup path: when a session resumes and no notification has arrived, check the run once.
• Bring back the concrete polling limit for direct callers: every 5-10s, up to ~10 polls, then STOP and report.
• Bump build-and-test and author-in-app-connector to 0.3.0 and the bundle to 0.5.0, with matching updates to the eval scenarios and test pins.

I scanned the full PR diff (9 files) for security and correctness problems. The incremental diff was complete (partial: false), so nothing was left out. I also checked that the locked literals in skills_bundle.test.ts still appear in the edited skills. No repo-local criteria loaded, so I applied only the base criteria. Activation is still a human-only step, as the PR description says.

Security Issues

None found.

Correctness Issues

None found.

Suggestions

• New (medium confidence): skills/build-and-test/SKILL.md:31-33 and :42-44, skills/author-in-app-connector/SKILL.md:43-45, :59-61. "Check ... once before re-dispatching" doesn't say what to do when the run is still RUNNING or PENDING. It reads as if re-dispatching always follows, which contradicts "rather than redispatching" a few lines later. An agent that resumes mid-build could start a duplicate build or test.
• New (high confidence): evals/skills-bundle/README.md:5,19,64 and evals/README.md:18. These still describe v0.4.0 as the current bundle and say the earlier skills are "unchanged", but bundle.json is now 0.5.0 and two skills changed in this PR.
• Prior — still present (low confidence): skills/diagnose-authoring-failure/SKILL.md:42-45. This skill still tells agents to poll c1_connector_authoring_get_test_run_evidence with no exception for hosts with wakeup. Its literals "Poll with backoff" and "row after ~10 polls, stop and report" are locked in skills_bundle.test.ts:177-178, so changing the skill means updating the test too.

Resolved prior findings

• Fixed: no fallback when a wakeup notification never arrives, and the undefined "bounded wait". The resume check and the "every 5-10s, up to ~10 polls" limit now appear at skills/build-and-test/SKILL.md:31-38,42-49 and skills/author-in-app-connector/SKILL.md:43-46,59-63.
• Fixed: skill versions were not bumped. Now skills/build-and-test/SKILL.md:4 and skills/author-in-app-connector/SKILL.md:4 are at 0.3.0, and evals/skills-bundle/bundle.json:2,4,7 puts the bundle at 0.5.0, with the scenario and test pins updated to match.

Prompt for AI agents

Verify each finding against the current code and only fix it if needed.

## Suggestions

In `skills/build-and-test/SKILL.md`:
- Around line 31-33 and 42-44: The resume fallback says "check ... once before re-dispatching", which implies a re-dispatch always follows. Reword it to give the action for each result: if the run is terminal, continue to the GATE; if it is still RUNNING/PENDING (or evidence is NotFound while pending), keep waiting or report pending and do not redispatch; re-dispatch only if the run/test is not found or has failed. Apply the same wording in `skills/author-in-app-connector/SKILL.md` around lines 43-45 (S5) and 59-61 (S10).

In `evals/skills-bundle/README.md`:
- Around lines 5, 19, 64: Add a "v0.5.0" section above v0.4.0 that notes build-and-test and author-in-app-connector were bumped to 0.3.0 for wait-for-notification build/test behavior, and change the pinned `skillBundle.version` reference at line 64 to `0.5.0`. Also update `evals/README.md` line 18 ("v0.4.0 manifest") to v0.5.0.

In `skills/diagnose-authoring-failure/SKILL.md`:
- Around line 42-45: The draft-test FAIL reading tells agents to poll get_test_run_evidence with no exception for hosts with wakeup. Optionally add: "On hosts that deliver completion notifications, read the evidence once after the notification instead of polling." Keep the locked literals "Poll with backoff" and "row after ~10 polls, stop and report" so `evals/runner/skills_bundle.test.ts` still passes.

[github-actions]

🟡 Suggestion (medium confidence): "check get_run once before re-dispatching" reads as if re-dispatching comes next no matter what the check returns. That conflicts with the direct-caller rule a few lines down ("rather than redispatching the build"). Please say what to do for each result: continue on terminal, and wait or report pending on RUNNING/PENDING. Only re-dispatch when the run is not found. Otherwise an agent that resumes mid-build may start a duplicate build. The same wording appears in step 4 and in author-in-app-connector S5/S10.

[github-actions]

🟡 Suggestion (high confidence): the bundle moves to 0.5.0, but evals/skills-bundle/README.md still documents v0.4.0 as current. Line 19 says "The seven prior skills are unchanged; the bundle version is 0.4.0", and line 64 says scenarios pin 0.4.0. evals/README.md:18 also says "v0.4.0 manifest". Please add a v0.5.0 section that notes the build-and-test and author-in-app-connector 0.3.0 bumps, and update the pinned-version references.

[github-actions]

General PR Review: skills: wait-for-notification build/test instructions for hosts with wakeup

Blocking Issues: 0 | Suggestions: 3 | Threads Resolved: 0
Criteria: Criteria status: none loaded - .claude/skills/ci-review.md was not found at trusted base 047cbde673c4.
Review mode: incremental since 83840c1
View review run

Review Summary

This PR changes when the agent waits for build and draft-test results. In the build-and-test skill (steps 3/4) and the author-in-app-connector skill (S5/S10), agents on hosts with wakeup now wait for a completion notification instead of polling. Direct API callers still poll get_run / get_test_run_evidence.

The new commits since 83840c1 do three things:

• Add a fallback for the wakeup path: when a session resumes and no notification has arrived, check the run once.
• Bring back the concrete polling limit for direct callers: every 5-10s, up to ~10 polls, then STOP and report.
• Bump build-and-test and author-in-app-connector to 0.3.0 and the bundle to 0.5.0, with matching updates to the eval scenarios and test pins.

I scanned the full PR diff (9 files) for security and correctness problems. The incremental diff was complete (partial: false), so nothing was left out. I also checked that the locked literals in skills_bundle.test.ts still appear in the edited skills. No repo-local criteria loaded, so I applied only the base criteria. Activation is still a human-only step, as the PR description says.

Security Issues

None found.

Correctness Issues

None found.

Suggestions

• New (medium confidence): skills/build-and-test/SKILL.md:31-33 and :42-44, skills/author-in-app-connector/SKILL.md:43-45, :59-61. "Check ... once before re-dispatching" doesn't say what to do when the run is still RUNNING or PENDING. It reads as if re-dispatching always follows, which contradicts "rather than redispatching" a few lines later. An agent that resumes mid-build could start a duplicate build or test.
• New (high confidence): evals/skills-bundle/README.md:5,19,64 and evals/README.md:18. These still describe v0.4.0 as the current bundle and say the earlier skills are "unchanged", but bundle.json is now 0.5.0 and two skills changed in this PR.
• Prior — still present (low confidence): skills/diagnose-authoring-failure/SKILL.md:42-45. This skill still tells agents to poll c1_connector_authoring_get_test_run_evidence with no exception for hosts with wakeup. Its literals "Poll with backoff" and "row after ~10 polls, stop and report" are locked in skills_bundle.test.ts:177-178, so changing the skill means updating the test too.

Resolved prior findings

• Fixed: no fallback when a wakeup notification never arrives, and the undefined "bounded wait". The resume check and the "every 5-10s, up to ~10 polls" limit now appear at skills/build-and-test/SKILL.md:31-38,42-49 and skills/author-in-app-connector/SKILL.md:43-46,59-63.
• Fixed: skill versions were not bumped. Now skills/build-and-test/SKILL.md:4 and skills/author-in-app-connector/SKILL.md:4 are at 0.3.0, and evals/skills-bundle/bundle.json:2,4,7 puts the bundle at 0.5.0, with the scenario and test pins updated to match.

Prompt for AI agents

Verify each finding against the current code and only fix it if needed.

## Suggestions

In `skills/build-and-test/SKILL.md`:
- Around line 31-33 and 42-44: The resume fallback says "check ... once before re-dispatching", which implies a re-dispatch always follows. Reword it to give the action for each result: if the run is terminal, continue to the GATE; if it is still RUNNING/PENDING (or evidence is NotFound while pending), keep waiting or report pending and do not redispatch; re-dispatch only if the run/test is not found or has failed. Apply the same wording in `skills/author-in-app-connector/SKILL.md` around lines 43-45 (S5) and 59-61 (S10).

In `evals/skills-bundle/README.md`:
- Around lines 5, 19, 64: Add a "v0.5.0" section above v0.4.0 that notes build-and-test and author-in-app-connector were bumped to 0.3.0 for wait-for-notification build/test behavior, and change the pinned `skillBundle.version` reference at line 64 to `0.5.0`. Also update `evals/README.md` line 18 ("v0.4.0 manifest") to v0.5.0.

In `skills/diagnose-authoring-failure/SKILL.md`:
- Around line 42-45: The draft-test FAIL reading tells agents to poll get_test_run_evidence with no exception for hosts with wakeup. Optionally add: "On hosts that deliver completion notifications, read the evidence once after the notification instead of polling." Keep the locked literals "Poll with backoff" and "row after ~10 polls, stop and report" so `evals/runner/skills_bundle.test.ts` still passes.

---

Reviewed commit: 78796aeb645d

[github-actions]

No blocking issues found — see the full review report