dream(performance): #3183 wire real cold-start measurement, remove fabricated V2-vs-V3 benchmark (evaluated, ACCEPT-scoped)

[ruvnet]

1. Hypothesis

Given the CLI cold-start benchmark suite (v3/@claude-flow/performance/benchmarks/startup/cli-cold-start.bench.ts), which currently reports a "V2 vs V3" speedup and other headline numbers purely from hardcoded setTimeout() delays without exercising any real code path, when the suite is rewired to call its own pre-existing (but dead) measureColdStart() function — spawning a real node bin/cli.js child process — and the fabricated V2-vs-V3 comparison is removed rather than "fixed" (no V2 binary remains to compare against), then the suite should report a genuine measured wall-clock number reflecting actual current CLI behavior for at least one real subcommand, subject to: (1) existing tests remain green; (2) the fix touches only the benchmark/test harness and one documentation table, not production CLI code; (3) $0 evaluation cost, zero LLM calls; (4) the honest scope limitation (measures --version only) is disclosed rather than hidden.

2. Candidate

• cli-cold-start.bench.ts: exported measureColdStart() (was private and dead — zero call sites before this diff); added runRealColdStartMeasurement() + CLI_BIN_PATH; wired the real measurement into runColdStartBenchmarks(); removed the "V2 vs V3 Comparison Simulation" block (setTimeout(100) vs setTimeout(20), no code between them — a "5.00x speedup" guaranteed by construction on every run).
• __tests__/cli-cold-start.test.ts (new): 4 deterministic Vitest assertions.
• ADR-STATUS-SUMMARY.md: corrected 2 rows in the Performance Targets table (CLI Startup — was "✅ Achieved" fed by the fabricated benchmark; HNSW — an already-known-false 150x-12,500x figure one row up, same table, same defect class).

3. Evaluation Receipt

Real evaluator: Vitest 4.1.8, deterministic, zero LLM calls, $0 cost.

Baseline vs. candidate isolated via git stash of just the .bench.ts file (test file kept): all 4 new tests fail against baseline (measureColdStart/runRealColdStartMeasurement/CLI_BIN_PATH not exported pre-fix; fabricated-pattern guard also fails) and pass against candidate.

Full @claude-flow/performance suite: 99/99 passing (95 pre-existing + 4 new; pre-existing 95 identical both ways). tsc --noEmit: zero errors.

Real measured number: node bin/cli.js --version, 5 runs — mean 54.65ms, min 51.94ms, max 57.92ms. Well under the 500ms target.

4. Baseline Comparison

	Baseline	Candidate
measureColdStart exported/callable	No (dead code)	Yes
Real subprocess-based number reported	No	Yes (54.65ms mean, n=5)
"V2 vs V3 Speedup"	5.00x, guaranteed by construction every run	Removed — no fabricated replacement
New regression tests	—	4/4 pass
Full package suite	95/95	99/99
tsc --noEmit	clean	clean

5. Darwin Lineage

Skipped — scope mismatch (dead-code-wiring fix with a binary "called or not" evaluation, not a tunable parameter with a search space). Confirmed real Darwin interface via npx ruvector harness darwin --help tonight (JSON genome config + --execute).

6. Flywheel Evidence

No signed @metaharness/flywheel bundle (deterministic Vitest evidence targets a different evaluation shape than the LLM-task corpus that tooling replays/verifies) — consistent with every accepted night since 2026-08-18. Evidence retained as: 4 new tests + linked issue + gist + stash-isolated baseline/candidate result.

7. Reward Hack Check

Manual checklist (no standalone reward-hack CLI reachable this session): no test weakened (purely additive), no gold-label leakage (none exists in this code), no cherry-picking (full 99-test suite both ways), no seed manipulation (real subprocess timing, variance disclosed via min/max), zero cost, no undocumented caching. The one real limitation (--version only, not full CLI init) is disclosed in-code and in the linked issue/gist, not hidden.

8. Security Review

runRealColdStartMeasurement() spawns a fixed, hardcoded command with shell: false (unchanged from the pre-existing helper) — no user-controlled input reaches the spawned args. Dev/benchmark tooling only, no new production-facing surface.

9. Regression Analysis

No production CLI code touched. ADR-STATUS-SUMMARY.md changes are documentation-only. Full package test suite green (99/99), tsc --noEmit clean, pre-existing 95 tests identical pass/fail both sides of the stash isolation.

10. ADR

None — scoped wiring/correctness fix + one doc-table correction, consistent with repo convention for this class of finding (no ADR for 08-24/08-25/08-30's similar fixes).

11. Research Gist

docs/dream-cycle/dream-gist-2026-09-05.md (this branch) — full competitor analysis, external sources (arXiv:2606.20695, Pebblous 2026 benchmark-trust-crisis report, Qdrant/Milvus/LangGraph/CrewAI/OpenAI comparisons), and this cycle's security (plugin supply-chain enforcement gap) and hive-mind (Raft election safety gap) scan findings, neither implemented tonight (out of scope for the performance DEEP surface, documented as follow-ups).

12. Issue

#3183

13. Witness

Field	Value
Session commit	db4991967c45c6f72133dff0bb80b0a492960fc1
Gist SHA-256 (pre-witness content)	0aae2712a54567521f308721205cbcbded0dadada2ab052810476b8e94d60fce
Witness stamp	04f903bab9006e79cf201c498058d028d66a0de7d6ef17e97325638336d1e98c

14. Merge Policy

Human review required. Do not self-merge. Do not autonomously promote Flywheel state.

---

🤖 Generated with RuFlo

https://claude.ai/code/session_01YQUz3bkMpqgtLXgkYa1LFF

---

Generated by Claude Code

[ruvnet]

CI red — not this PR's failure, confirmed via base-branch reproduction.

All 8 failing checks (Audit root (critical-blocking), Security & Code Quality, Test Suite (ubuntu-latest), 🚀 Integration Test Setup, and the queued/failed smoke-test matrix jobs) fail at the same root step: npm install at the repo root errors with ETARGET No matching version found for @claude-flow/mcp@3.0.0-alpha.10. This is a workspace-internal package version pinned in root package.json that isn't resolvable via plain npm install outside the workspace-link context.

Confirmed pre-existing and unrelated to this diff:

• CVE Audit Gate → Audit root (critical-blocking) fails identically on main at this PR's exact base commit (db4991967c45c6f72133dff0bb80b0a492960fc1) — run 33637170523, same "Install root dependencies (lockfile only)" step, same error.
• Already tracked: Unpublished @claude-flow/mcp alpha.10 pin blocks install-dependent CI #3095 ("Unpublished @claude-flow/mcp alpha.10 pin blocks install-dependent CI") and [verification] HIGH: Main CI broken — pnpm-lock.yaml outdated after @claude-flow/mcp pin to 3.0.0-alpha.10 #3101 ("Main CI broken — pnpm-lock.yaml outdated after @claude-flow/mcp pin to 3.0.0-alpha.10"), both still open.
• One attempted fix exists, chore: sync pnpm lockfile for mcp alpha.10 #3104, but it's incomplete for this failure: it only bumps the specifier in v3/pnpm-lock.yaml, not the root package.json/package-lock.json that npm install --package-lock-only at repo root actually reads — porting it here would not turn this green. I'm not widening this PR to fix an unrelated dependency-publishing/version-lockstep issue (needs either publishing @claude-flow/mcp@3.0.0-alpha.10 to the registry or repointing root's pin to a resolvable version — a repo-wide release-management fix, not a benchmark-harness fix).

Not re-running: this is a deterministic ETARGET failure with a known cause and no fix available to apply, not a flake — a re-run would reproduce the same error.

This PR's own diff (v3/@claude-flow/performance/**) was validated locally instead: full @claude-flow/performance suite 99/99 passing, tsc --noEmit clean, baseline-fails/candidate-passes confirmed via git stash isolation (details in the PR description and linked issue #3183/gist). Tracking #3095/#3101 for when this PR is actually mergeable against CI.

---

Generated by Claude Code