Skip to content

fix: show attempts in test error logs of retried tests - #11696

Open
hi-ogawa wants to merge 41 commits into
mainfrom
fix/issue-11454
Open

hi-ogawa wants to merge 41 commits into
mainfrom
fix/issue-11454

Conversation

@hi-ogawa

@hi-ogawa hi-ogawa commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Currently, the failure summary prints one FAIL header per occurrence of an error, so retried tests and repeated expect.soft errors show identical header lines with no explanation. This PR prints one header per test instead, with a label:

  • initial run, 1st retry, 2nd repeat, or 1st retry of 1st repeat names each attempt that produced the error, shown when the test ran more than once
  • N times counts an error repeated within one attempt

Retry

test('always fails', { retry: 2 }, () => {
  expect(1).toBe(2)
})

🔴 Before:

 FAIL  basic.test.js > always fails
 FAIL  basic.test.js > always fails
 FAIL  basic.test.js > always fails
AssertionError: expected 1 to be 2 // Object.is equality

🟢 After:

 FAIL  basic.test.js > always fails (initial run, 1st retry, 2nd retry)
AssertionError: expected 1 to be 2 // Object.is equality

Soft

test('soft', () => {
  for (let i = 0; i < 2; i++) {
    expect.soft(1).toBe(2)
  }
})

🔴 Before:

 FAIL  basic.test.js > soft
 FAIL  basic.test.js > soft
AssertionError: expected 1 to be 2 // Object.is equality

🟢 After:

 FAIL  basic.test.js > soft (2 times)
AssertionError: expected 1 to be 2 // Object.is equality

Examples

Retry errors at the same location

image

Retry errors at the different locations

image

hi-ogawa and others added 26 commits October 20, 2026 15:57
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hi-ogawa and others added 3 commits October 22, 2026 12:34
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hi-ogawa and others added 6 commits October 22, 2026 13:02
…etry

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hi-ogawa
hi-ogawa marked this pull request as ready for review October 22, 2026 06:59
Base automatically changed from fix/issue-11614 to main October 23, 2026 01:13
# Conflicts:
#	packages/vitest/src/node/reporters/base.ts
#	test/browser/specs/failure-screenshot.test.ts
@netlify

netlify Bot commented Oct 23, 2026

Copy link
Copy Markdown

✅ Deploy Preview for vitest-dev ready!

Name Link
🔨 Latest commit f6ea3db
🔍 Latest deploy log https://app.netlify.com/projects/vitest-dev/deploys/6adad7c68eec8928039adefa
😎 Deploy Preview https://deploy-preview-11696--vitest-dev.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

hi-ogawa and others added 5 commits October 23, 2026 15:37
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@hi-ogawa

Copy link
Copy Markdown
Collaborator Author

@sheremet-va Terminal output screenshot attached! 📸

@hi-ogawa
hi-ogawa requested a review from sheremet-va October 23, 2026 08:57

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Failed test's FAIL header is printed once per retry attempt

1 participant