Repository navigation
Comment on a pull request what failed in CI and how to fix it, and welcome a first pull request - #1203
Merged
Conversation
…lcome a first pull request CI's last job writes a small report with the run's read-only token: the failing jobs and steps, the why-it-went-red verdict, the failing tests with what each said, and for a failed lint the biome --write command and its diff. A separate workflow on workflow_run reads it as data, without checking out or running anything of the pull request, finds the pull request by the run's head SHA and keeps one comment on it, rewritten after every run.
…at keep the tree's ledgers
…every control and format character, and skip Dependabot's runs Two open pull requests can share a head SHA; the run belongs to one head repository and branch, and only that pull request is commented on. The text of the report loses every Unicode Cc and Cf character (tag characters and the Arabic letter mark included) and the line and paragraph separators, keeping the tab and the line end. A run Dependabot started gets a read-only token, so the comment job is skipped for it.
… kept tab, the kept line end and the event with no head branch The report's text also loses U+E0000 to U+E01EF whole, unassigned points and supplementary variation selectors included; U+FE00 to U+FE0F stay for emoji. A case holds that a tab and a line end survive into the lint diff, and one that an event naming no head repository or branch comments nothing. The comments and the page say the pull request is found by head SHA, repository and branch, and the page names the limit of one branch open against two bases.
Member
Author
|
Review: approved at e5e301e. An independent review read the change against a hostile fork: nothing of the fork is checked out, installed or run in the job that holds This merges after #1204, which adds the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
When CI is over for a pull request, one comment on the pull request says what failed and how to fix it, rewritten in place after every run, and a contributor's first pull request is welcomed.
what-failedjob inci.yml(pull_request, read-only token:actions: read,contents: read; no secret). It waits for every other job and writes a small JSON artifact,what-failed: the failing jobs and steps (from the API), the why-it-went-red verdict of each cell, the failing tests with what each said (from the vitest JSON reports; the guards job now writes one too), and, when aLintstep failed, the filesbiome check --writechanges and the diff (cut at 30,000 characters). It iscontinue-on-error, so it never turns a run red..github/workflows/what-failed.yml(workflow_runon CIcompleted;pull-requests: writeis its only write, plusactions: readto download the other run's artifact andcontents: readfor the checkout of the default branch). It never checks out or runs the pull request's code: it checks out the default branch only (noref, norepository, no credential kept), installs nothing, downloads the artifact under$RUNNER_TEMP, and runs.github/what-failed/comment.mjson the event file and the report, with no${{ }}in the shell line.comment.mjsreads the report withJSON.parseonly, refuses it whole above 256 KiB or if a field is off its shape or limit (unknown fields included), and puts every text of it inside a fenced block longer than any backtick run in it, with control, zero-width and bidi characters removed and a zero-width space after every@. It finds the pull request by the run's head SHA through the API (workflow_run.pull_requestsis empty for forks) and keeps one comment, found by a hidden marker and by being authored bygithub-actions[bot]. OnFIRST_TIME_CONTRIBUTOR/FIRST_TIMERit opens with a welcome naming the author by the login the API gave (the only mention), pointing at CONTRIBUTING.md andpnpm check:contribution. A green run updates an existing comment to say so; it opens a new one only to welcome a first pull request..github/what-failed/README.md, with what it does not cover; a bullet in CONTRIBUTING.md and a changelog entry.Every action is pinned by commit SHA. No secret and no
pull_request_targetanywhere.Tests
the-comment-runs-nothing-of-the-fork.test.ts(reads the workflow files): the onlyworkflow_runworkflow is started by nothing else, holds no permission at file level and onlyactions: read,contents: read,pull-requests: writein its job; no checkout withref/repository, credentials not kept; the only shell line is the onenode .github/what-failed/comment.mjscall; the artifact goes outside the workspace; actions pinned;comment.mjsparses the report throughreadReportand runs no code (noeval,Function, dynamicimport, onlyghas a child); the CI job holds only reads andneedsevery other job. A last case feeds the reader each defect (checkout of the head, a write, a second trigger, a pasted expression, an install, a download into the workspace, a tag).the-pull-request-is-told-what-failed.test.ts: the reader refuses non-JSON, oversize, unknown fields, an over-limit message, a climbing or dashed path; the builder, given an@everyone, HTML, a</details><script>, a fence breaker and the marker itself, keeps all of it inside fences, lets no@of the report notify, carries no HTML outside a fence, and stays under 60,000 characters;tellwith a fake API finds the pull request by head SHA, rewrites its own comment and never one somebody else forged with the marker, skips a moved head and a cancelled run; the collector's output is accepted by the reader and stays within its limits.Mutation
Put a checkout of the head back in
.github/workflows/what-failed.yml(ref: ${{ github.event.workflow_run.head_sha }}on the checkout step):the-comment-runs-nothing-of-the-forkgoes red with.github/workflows/what-failed.yml: job comment checks out something else. Restored, green.Not proven here
GitHub starts a
workflow_runworkflow only from the file on the default branch, so the comment is first seen working on the first pull request after this merges.