ci: require extended tests in the merge queue - #25203
Conversation
|
Another think I was thinking was running only the required ci jobs on certain prs. For example a PR changing only docs should not run the full suite, and we can skip all the rust jobs on such prs. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #25203 +/- ##
==========================================
- Coverage 81.93% 81.93% -0.01%
==========================================
Files 1133 1133
Lines 423529 423529
Branches 423529 423529
==========================================
- Hits 347032 347024 -8
- Misses 55907 55914 +7
- Partials 20590 20591 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Yes, this is a good idea. I think it got disabled in the past because there is some doc pages, like the config settings, that actually do need to run the We originally had it in
I think the issue is something related to not being able to trigger for merge groups |
alamb
left a comment
There was a problem hiding this comment.
Thanks @kumarUjjawal -- the code makes sense to me (and Claude code)
| # Run extended tests (with feature 'extended_tests') | ||
| linux-test-extended: | ||
| name: cargo test 'extended_tests' (amd64) | ||
| if: github.event_name != 'pull_request' |
There was a problem hiding this comment.
I think it would help me to have a note here about what this is doing
Something like
# long running job: only run on merges to main and merge_queue| merging into `main`. All three jobs must pass: Rust tests with `extended_tests`, | ||
| forced hash-collision tests, and the [sqlite test suite]. | ||
|
|
||
| On ordinary PR updates, GitHub reports these jobs as skipped to conserve CI |
There was a problem hiding this comment.
I am not sure we need to talk about ". These skipped checks allow the PR to enter the merge queue, where the jobs run against the combined merge-group commit." -- that is perhaps a somewhat irrelevant implementation detail for contributors
| the entire [sqlite test suite]. You can run the extended tests | ||
| locally by following the [instructions in the documentation]. | ||
| DataFusion runs the extended tests in [extended.yml] in the merge queue before | ||
| merging into `main`. All three jobs must pass: Rust tests with `extended_tests`, |
There was a problem hiding this comment.
I don't think we need to enumerate the test names here
I do think it would be nice to keep the rationale that the tests take a long time to run but do often fail
|
Thank you @alamb |
|
Thank you! |
Which issue does this PR close?
Rationale for this change
Extended-test failures can reach
mainbecause the checks are optional on PRs and run after merge. Require all three suites in the merge queue so failures block merging without running the suites on every PR update.What changes are included in this PR?
pull_requesttrigger and amerge_grouptrigger. Skip the three jobs on PR updates so their required checks allow queue entry..asf.yaml.main. Preserve release-branch pushes and manual dispatch, including the optional PR head SHA.Test commands, feature flags, runners, concurrency settings, and benchmark checks are unchanged.
What is the testing strategy for this PR?
This change only affects CI scheduling and documentation, so it adds no Rust tests.
git diff --checkpass.parallelblocks on the base and this branch. Both workflows pass when those existing blocks are flattened in temporary lint inputs.The initial hosted PR run reports all three extended jobs as skipped. The hosted required-status-check validator also passes. Merge-group execution and enforcement still need validation when the PR enters the queue.
Are there any user-facing changes?
Contributors wait for extended tests in the merge queue instead of on PR updates or after merge. There are no query behavior or public API changes.