Skip to content

fix(sched): preserve fair group state and streamline timer paths - #2573

Merged
fslongjin merged 5 commits into
DragonOS-Community:masterfrom
fslongjin:codex/sandbox-scheduling-timer-optimizations
Oct 28, 2026
Merged

fslongjin merged 5 commits into
DragonOS-Community:masterfrom
fslongjin:codex/sandbox-scheduling-timer-optimizations

Conversation

@fslongjin

@fslongjin fslongjin commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

This follows #2570 and fixes several scheduler/timer mechanisms exposed by snapshot-backed sandbox workloads. In particular, switching between Fair threads in one group currently renews the unchanged parent entity's RUN_TO_PARITY protection; an earlier eligible competitor can then lose the next selection. Preserve the common ancestors and update only the divergent entity chains.

The follow-up commit also:

  • Settles CFS bandwidth at switch-out and retains tick/pick quota enforcement while removing per-switch runtime-exhaustion event maintenance.
  • Uses cancellable monotonic deadlines for process nanosleep, including short sleeps, with fallible preparation, signal handling and a coarse-timer fallback. Hardware stalls and shutdown delays remain separate.
  • Adds capability-gated TSC-deadline programming, the APIC bus-timer fallback, terminal deadline clearing, and earliest-deadline-only local/remote reprogramming.
  • Keeps clocksource conversion state coherent and handles backward counter deltas without converting them into huge elapsed intervals.
  • Releases affinity-update serialization before waiting for placement and notifies after releasing the PI guard.
  • Removes unnecessary signal/ptrace return work while preserving pending-signal publication, saved-mask restoration and restart/freezer behavior.

Dependency and review scope

#2570 has merged into master as 37c7894. This branch is synchronized with that squash merge; the PR contains the follow-up code/tests and their dunitest CI configuration. The synchronization resolved duplicate prerequisite changes without changing the validated code tree. The subsequent feature-only adjustment separates scheduler fixtures from MM testing. The implementation is in commit 3006ca9.

Group-switch and affinity-placement fixtures are gated by the independent, default-disabled sched_selftest feature. dunitest CI explicitly enables mm_selftest,sched_selftest; ordinary production builds enable neither scheduler fixture.

The changes include production code and focused kernel selftests/dunitest coverage. No investigation reports, benchmark collectors, temporary probes, environment identifiers or test-machine configuration are included.

Validation

  • Fresh make kernel and FMT_CHECK=1 make fmt (including Clippy) passed.

  • The Fair-group witness fails its parent-token and competing-winner assertions before the fix and passes on the fixed 2/4-vCPU kernels.

  • Existing focused 2/4-vCPU guest coverage: 505 GTest executions passed, covering timekeeping/scheduling, group quota, memory.high and cpuset; this is not a claim that the entire repository suite passed.

  • Fresh normal-kernel 2-vCPU regression: 27 group-quota/memory.high tests and 15 cpuset tests passed, no skips; both QEMU instances exited cleanly and the input disk was unchanged.

  • Three independent review personas checked correctness, security/concurrency, Linux semantics, lifecycle and performance; no confirmed new blocking findings.

  • Feature separation validation: default and sched-only kernel builds, mm-only and combined Cargo checks, and format checking passed. The sched-only 2-vCPU guest passed both scheduler fixtures with cleanup; the default ELF contains neither fixture's symbols.

Performance limits

A fixed normal-kernel comparison ran 18 batches with 360 measured creates and 90 warmups at concurrency 1/5/10. All completed, but the Fair-group change's pooled create medians versus its immediate predecessor changed by +11.63%, -51.59% and +3.92%, respectively; the two c5 rounds had opposite directions. A separate handoff microbenchmark improved by 2%–16% across four cases. These observations do not establish stable end-to-end gains or creation faster than Linux. Host contention remains an expected source of variability; adverse samples are retained.

The earlier fallback/watchdog and host-counter limitations documented in #2570 remain relevant. This PR does not claim to repair arbitrary incoherent VMM snapshot restoration or to close the broader sandbox performance investigation.

Use the registered per-vCPU pvclock for scheduler timestamps independently
of the POSIX clocksource selection. Keep sampling and local anchor updates
on one CPU, and re-anchor source transitions without charging raw epoch
differences to task runtime. Preserve the native TSC fallback.

Register permanent, aligned KVM steal-time records before scheduling starts
on each CPU. Read cumulative host runqueue delay with the KVM version
protocol and subtract it from the target runqueue's task clock after IRQ
time. Account CPU usage through an independent cursor that retains bounded
tick balances and handles a cumulative-source reset without underflow.

Revoke both KVM time registrations before CPU terminal-state publication
and before kexec reuses old RAM. Reuse the existing peer stop protocol,
prevent relocation when peers have not stopped, and use an explicit
non-returning halt rather than a panic that may unwind. Describe the
relocation assembly's one-way transfer with a non-returning function type.

Extend the existing read-only timekeeping selftests and dunitest coverage
for clock-source handoff, bounded and independent steal consumption,
cumulative resets, the shared ABI reader, and terminal CPU state classes.

Validation: kernel build, format/Clippy, RISC-V and LoongArch checks;
focused 2/4-vCPU guest regression; final pause, host-contention, SDK and
2/4-vCPU replacement-kernel kexec tests. Non-PV CPU timer watchdog tests
have intermittent failures in both baseline and candidate, including one
candidate parameter not reproduced on baseline; retain this uncertainty
rather than asserting a fully clean fallback regression result.

Signed-off-by: longjin <longjin@dragonos.org>
Integrate current upstream changes before the scheduler and timer follow-up.

Signed-off-by: longjin <longjin@dragonos.org>
Keep common Fair ancestors selected when switching between threads in the
same group. Only put/set divergent entity chains, so a thread handoff does
not renew an ancestor's RUN_TO_PARITY protection and hide an earlier
eligible competitor. Settle bandwidth on entity switch-out and retain the
normal tick/pick quota enforcement instead of maintaining and reprogramming
a runtime exhaustion event on every context switch.

Make process nanosleep use cancellable monotonic deadlines, including short
sleeps, with a coarse-timer fallback and fallible preparation. Recheck the
original absolute endpoint after notifications and signal races. Keep ACPI
Stall and shutdown delays on a separate calibrated busy-wait path.

Use TSC-deadline mode when available, with the existing APIC bus-timer
fallback, required ordering, and explicit deadline clearing on terminal
CPU paths. Reprogram deadline owners only when the earliest event changes,
including remote-owner notification. Keep clocksource conversion updates
coherent and reject backward counter deltas instead of fabricating a wrap.

Release affinity-update serialization before waiting for task placement,
and notify placement waiters after releasing the task PI guard. Avoid
unnecessary signal/ptrace return work while preserving pending-signal
publication, mask restoration, restart and freezer responsibilities.

Add focused kernel selftests and dunitest coverage for group selection,
affinity placement, signal races, short sleeps and timekeeping boundaries.
The normal scheduler paths contain no temporary tracing instrumentation.

Validation: make kernel; FMT_CHECK=1 make fmt including Clippy; independent
three-persona review; existing focused 2/4-vCPU regression with 505 GTest
executions. Group-selection assertions fail before the fix and pass after.
Creation benchmarks remain mixed, including adverse c1/c10 medians; this
change does not claim consistent end-to-end creation gains over Linux.

Fresh normal-kernel 2-vCPU validation: 27 quota/memory.high and 15 cpuset
tests passed without skips; both guest instances exited cleanly.

Signed-off-by: longjin <longjin@dragonos.org>
@github-actions github-actions Bot added the Bug fix A bug is fixed in this pull request label Oct 28, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-28T16:08:14.775638Z
We couldn’t complete this review. Comment @codex review to try again.
0a119ec Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

An unknown error occurred
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Resolve duplicate prerequisite changes after the squash merge of DragonOS-Community#2570. The upstream tree matches the previously integrated prerequisite tree; preserve the validated follow-up tree without code changes.

Signed-off-by: longjin <longjin@dragonos.org>
Add a default-disabled sched_selftest feature for group-switch and affinity
placement fixtures instead of coupling scheduler tests to mm_selftest.
Move both module/call gates and their test-only PID/wait-queue helpers to
the new feature. Keep it independent of MM test configuration.

Enable both features explicitly in dunitest CI and update the userspace
skip contract. Production scheduler algorithms and default features are
unchanged.

Validation: default and sched-only make kernel; mm-only and combined
Cargo checks; cargo fmt --check. A 2-vCPU sched-only guest passed the real
group-selection and affinity-placement fixtures with cleanup. The default
ELF contains neither fixture's symbols. Independent plan/code review passed.

Signed-off-by: longjin <longjin@dragonos.org>
@github-actions github-actions Bot added the test Unitest/User space test label Oct 28, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

An unknown error occurred
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@fslongjin
fslongjin merged commit 9894665 into DragonOS-Community:master Oct 28, 2026
18 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug fix A bug is fixed in this pull request test Unitest/User space test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant