Skip to content

fix(mm): recover physical allocations at lock-free transaction boundaries - #2580

Merged
fslongjin merged 2 commits into
DragonOS-Community:masterfrom
fslongjin:codex/mm303-global-allocation-recovery
Nov 4, 2026
Merged

fslongjin merged 2 commits into
DragonOS-Community:masterfrom
fslongjin:codex/mm303-global-allocation-recovery

Conversation

@fslongjin

Copy link
Copy Markdown
Member

File-cache allocation can fail at the physical frame admission boundary after memory charging succeeds. Previously that failure became ordinary ENOMEM, so file I/O returned immediately even when reclaimable cache remained. This can surface as SQLite IOERR_WRITE or dependency-file read failures under memory pressure.

Introduce explicit physical-allocation errors and compose the existing charge/global recovery machinery at resource-lock-free transaction boundaries. Recovery has a fixed 10-second transaction budget; IRQ, non-schedulable and NOIO contexts cannot enter it. Ordinary system errors, VA exhaustion and structurally unsupported physical layouts stay terminal. GlobalAlloc remains nonblocking.

Preserve published write prefixes, deferred completions and exact synchronization ranges across recovery. Propagate loop backing failures out of NOIO and handle physical fault-retry failures without misclassifying them as SIGBUS. Most call-site changes migrate the existing recovery state rather than add new recovery loops.

Validation:

  • Default and mm_selftest kernel builds, RISC-V compile check, make fmt and FMT_CHECK=1 make fmt including all-features clippy passed.
  • Same 1 GiB/ext4 regression with 128 MiB existing extents and 700 MiB resident anonymous memory: baseline pwrite returned ENOMEM; fixed kernel completed overwrite, fsync, thread lifetimes, full reverse data validation and anonymous-memory validation without an OOM kill. Hardware breakpoints confirmed six physical recovery entries from File::do_write_user, without production instrumentation.
  • Kernel-buffer, page-cache/slab accounting, global reclaim including memory.low, pwritev2, fdatasync, file-lock and loop memory-I/O guest regressions passed (44 targeted cases).
  • Same SQLite workload with 736 MiB anonymous memory and 128 MiB reclaimable cache: baseline failed reading the cache file with ENOMEM; fixed kernel completed transactions, checkpoint and integrity_check at roughly 6 MiB free memory with 60 MiB file cache.
  • Original OpenCode runtime completed one initial tool workflow and one restarted session-resume workflow in a 1 GiB guest, with unchanged inputs and no managed-process leaks.

Limits: the larger sparse-growth stress workload exceeded its original 120-second test budget while metadata/BIO progress continued; this PR does not claim to resolve that throughput limit. A separate 784 MiB ballast workload still reaches genuine OOM when its runtime grows beyond available reclaimable capacity. Short Agent smoke tests are not long-run stability or tail-latency evidence.

…ries

Preserve physical frame and fixed-layout allocation failures as typed errors
instead of collapsing them into ordinary ENOMEM. Compose the existing charge
and global recovery states in MM, with a fixed physical recovery deadline and
explicit task, IRQ and NOIO restrictions. Keep GlobalAlloc nonblocking and keep
VA exhaustion and unsupported physical layouts terminal.

Carry the error through cache births, CPU buffers, kernel stacks and fault
retry tokens. Preserve successful write prefixes and delegated completions;
retry only the retained synchronization range after payload publication. Let
loop backing failures escape NOIO without losing physical provenance.

Add pressure-driven file syscall regression coverage and extend allocation,
completion and rollback selftests. The same 1 GiB regression fails with ENOMEM
before this change and completes after it, including synchronization, thread
lifetimes and full data validation. Verify actual recovery from File writes
with hardware breakpoints without adding production diagnostics.

Validated with default and mm_selftest kernel builds, RISC-V check, formatting
and all-features clippy, focused guest memory/VFS/loop regressions, a low-water
SQLite integrity workload and OpenCode initial/resume smoke tests.

Signed-off-by: longjin <longjin@dragonos.org>
@github-actions github-actions Bot added the Bug fix A bug is fixed in this pull request label Nov 4, 2026
@fslongjin

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-04T02:48:11.271001Z e11dd1c Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: de49e7c8d9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Move the unchanged task/IRQ/preemption eligibility predicate to the always
available MM allocation module. Re-export it for existing vmalloc callers so
LoongArch recovery no longer depends on an architecture-gated VA backend.
Preserve the initialization short-circuit and the existing NOIO policy.

Correct the socket and ELF scratch birth fixtures to check a real single
Physical allocation refusal before delivering its terminal errno to the
consumer. Keep one-attempt, no-consumption, file-position and accounting
assertions; do not require the production recovery loop to make only one
attempt against an intentionally permanent refusal.

Reproduced the original LoongArch compile error and both failing retirement
checks before the fix. Validated LoongArch check and release build, default
and CI-feature x86 kernel builds, formatting/all-features clippy, and 45 guest
cases including the complete retirement report and related MM/VFS/loop suites.

Signed-off-by: longjin <longjin@dragonos.org>
@fslongjin

Copy link
Copy Markdown
Member Author

Fixed both CI regressions in e11dd1c.

  • LoongArch build: common physical-allocation recovery depended on vmalloc::task_context, although vmalloc is architecture-gated. The unchanged eligibility predicate now lives in the always-built MM allocation module, with a re-export for existing vmalloc callers. IRQ/preemption checks and NOIO restrictions are preserved.
  • PageRetirement selftest: the socket and ELF fixtures permanently refused physical backing while expecting a recovery-enabled operation to attempt allocation only once. They now verify a real single-attempt Physical refusal and then test the consumer's terminal-error handling. The one-attempt, datagram preservation, zero-length, ELF-position and accounting assertions remain intact; production recovery is unchanged.

Both failures were reproduced before editing. After the fix, LoongArch check and release build, RISC-V check, default x86 and CI-feature kernel builds, formatting/all-features clippy, and all 45 cases in nine targeted guest suites passed without skips. The complete PageRetirement report now passes, including both scratch checks. The original Integration Test also completed successfully.

@fslongjin

Copy link
Copy Markdown
Member Author

@codex review

@github-actions github-actions Bot added the test Unitest/User space test label Nov 4, 2026
@fslongjin
fslongjin merged commit 11aa98f into DragonOS-Community:master Nov 4, 2026
18 of 19 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e11dd1c13b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread kernel/src/mm/allocation.rs Resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug fix A bug is fixed in this pull request test Unitest/User space test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant