Repository navigation
accel/amdxdna: support graceful hw context destroy and restore on aie4 - #1635
Conversation
Will you be able to develope a shim_test for this too? |
There was a problem hiding this comment.
Pull request overview
This PR adds AIE4-specific support for preserving and restoring hardware context state across runtime auto-suspend/resume by using firmware “graceful destroy” restore IDs and persisting the firmware restore-pool across the suspend power boundary.
Changes:
- Implement graceful hwctx destroy on suspend, capturing per-context
restore_idand saving the firmware context restore pool into a driver buffer for resume. - Restore the saved restore pool on resume before recreating contexts, and fall back to fresh context creation if pool save/restore fails (clearing restore IDs and aborting parked jobs).
- Add a shim_test that holds an idle context across multiple runtime auto-suspend cycles and submits work after each resume to guard the restore path from regressions.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| test/shim_test/shim_test.cpp | Adds an AIE4 + native-driver device filter and registers the new suspend/restore regression test. |
| test/shim_test/io_test.cpp | Implements runtime PM polling via sysfs and adds TEST_ctx_restore_across_suspend. |
| drivers/accel/amdxdna/amdxdna_ctx.c | Resumes the device before hwctx erase/destroy so firmware restore data can be reclaimed when destroying a suspended context. |
| drivers/accel/amdxdna/aie4_pci.h | Adds restore_id to hwctx private state and adds a device-level saved restore-pool handle plus message prototypes. |
| drivers/accel/amdxdna/aie4_pci.c | Saves/clears/restores the firmware ctx restore pool across suspend/resume; switches suspend path to graceful destroy for non-clean shutdowns. |
| drivers/accel/amdxdna/aie4_msg_priv.h | Introduces new AIE4 mailbox opcodes and request/response structs for restore-pool get/set. |
| drivers/accel/amdxdna/aie4_message.c | Implements get-size/get/set message helpers for the context restore pool. |
| drivers/accel/amdxdna/aie4_ctx.c | Threads restore IDs through graceful destroy and into subsequent create; aborts parked jobs when graceful destroy returns restore_id=0. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
d240b03 to
401a100
Compare
Yes, a shim test is added in the PR. |
39213e1 to
d9adf1b
Compare
2fdea67 to
d45ffb9
Compare
d45ffb9 to
2fa30a0
Compare
1033f78 to
6283c7d
Compare
675f949 to
bd584a4
Compare
|
Retest this please |
e23643b to
87e4f90
Compare
xdavidz
left a comment
There was a problem hiding this comment.
The comments look good to me, thanks for handling this.
Please double check with @wendyliang25 if this handling look ok by her from TDR point of view.
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved critical and moderate findings affect parked-job recovery, DMA cache coherency, and suspended cleanup.
Get a fresh assessment by requesting another Copilot review.
Review details
Suppressed comments (1)
drivers/accel/amdxdna/aie4_pci.c:472
- After
GET_CTX_RESTORE_POOLfirmware writes this DMA buffer, but this path never invalidates the CPU cache before retaining it for the later SET. On the non-coherentdma_alloc_noncoherentpath, the resume can therefore hand firmware stale pool bytes; the existing telemetry read path flushes after a device write for the same reason. Flush the buffer after a successful get.
ret = aie4_msg_get_ctx_restore_pool(ndev, to_dma_addr(buf_hdl, 0),
to_buf_size(buf_hdl));
- Files reviewed: 8/8 changed files
- Comments generated: 3
- Review effort level: Lite
1629cef to
4b46684
Compare
INVALID. drm_clflush_virt_range() at line 469 already writes back and invalidates, and the CPU never touches the buffer between the GET and the SET. Its own cited precedent flushes after only because the CPU is about to read. |
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
User-mode abort handling and bidirectional DMA mapping must be corrected.
Get a fresh assessment by requesting another Copilot review.
Review effort: Balanced
Findings: 2
Open (2)
Resolved since last review (3)
4b46684 to
1629cef
Compare
Suspend destroys hw contexts non-gracefully, so firmware discards their state and resume rebuilds from scratch. Destroy gracefully instead and pass the returned restore_id to the next create. That state lives in a firmware restore pool that a firmware suspend wipes, so save it into a driver buffer and hand it back before the contexts are recreated. Reset and TDR stay non-graceful, and an FLR drops the ids and the pool. A graceful destroy that returns no id preserved nothing, so the suspend aborts that context's parked jobs once it is gone. Firmware reports that as NO_RESTORE. Add shim tests for a context held across auto-suspend, several held so the pool has more than one entry, and one destroyed while suspended. Signed-off-by: Soham Donwalkar <Soham.Donwalkar@amd.com>
1629cef to
ac4e73e
Compare
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
Firmware state persistence, runtime power management, and context teardown interactions require final hardware-backed human validation.
Review effort: Balanced
Findings: None



Suspend destroys hw contexts non-gracefully, so firmware discards their state and resume rebuilds from scratch. Firmware can instead park a context and return a restore_id for a later create to replay.
Destroy gracefully on suspend, keep the restore_id and pass it to the next create. The parked state lives in a firmware context restore pool that does not survive a firmware suspend, so copy it into a driver buffer once all contexts are destroyed and hand it back after the partition is recreated. Reset and TDR stay non-graceful, and an FLR drops both the ids and the pool. If firmware returns restore_id 0, or the pool cannot be saved, the ids are cleared and parked jobs aborted so resume builds fresh contexts.