Repository navigation
Enhance image management with DeleteImage API and improved resilience - #102
Merged
Merged
Conversation
… preheat all platforms by default Image deletion is now delegated to dfdaemon's DeleteImage API on every seed peer, keyed by the manifest URL (with retries), instead of being resolved and removed task-by-task on the client side. Preheat tasks are tagged with the image reference and platform so later deletions can match them, and image preheat covers all manifest platforms unless one is specified. Examples drop the obsolete scope parameter and print per-peer deleted/retained results. E2E tests gain multi-platform and digest-reference cases for preheat and delete, platform-aware SDK/driver helpers (Go and Rust), inline blob fixtures, verification of deleted vs retained task IDs per seed peer, and bumped dragonfly dependencies and chart images. Signed-off-by: Gaius <gaius.qi@gmail.com>
…ate platforms Make the Go and Rust client-request libraries resilient and consistent: - DeleteImage no longer fails the whole operation when a single seed peer errors; failed peers are skipped and remaining peers are still processed. - Seed peer requests in DeleteImage are now bounded by a concurrency limit instead of fanning out to all peers at once. - PreheatImage deduplicates manifests that resolve to the same platform so identical layers are not preheated repeatedly. - StatImage validates the requested platform up front and returns a clear error for unsupported or malformed values. Tests and READMEs are updated to cover the new behavior. Signed-off-by: Gaius <gaius.qi@gmail.com>
… instead of the scheduler Resolve image manifests in stat_image and ask seed peers directly for layer stat results, removing the scheduler round-trip. Both the Go and Rust client libraries are updated along with their READMEs. Add a stat-image command to the Go and Rust e2e drivers and an e2e test that verifies cached image layers are reported correctly. Signed-off-by: Gaius <gaius.qi@gmail.com>
gaius-qi
requested review from
BruceAko,
Fricounet,
anjia0532 and
hhhhsdxxxx
November 11, 2026 15:54
gaius-qi
enabled auto-merge (squash)
November 11, 2026 15:54
Drop the "since 1.11.0" and "since 1.13.0" breaking-change paragraphs and the behavior details about seed peers recording tasks per image/platform and omitting failed seed peers from the delete response in the Go and Rust client READMEs, and remove the multi-platform preheat, delete and stat description from the e2e README. In the e2e SDK helper, make `run` take the driver command explicitly and append the `--scheduler` endpoint itself, so each wrapper no longer repeats the command name and scheduler flag; error and log messages use the command directly instead of the first argument. Signed-off-by: Gaius <gaius.qi@gmail.com>
…config Add `backend.enableCacheTemporaryRedirect: false` to the seed client section of the test chart configuration so the test deployment does not cache HTTP temporary redirect responses. Signed-off-by: Gaius <gaius.qi@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Makes the image APIs of
client-request(Go and Rust) image-aware.PreheatImage,StatImageandDeleteImagenow all talk to the seed peers directly; the scheduler is only used to list the seed peers.sequenceDiagram participant SDK participant Registry participant SeedPeers as Seed peers (dfdaemon) Note over SDK,SeedPeers: PreheatImage SDK->>Registry: resolve manifests (all platforms, or one) SDK->>SeedPeers: DownloadTask(url, image{reference, platform}) Note over SDK,SeedPeers: StatImage SDK->>Registry: resolve manifests (all platforms, or one) SDK->>SeedPeers: StatLocalTask(task id) for every layer Note over SDK,SeedPeers: DeleteImage SDK->>SeedPeers: DeleteImage(manifest url, platform) SeedPeers-->>SDK: deleted / retained task idsPreheatImageStatImageStatImagegRPC, which needs the scheduler job service (redis).StatLocalTask.DeleteImageDeleteImageon every healthy seed peer, without resolving the image. A task still referenced by another image or platform is retained. Returns the deleted and retained task ids of each seed peer.API changes
*string/Option<String>), unset means all the platforms, and it is validated:containerd/platformsin Go, theos/archform in Rust.StatImageandDeleteImagefan out to the seed peers with a concurrency limit. A seed peer that fails is left out of the response instead of failing the request.Notes
DeleteImagedeletes nothing for it.StatImagenow needs the registry to be reachable from the SDK, asPreheatImagedoes.linux/armselectsarm/v7only) and normalizes aliases such aslinux/aarch64; Rust selects every index entry with thatos/archand sends the platform as given.d7y.io/api/v2to v2.3.10 andd7y.io/dragonfly/v2to v2.5.4-rc.0,dragonfly-apito 2.3.10, and the crate version to 1.13.0. The e2e cluster uses scheduler v2.5.4-rc.0 and client v1.5.12.Tests
PreheatImage,StatImageandDeleteImageagainst an in-test TLS registry and mock seed peers (targets andDownload.imageper platform, cached layers per seed peer, a failed seed peer left out, retries, invalid arguments).delete_imageand for the invalid arguments of the three APIs. The success paths ofpreheat_imageandstat_imageneed a registry and are covered by the e2e tests.preheat-image --platform,stat-imageanddelete-image --platform. The suite grows from 31 to 53 specs, run for both SDKs: preheat all platforms, one platform and by digest; stat by platform and after delete; delete by platform, platforms preheated one by one, a platform or an image that is not preheated, a tag and a digest reference sharing a platform, two tags sharing one blob, and an image preheated by one SDK and deleted by the other.Unit tests pass locally (
go test -race,cargo test --all-features, clippy with and without thepreheatfeature). The e2e specs run in CI.Related Issue
Motivation and Context
Deleting an image by resolving its manifest removes blobs that other images still share, and fails when the registry is unreachable or the tag has moved. dfdaemon v1.5.12 records the image and the platform every task is downloaded for, so the SDK passes them at preheat and lets dfdaemon decide what to delete.
StatImagewent through the scheduler job service, which only exists when redis is deployed. Asking the seed peers directly makes it work likePreheatImageandDeleteImagein a scheduler-only deployment.Screenshots (if appropriate)
N/A
🤖 Generated with Claude Code