Conversation
## Summary Nested background physics reset became enabled by default and opted the Galileo scene into a reset path that it does not use. The Galileo scene contains a dolly prim that identified as an articulation from its USD schema, but PhysX does not expose a corresponding tensor articulation. The user is informed of this via an error message that is not fatal. This PR disables `reset_nested_physics` to restore prior behaviour and prevent users from getting this error message. Signed-off-by: Peter Du <peterd@nvidia.com>
## Summary Record per-step trajectories in eval runs. This same branch was previously merged into `develop` in #1008. Now the target is merging it into `main`. --------- Signed-off-by: Mattia Rossi <mrossi@nvidia.com> Signed-off-by: Clemens Volk <cvolk@nvidia.com> Co-authored-by: Clemens Volk <cvolk@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
## Summary Add PhysX deformable pick-and-place support ## Detailed description - Introduces `DeformableObject` with backend-specific spawner configs, nodal reset events, and placement integration on top of the existing `ObjectBase`. `DeformableObject` is supported for placement using rest shape bbox from USD or from spawner size. - Extends `PickAndPlaceTask`, spatial predicates, and `ArenaWorld` so deformable pick objects use placement-only success (no contact sensor). - Adds the `droid_deformable_pick_and_place` environment with selectable deformable assets (cube, surface, teddy bear) and PhysX smoke/regression tests. Verified cube pick and place with teleop. Works with OpenPi but the picking is not always reliable. ## Command to run: python submodules/IsaacLab/scripts/environments/teleoperation/teleop_se3_agent.py --viz kit --task droid_deformable_pick_and_place --embodiment droid_differential_ik --external_callback isaaclab_arena.environments.isaaclab_interop.environment_registration_callback <img width="640" height="400" alt="Screenshot from 2026-09-08 00-23-56" src="https://github.com/user-attachments/assets/bc269448-6927-4e19-a9a9-502892c23383" /> ## Known issues to be addressed to follow up PRs: - Switching deformable objects and pnp to Newton (dependent on Newton-compatible Droid support) - Improve geometry-based object_supported_by predicate for deformable+rigid to use rigid vertices . This requires extension of ArenaWorld to hold cache of rooted object mesh/vertices. - Deformable material / simulation cfg optimization for the task is deferred until we support Newton. --------- Signed-off-by: Qian Lin <qianl@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
…ate (#1260) Signed-off-by: Xinjie Yao <xyao@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Signed-off-by: Xinjie Yao <xyao@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
…1266) Signed-off-by: Qian Lin <qianl@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Fix collision bounds for objects rotated by `RotateAroundSolution`. Add regression tests for marker-only yaw rotations and overlapping placements. ## Why this change? - `RotateAroundSolution` could rotate an object without rotating the bounding box used for collision checks. - This let overlapping objects pass placement validation. ## What changes? - Include the `RotateAroundSolution` rotation when computing collision bounds, so checks account for the object's rotated shape. - Add regression tests for rotated bounds and for an overlapping placement that previously passed validation. ## Validation - 170 focused placement tests passed. - Of the nine new regression cases, seven fail without the fix; all nine pass with it. - `pre-commit run --all-files` passed.  Initial placement: incorrect bounds allow overlap (left); corrected bounds with mesh collision checking avoid it (right). Mesh configuration is separate from this PR. --------- Signed-off-by: zhx06 <zihaox@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Signed-off-by: Xinjie Yao <xyao@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Signed-off-by: Qian Lin <qianl@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
…1276) ## Summary Add release and withdrawal predicates for insertion task ## Detailed description - Insertion tasks could require gripper release and gripper withdraw as success criteria - Add articulation state query into ArenaWorld - Include configurable release thresholds and TCP withdrawn distance --------- Signed-off-by: Xinjie Yao <xyao@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
## Why USD utilities were split between top-level `utils/usd_*.py` modules and the existing `utils/usd/` package. ## What - Consolidates four USD utility modules under `isaaclab_arena.utils.usd `and updates all package, test, and example imports. No intended behavior change. ## Before USD utilities used mixed locations such as: - utils.usd_articulation - utils.usd_helpers - utils.usd_pose_helpers - utils.usd_prim_tree ## After USD utilities share one consistent namespace: - utils.usd.articulation - utils.usd.helpers - utils.usd.pose - utils.usd.prim_tree Signed-off-by: Xinjie Yao <xyao@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Signed-off-by: Peter Du <peterd@nvidia.com> Signed-off-by: Qian Lin <qianl@nvidia.com> Signed-off-by: Alex Millane <amillane@nvidia.com> Co-authored-by: Peter Du <peterd@nvidia.com> Co-authored-by: Qian Lin <qianl@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Signed-off-by: Peter Du <peterd@nvidia.com> Signed-off-by: Qian Lin <qianl@nvidia.com> Signed-off-by: Alex Millane <amillane@nvidia.com> Co-authored-by: Peter Du <peterd@nvidia.com> Co-authored-by: Qian Lin <qianl@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Require a concise opening summary and replace rigid PR-description limits with plain-English writing guidance. Update the template, repository instructions, and commit-and-pr skill. ## Why this change? A PR description should explain the problem, the solution, and what changes for users without requiring readers to follow the earlier discussion. The current template limits the summary to 50 characters, while the repository guidance requires a fixed number of bullets. This encourages compressed lists of changes rather than a readable explanation of why they belong together. ## What changes? Require 1–2 concise opening sentences stating the changes before any heading. Use imperative verbs such as "Add", "Fix", or "Update" and omit introductory phrases such as "This PR adds". Follow that summary with prompts for motivation, the solution, an optional usage example, and validation. The template encourages short paragraphs, plain English, and enough context for a teammate who has not followed the discussion. Small changes can still have short descriptions, and authors should remove sections that do not apply. When an API or its usage changes, the template asks for a small before/after example of the same operation and an explanation of any behavior difference. Update `AGENTS.md` and the `commit-and-pr` skill to match. The skill reads the repository template instead of keeping its own copy of the format. ## Validation - All-file pre-commit checks and `git diff --check` pass. No runtime code changed, so simulation tests were not run. - The skill validator rejects the existing `disable-model-invocation` metadata field. That field is unchanged; no skill invocation settings were modified. --------- Signed-off-by: Clemens Volk <cvolk@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
The bigger idea is: progress and task success should come from the same definition and use the same state. Previously, progress tracking recorded milestones, while a separate success check decided when to finish. Those two could disagree—for example, placement could trigger success even though the tracked lift stage never happened. Now the responsibilities are clearer: 1. **The task declares what ends the episode.** A new `TaskTerminationCfg` puts success objectives, failure conditions, and the time limit together in `get_termination_cfg()`. 2. **The progress tracker remembers what has happened.** `ProgressTracker` knows which stage is active and which stages have finished. Its completion state is now the source of task success, not just a record for progress reporting. 3. **The builder connects this to Isaac Lab.** `ArenaEnvBuilder` translates the task's criteria into Isaac Lab termination terms. For tasks with success objectives, it automatically installs one shared `ProgressBasedSuccessTerm` term, which owns the tracker. This removes local `TerminationsCfg` classes and its separate `get_progress_objectives()` hook. The task's completion criteria now live in one place. During termination evaluation, the shared term advances the tracker. This prepares the next change: stateful predicates. For settle → lift → rest for 10 seconds → place, the tracker could maintain the rest counter while that stage is active. That counter could use the same episode-reset path. **Scope and merge dependency:** This MVP migrates the existing PickAndPlaceTask, OpenDoorTask, and NoTask. OpenDoor uses its existing sequence: move away from the reset position → open past the threshold. Other tasks, composite/sequential task support, and documentation updates are in #1256. Merging this PR alone breaks environment construction for unmigrated tasks, so both PRs should be ready before merging. Mimic is outside this work's scope. --------- Signed-off-by: Clemens Volk <cvolk@nvidia.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Signed-off-by: Xinjie Yao <xyao@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Signed-off-by: Xinjie Yao <xyao@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Adds ObjectDisappearVariation, a RunTimeVariationBase that teleports a distractor object to a far-away pose with a configurable probability. ## Detailed description - Distractor clutter was fixed for every episode, so a policy always saw the same set of non-task objects. This adds a per-object knob to thin it out at random. - Adds `ObjectDisappearVariation`, a run-time variation that teleports an object far out of the scene with a configurable probability, drawn per environment on every reset. Backed by a new reusable `BernoulliSampler`. - Attached automatically to every rigid object and disabled by default, mirroring `ObjectMassVariation`, so it is reachable as `<object>.disappear.enabled=true` from the CLI or an experiment config without editing the scene. - Teleporting happens in a reset event rather than a spawn pose: relation placement rewrites non-anchor poses on reset, and variation events are composed after it, so the teleport is what survives --------- Signed-off-by: viiik-inside <vramasamy@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Add runner-owned consecutive-step requirements in `tasks/predicates/temporal.py` and use them in `PickAndPlaceTask`. Support environment-initialized instantaneous checks without requiring manager inheritance. ## Why this change? Some tasks need a condition to remain true, not just become true briefly. For example, an object should stay correctly placed and stable for several steps before success. [#1255](#1255) gave task progress and success one update and reset lifecycle. This change uses that lifecycle for consecutive-step counters, without adding another manager term or reset callback. ## What changes? - `TrueForConsecutiveStepsCfg` declares the predicate and required number of control steps. It holds no runtime state. - `ProgressObjectiveRunner` creates an internal `_TrueForConsecutiveSteps` instance for each occurrence. Counting starts when that requirement becomes active; a false result clears the streak. Episode resets clear the selected environments' counters through `TaskSuccessTerm` and `ProgressTracker`. - A requirement can wrap a callable or an instantaneous `TerminationTermCfg`. Configured callable classes receive `(cfg, env)`; they do not need to inherit from `ManagerTermBase`. Scene references are resolved before construction. - `ProgressTracker` prevents repeated updates for the same step from advancing progress twice. Skipped steps interrupt a streak. Direct calls to `ProgressTracker.step()` with temporal requirements must supply per-environment step indices; `TaskSuccessTerm` already supplies them. - `PickAndPlaceTask` can require its complete placement condition to hold for multiple steps. `placement_consecutive_steps=1` remains the default. - `objects_below_velocity_thresholds()` exposes a rest check without pose recording or counters. The existing `objects_settled()` behavior is unchanged. - The existing `ProgressTracker.get_predicate()` accessor unwraps temporal requirements for passive diagnostic reads. The unused `termination_term_result()` adapter is removed. Requirements work in the middle or at the end of a sequence. To require overlapping conditions, combine their instantaneous results and wrap that combined predicate once. Composite tasks' final-condition checks continue updating the final requirement without counting twice. Follow up [#1305](#1305) migrates `ConsecutivePredicate`, `CompositePredicate` ## Usage example Here `settled`, `lifted`, and `placed_and_stable` are already-configured predicate callables. Their arguments are unchanged. Before: one successful placement check completes the sequence. ```python ProgressObjective( name="pick_and_place", predicate_sequence=[settled, lifted, placed_and_stable], ) ``` After: placement must hold for ten consecutive control steps. ```python from isaaclab_arena.tasks.predicates.temporal import TrueForConsecutiveStepsCfg ProgressObjective( name="pick_and_place", predicate_sequence=[ settled, lifted, TrueForConsecutiveStepsCfg( predicate=placed_and_stable, required_steps=10, ), ], ) ``` Success is reported on the tenth qualifying step, without an extra policy action. ## Validation - `pre-commit run --all-files`: passed. - 65 simulation-independent test bodies run directly in the development container: passed, covering the new temporal API and retained legacy initialization, reset, and composition behavior. These calls bypass the simulator wrappers; they are not a full pytest run. - Simulator-backed pytest remains blocked before test bodies by duplicate `--enable_cameras` registration in the local Isaac Lab checkout and Arena. Cleanup also reports missing `omni.timeline`. The local submodule change is excluded; end-to-end simulator validation remains outstanding. --------- Signed-off-by: Clemens Volk <cvolk@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Update the public agentic environment-generation endpoint to use `deepseek-ai/deepseek-v4.1-flash`. Refresh the model-selection guidance with the new benchmark snapshot. ## Why this change? The previous public default, `deepseek-ai/deepseek-v4-pro-0813`, is deprecated. DeepSeek V4.1 Flash passed the structural checks for all five documented example workflows. It's faster compared with Kimi K3 during inference. ## What changes? - Set `PUBLIC_ENDPOINT.model` to `deepseek-ai/deepseek-v4.1-flash`. - Replace the deprecated model and its statistics in the model-selection documentation. ## Validation - Passed tests - 15/15 passed on benchmark on documented prompts --------- Signed-off-by: Xinjie Yao <xyao@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Signed-off-by: Qian Lin <qianl@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Signed-off-by: Clemens Volk <cvolk@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Load saved object and robot root poses from companion JSONL files and restore complete layouts on reset. Part 2 of #1242. Independent of clutter generation in #1290. ## Why this change? Reuse recorded layouts without solving placement again. Keep pose data separate from the environment definition so one scene can replay many layouts. ## What changes? - Add `PlacementLayouts` to read and write the episode variations format. - Add `ArenaEnvBuilderCfg.placement_layouts_path`, set through `--placement_layouts`. The builder loads and owns the layouts for both YAML and Python environments. - Draw layouts from one shared queue in reset-request order. Wrap on exhaustion and leave other environments unchanged during partial resets. - Reuse the pooled placement pose writer, replace fixed-pose reset events for cached assets, and zero root velocities. - Validate pose data, asset coverage, and reset ownership. - Add a 10-record example in `test_data/placement_replay.jsonl`. Replay bypasses solving and does not repeat geometry, reachability, or settling checks. Recordings must match the scene and robot configuration. Robot replay restores root poses, not joint states. ## Usage example Solve placement: ```bash /isaac-sim/python.sh isaaclab_arena/scripts/environment_runner.py \ --env_spec scene.yaml ``` Replay saved placement: ```bash /isaac-sim/python.sh isaaclab_arena/scripts/environment_runner.py \ --env_spec scene.yaml --placement_layouts layouts.jsonl ``` For registered Python environments, put `--placement_layouts` before the environment subcommand. Python callers can use `ArenaEnvBuilderCfg(placement_layouts_path="layouts.jsonl")` or supply `PlacementLayouts` in memory. Paths are relative to the working directory. No replay field is added to the environment YAML. ## Validation - 32 focused tests passed, covering YAML and Python construction, queue wrapping, partial resets, robot roots, and 200 physics steps after replay. - Collection-import isolation passed. - Pre-commit passed on all files and the JSONL fixture. --------- Signed-off-by: zhx06 <zihaox@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Signed-off-by: Xinjie Yao <xyao@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Signed-off-by: Xinjie Yao <xyao@nvidia.com> Co-authored-by: Lionel Gulich <lgulich@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
…1284) Signed-off-by: Xinjie Yao <xyao@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Signed-off-by: Qian Lin <qianl@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Re-organize the Arena images builds using docker build stages and dedicated helper scripts for installing. User-facing docker scripts remain the same as before ## Why this change? * Makes the Dockerfile easier to read, maintain and extend * Avoid pip install download on source code changes (all `.toml` deps are now installed in the arena-deps stage) * Previous Dockerfile installed Arena dependencies and compiled cuRobo **after** copying Arena source, triggering lengthy rebuilds. The perceived speedups are moderate for dev not using cuRobo. Main advantage is the structured docker build with existing patterns to follow for future modifications of the dockerfile. ## What changes? - Install dependencies before copying Arena source. Read project dependencies and the full `dev` extra from `pyproject.toml`, and move installation commands into focused setup scripts. - Build the cuRobo wheel in a separate stage based on Isaac Lab. Install it through a read-only mount so the CUDA build toolchain and wheel archive stay out of the final images. - Add `build_docker.sh` for builds without launching a container. Share it between the launcher and NGC publisher. Preserve the default developer image and `-c` shortcut; replace `INSTALL_CUROBO` with `dev` and `dev-curobo` targets. - Ignore pytest caches and Emacs temporary files. Create the runtime user's home directory and remove the unused duplicate pytest alias. CI still tests the published images. Building branch images before tests and adding remote-cache integration remain separate work. ## Usage example Replace the old feature argument in direct Docker builds. Before: ```bash docker build -f docker/Dockerfile.isaaclab_arena \ --build-arg INSTALL_CUROBO=true -t isaaclab_arena:curobo . ``` After: ```bash docker build -f docker/Dockerfile.isaaclab_arena \ --target dev-curobo -t isaaclab_arena:curobo . ``` The existing `./docker/run_docker.sh -c` and `./docker/push_to_ngc.sh -c -p` commands remain supported and select the cuRobo developer image. Without `-c`, both scripts default to the developer image without cuRobo. --------- Signed-off-by: David Tingdahl <dtingdahl@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Signed-off-by: Peter Du <peterd@nvidia.com>
Signed-off-by: viiik-inside <vramasamy@nvidia.com> Signed-off-by: Xinjie Yao <xyao@nvidia.com> Signed-off-by: Qian Lin <qianl@nvidia.com> Co-authored-by: Xinjie Yao <xyao@nvidia.com> Co-authored-by: Qian Lin <qianl@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Revert CI images back to latest tag now that the lab 3.0 commit update PR has been merged. Signed-off-by: Peter Du <peterd@nvidia.com>
Signed-off-by: Clemens Volk <cvolk@nvidia.com> Co-authored-by: arena-review-bot[bot] <290456231+arena-review-bot[bot]@users.noreply.github.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Signed-off-by: zhx06 <zihaox@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
**What**: Build fresh Arena images in premerge instead of postmerge. Remote caching is used to speedup the build: with a "warm" cache, the build time is similar to the current image pull time (~5 min). Remote cache is updated post-merge. **Why** Avoid manual "docker dance" every time a dependency is updated. **Details** - Reusable CI step: `build_and_push_image_steps` that builds and uploads an image while reading from the remote cache. - Report per-step cache hits and elapsed times, including cached-layer downloads. Save raw BuildKit logs as artifacts. **Validation** An [isolated cold-cache comparison](https://github.com/isaac-sim/IsaacLab-Arena/actions/runs/35326915903) prepared a locally runnable `dev` image from the NGC cache in 4m59s, versus 4m45s to pull the published main image. Both started with empty Docker storage; all 15 build steps were cached. This is one paired sample, separate from this PR's build/push/test job arrangement. Experiment-only tooling is excluded from the PR. Effect of main-only cache refresh will be inspected once this PR is merged. --------- Signed-off-by: David Tingdahl <dtingdahl@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Signed-off-by: Clemens Volk <cvolk@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Add `ClutterOn` to generate collision-checked release poses through `ObjectPlacer`. Separate candidate generation and validation from placement orchestration. ## Why this change? Clutter needs release poses above a support, with space between objects before they drop. A relation keeps this intent in the environment definition. This is the release-pose portion of [#1290](#1290). ## What changes? - Add clutter initialization, loss and validation. Release poses must clear their support; ordinary `On` contact tolerance does not apply. - Keep each layout's identity, geometry, loss and validation in `PlacementCandidate`. `PlacementCandidateBatch` selects and stacks complete layouts. - Extract `PlacementCandidateGenerator` and `PlacementValidationRunner`. Generation returns oriented bounds and initial clutter clearance before solving. - Account for fixed support rotations and preserve base tilt when sampling world-Z yaw. These shared geometry corrections can change ordinary layouts too. - Recompute ranking losses and printed final loss at the returned positions. Loss history retains the values measured before optimizer steps. Custom validators now receive `PlacementCandidateBatch` through `validate_batch(batch, collision_objects)` and return one verdict per candidate. The in-tree validators and cuRobo adapter use this interface. Supports must be fixed `IsAnchor` assets with horizontal quarter-turn rotations. Tilted clutter requires BBOX collision; unsupported MESH configurations fail explicitly. This does not add full-rotation MESH support for ordinary relations. This change produces release poses without running physics settling. Direct callers should check `PlacementResult.success`; pooled placement can disable failed-layout fallbacks with `allow_best_loss_fallbacks=False`. ## Usage example ```yaml relations: - kind: is_anchor subject: table - kind: clutter_on subject: cube reference: table params: spread: 0.2 random_yaw: true ``` A valid release places the cube above the table. It drops when physics starts. ## Validation Tested local revision `128780678e` plus the documentation and import cleanup: - 220 focused tests passed: clutter, candidate initialization, validation, reproducibility, mesh collision, facing, pose/reset transforms and import hygiene. - 43 live integration tests passed: transformed USD references, background collisions and cached replay/reset. - Pre-commit passed on the three changed files; `git diff --check` passed. --------- Signed-off-by: zhx06 <zihaox@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
#1316) Signed-off-by: Clemens Volk <cvolk@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
## What: * Start Kit **before** the interactive runner registers environments, so USD imports use Isaac-Sim's `usd-core` instead of the IsaacLab one. * Preserve the original CLI arguments across startup and exercise the runner's real startup path in the subprocess regression test. ## Why: Runner crashed on USD loading due to conflicting versions of `usd-core` between Isaac-Sim and Isaac-Lab. Other runners seems to already be creating SimulationApp Kit before environment registration and are thus not suffering from this issue. Issue was seemingly introduced in #1270 Signed-off-by: David Tingdahl <dtingdahl@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Add offline collection of settled `ClutterOn` layouts through `collect_settled_placements()`. Reuse the recording pipeline and shared validators, with clutter-specific preparation and support-containment checks. ## Why this change? [Clutter placement](#1318) produces collision-checked release poses above a support. Physics must drop the objects before those poses describe a settled layout. Extend the collector from [placement recording](#1308) so ordinary placements and clutter use the same reset, simulation and acceptance path. ## What changes? - Detect `ClutterOn` in `collect_settled_placements()` and run `prepare_clutter_settling()` before collection. Check support geometry, gravity and object mobility; reject object sets. - Choose ordinary or clutter validator defaults when `params` is omitted. Preserve explicit validator settings unchanged. - Make the shared `PoseShiftValidator` exclude intentional clutter drops. Root velocity checks still apply to clutter; displacement limits still apply to other objects. - Add `SupportContainmentValidator`, which checks captured bounds and poses against the support footprint and minimum resting height. Containers can specify a local height below their rim. Accepted layouts retain validator settings and results. - Separate `discover_passive_assets()`, which returns original scene assets, from `get_placement_collision_objects()`, which handles collision modes, aggregation and anchored-reference exclusions. - Keep offline preparation and validation under `offline_placement/`, with no imports from the online placement path into that package. Each batch uses the normal reset to select pooled poses. Collection returns `SettledPlacementResult` with accepted root poses, source indices and rejection reasons. Partial or zero acceptance is a normal result; candidates are not automatically retried. The caller owns the environment, which remains at its final state on success or failure. Supports must be fixed anchors with upright quarter-turn orientations. By default, containment requires a verified flat rectangular top. Bins and bowls can configure `support_containment.minimum_resting_heights_m` by support scene key, in scaled support-local metres. Release placement remains above the full support bounds. This checks the footprint and minimum height, not exact containment against curved walls. An `ObjectReference` to a flat tabletop or tray floor is also supported; its transform operations must be authored before construction. Other placement relations must already be resolved to fixed anchors. `RequiresReachability` is rejected only on `ClutterOn` objects; non-clutter fixed targets may retain their solver reachability requirements. Object sets must be resolved to individual objects before collection. Robot joints are not immobilized or recorded. This change returns poses in memory. It does not add a clutter command-line tool or file writer. ## Usage example After starting `SimulationApp` and constructing `arena_env` with `ClutterOn` relations: ```python from isaaclab_arena.environments.arena_env_builder import ArenaEnvBuilder from isaaclab_arena.environments.arena_env_builder_cfg import ArenaEnvBuilderCfg from isaaclab_arena.offline_placement.clutter_validators import default_clutter_validators from isaaclab_arena.offline_placement.settled_placement import collect_settled_placements from isaaclab_arena.offline_placement.settled_placement_params import SettledPlacementParams params = SettledPlacementParams( num_steps=480, validators=default_clutter_validators(), ) env = ArenaEnvBuilder(arena_env, ArenaEnvBuilderCfg(num_envs=4)).make_registered() try: result = collect_settled_placements( env, num_batches=2, params=params, scene_assets=arena_env.get_placement_assets(), render=True, log_progress=True, ) print(result.accepted_indices) print(result.rejections) finally: env.close() ``` Do not reset before collection. Two batches across four environments sample eight candidates. `num_steps` counts environment steps per batch, each containing the configured physics substeps. The example sets a longer drop window than the shared default. ## Validation - Passed 19 focused tests covering clutter collection on PhysX and Newton, geometry, recording and import hygiene. - Verified settling into the existing YCB bowl asset in PhysX, with accepted poses stable for another 200 physics steps. Covered configured heights, raised supports, below-floor rejection and footprint rejection. - Passed pre-commit, whitespace checks and the documentation build. --------- Signed-off-by: zhx06 <zihaox@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
**What** Simplify and update docs for improved legibility and clarity. Placement recorder: * Keep only information relevant to record & replay. * Move acceptance checks and rejection guidance to Placement Validation Installation * clarify cuRobo availability in the installation table. Before the table showed identical features for uv vs docker --------- Signed-off-by: David Tingdahl <dtingdahl@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Fix GR00T reference links, clarify placement-recording setup, and update the RL training-output example. Address the three documentation observations in [NVBug 6858297](https://nvbugspro.nvidia.com/bug/6858297). ## What changes? - Link introductory GR00T references to NVIDIA’s N1.6 overview. Point setup and fine-tuning links directly to rendered documentation sections at Arena’s pinned GR00T revision. - Add separate native uv and Docker setup tabs before the shared recording and replay commands. - Replace the outdated RL console example with the reported output. Explain that it shows a 20-iteration run and that timings and metrics vary. ## Validation - Fresh strict Sphinx build passed with zero warnings. - Pre-commit and whitespace checks passed. - All nine changed pages, runtime tabs, and external link destinations were checked. - Headless clamp recording and replay passed in both existing native uv and Docker installations. Each saved 15 of 16 layouts and completed three replay episodes, producing JSON results and HTML reports. - RL output labels and formatting match the pinned RSL-RL 5.4.1 logger. Training and viewer mode were not rerun. Concurrent replay checks exposed a separate shared HDF5 metrics-file collision. Running the workflows separately passed; this documentation change does not modify that runtime behavior. --------- Signed-off-by: zhx06 <zihaox@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Add offline physics settling for `ClutterOn` scenes to the shared placement recorder and save accepted layouts as replayable JSONL. Add maintained table and container examples with focused record/replay documentation and shared validation guidance. ## Why this change? `ClutterOn` produces collision-checked release poses above a support, but those poses are not the final states needed for evaluation. Clutter objects must fall, settle, pass post-physics checks, and then replay from the measured root poses without rerunning placement or settling. This builds on the shared settling and validation workflow from [C2 (#1319)](#1319) and uses the same recording entry point for ordinary and clutter scenes. ## What changes? - Extend `record_placement_layouts.py` to iterate pooled reset-and-settle batches until `min_layouts` is reached or `max_batches` is exhausted. - Use `SettledPlacementParams` for recording. Merge clutter defaults for `ClutterOn` scenes, including velocity, non-clutter pose shift, articulation-link shift, and support-containment checks. - Record rigid-object and articulation root poses, pre/post-physics validation reports, timestep metadata, and embodiment root exclusions in episode JSONL. - Write accepted layouts even when the batch budget ends below `min_layouts`; log the shortfall and exit cleanly. Leave the destination unwritten only when no layouts pass, and never overwrite an existing file. - Support flat surfaces and containers through per-support minimum resting heights. Add a kinematic YCB bowl variant for fixed container settling. - Resolve object-reference paths relative to the parent asset's actual runtime prim path, including custom parent paths. - Add maintained no-task environments for three tools on a table and three cubes in a bowl, with release/settled captures and separate recording/replay instructions. - Move clutter acceptance checks, containment requirements, and rejection guidance into Placement Validation. `PlacementRecordingCfg.num_layouts` is renamed to `min_layouts`. The removed `PlacementRecordingParams` type is replaced by `SettledPlacementParams`. Object sets must be resolved to concrete assets before recording. Recordings store root poses, not robot joint or controller state. `ClutterOn` objects cannot require reachability because settling changes the solver-validated poses. ## Usage example Record tool clutter headlessly: ```bash python isaaclab_arena/scripts/record_placement_layouts.py \ env_spec=isaaclab_arena_environments/clutter/franka_three_hammers_and_clamp_no_task.yaml \ output=outputs/clutter/tools_on_table.jsonl \ num_envs=4 min_layouts=10 layouts_per_env=4 max_batches=15 seed=42 \ settle.num_steps=480 \ +settle.validators.support_containment.minimum_resting_heights_m.office_table_background=0.5306 \ render=false --device cpu --viz none ``` Replay the accepted layouts interactively: ```bash python isaaclab_arena/scripts/environment_runner.py \ --env_spec isaaclab_arena_environments/clutter/franka_three_hammers_and_clamp_no_task.yaml \ --placement_layouts outputs/clutter/tools_on_table.jsonl \ --num_envs 1 --device cpu --viz kit ``` The clutter workflow guide includes the equivalent cube-in-bowl commands and explains the bowl-local minimum resting height. ## Validation - Focused partial-output API tests passed: **2 passed**. - The subprocess CLI test confirmed partial recordings are written and the app exits cleanly: **1 passed**. - The documented cube-in-bowl recording completed end to end. It rejected the first moving layout, accepted the second, and wrote one replayable layout. - Pre-commit hooks and `git diff --check` passed for the latest code, YAML, and RST changes. - The earlier branch validation ran **111 targeted tests**, including recording, JSONL metadata, rejection handling, replay, and PhysX/Newton CLI coverage. The full targeted set was not rerun after the latest workflow and documentation updates. --------- Signed-off-by: zhx06 <zihaox@nvidia.com> Signed-off-by: Qian Lin <qianl@nvidia.com> Co-authored-by: Qian Lin <qianl@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Fix nested background physics reset so kinematic rigid bodies no longer receive velocity writes that PhysX rejects. Share the dynamic-rigid USD classification helper with offline clutter support checks. ## Why this change? `ResetBackgroundPhysics` zeroes linear and angular velocity on nested background rigid bodies after each reset. PhysX logs an error when that write targets a kinematic body (`Body must be non-kinematic`). Kinematic backgrounds such as the office table can trigger the failure even when the scene behaves correctly otherwise. This addresses https://nvbugspro.nvidia.com/bug/6853023. ## What changes? - Add a USD helper that identifies enabled, non-kinematic rigid bodies. - Gate each `_RigidReset` velocity write using the corresponding live rigid prim. - Reuse `is_enabled_dynamic_rigid_body` in offline clutter geometry preflight (`prim_geometry_is_fixed` and `spawned_rigid_body_is_dynamic`). - Cover helper classification, skipped velocity writes, and mixed kinematic-parent/dynamic-child reset behavior. - Keep the existing maple-table test exercising dynamic rigid pose and velocity restoration. ## Validation Passing checks: - `pre-commit run --all-files` - `/isaac-sim/python.sh -m pytest -sv isaaclab_arena/tests/test_background_physics_reset.py isaaclab_arena/tests/test_rigid_bodies.py` — 10 passed - Standalone `test_rigid_reset_skips_velocity_write_when_disabled` — 1 passed - `test_rl_train_and_eval_lift_object` local rerun — 1 passed - Manually ran the following command before and after the fix and verified that the PhysX kinematic velocity errors disappeared: ```bash /isaac-sim/python.sh isaaclab_arena/scripts/record_placement_layouts.py \ env_spec=isaaclab_arena/tests/test_data/placement_replay.yaml \ output=outputs/clutter/repro_physx.jsonl \ num_envs=4 env_spacing=2 layouts_per_env=4 seed=42 \ settle.num_steps=120 render=true --device=cpu --viz kit ``` The full pytest phases were not run locally. The initial docs CI failure was an external `docs.python.org` inventory HTTP 503; the initial RSL-RL subprocess crash passed on a clean local rerun. --------- Signed-off-by: Qian Lin <qianl@nvidia.com> Signed-off-by: $(git config user.name 2>/dev/null || echo '') <$(git config user.email 2>/dev/null || echo '')> Signed-off-by: Peter Du <peterd@nvidia.com>
Document the Linux GLX workaround for commands that launch the interactive Newton viewer. ## Why this change? The Newton viewer can fail during PyOpenGL initialization because its active pyglet OpenGL context is not detected. Setting `PYOPENGL_PLATFORM=glx` before Python starts avoids the failure while the upstream issue is investigated in [NVBug 6853170](https://nvbugspro.nvidia.com/bug/6853170). ## What changes? Prefix the documented Newton viewer commands with `PYOPENGL_PLATFORM=glx`. Add the same guidance where the training workflow recommends enabling Newton visualization. This documentation-only workaround does not change headless or EGL runtime behavior. ## Validation - `pre-commit run --all-files` - Confirmed every documented `--viz newton` command includes the workaround, and the training guidance describes it. --------- Signed-off-by: Qian Lin <qianl@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Extract the relation solver's initialization into a new `isaaclab_arena/relations/initializers/` and add a `PlacementInitializerBase` interface. ## Why this change? - Initialization logic lived inline in `ObjectPlacer` - This makes way for new initialization methods. - Slims down the already-very-large `object_placer.py` file. ## What changes? - New package `isaaclab_arena/relations/initializers/`: - `PlacementInitializerBase` (one abstract method, `generate_initial_positions`) - `AnchorInitializer`, the existing behaviour. - `PlacementCandidateGenerator.generate_positions` delegates to the initializer, removing ~140 lines from `placement_candidate_generator.py` (`object_placer.py` is untouched — main extracted that code into the generator while this was in review) - `ClutterOn`'s release region moves into the shared sampling helper, so both initializers keep it - Small behaviour change to `AnchorInitializer`: walks up the whole `On` chain to find the nearest anchor, instead of resolving a single level. This is effectively a fix. There's no downside of doing this. ## Usage example Before — seeding was fixed ```python params = ObjectPlacerParams() ObjectPlacer(params=params).place(objects, num_envs=16) ``` After — seeding is a parameter: ```python params = ObjectPlacerParams(initializer_type=InitializerType.ANCHOR) ObjectPlacer(params=params).place(objects, num_envs=16) ``` ## Validation Placement success rate versus `main` Every environment in the robolab and kitchen_bench suites, solved 50 times per arm with the same placement seed and the same env spec loaded once, so seeding is the only variable. `main` runs its original inline seeding; the branch runs `AnchorInitializer`. | suite | environments scored | `main` | `AnchorInitializer` | mismatches | |---|---|---|---|---| | robolab | 38 | 93.4% | 94.4% | 4 | | kitchen_bench | 19 | 98.3% | 98.3% | 1 | Slight difference due to some numeric changes in `dtype` handling due to a cleanup, but no regressions. --------- Signed-off-by: alex <amillane@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Restore Arena's progress-tracking lifecycle in Mimic environments. Support class-based success terms across parallel dataset generation and demonstration annotation. ## Why this change? Isaac Lab Mimic extracts the configured success term before environment creation and evaluates it directly. After Arena unified task success under `TaskSuccessTerm`, Mimic attempted to call that `ManagerTermBase` class as a plain function. Once instantiated correctly, Arena's embodiment-specific Mimic environments still lacked `ArenaWorld`, progress tracking, and initial-rest-pose state. The documented GR1 workflows exposed two additional lifecycle problems: - Ten parallel generation workers read the shared success term during the same control step. The second read tried to advance `ProgressTracker` again at an unchanged step index. - Annotation initialized neither the class-based term nor its state. It also evaluated success only after replay, so progress-based predicate sequences never observed the intermediate control steps. Tracked by [NVBug 6850326](https://nvbugspro.nvidia.com/bug/6850326). ## What changes? - Add `IsaacLabArenaManagerBasedRLMimicEnv`, which combines Arena's environment lifecycle with Isaac Lab's Mimic interface. - Use the Arena-aware Mimic base for G1, Franka, and GR1T2 embodiments. - Make repeated `TaskSuccessTerm` reads at the same per-environment step idempotent while retaining strict progression checks for new steps. - Update the Isaac Lab submodule through [28a386f06](isaac-sim/IsaacLab@28a386f): - Instantiate and reset class-based success terms during generation. - Initialize and reset them during annotation, and evaluate them after every replayed action. ## Validation - Ran the documented GR1 Open Microwave Mimic generator with ten environments and one requested trial: ```bash /isaac-sim/python.sh submodules/IsaacLab/scripts/imitation_learning/isaaclab_mimic/generate_dataset.py \ --device cpu \ --generation_num_trials 1 \ --num_envs 10 \ --input_file /datasets/mimic-manager-success/arena_gr1_manipulation_dataset_annotated.hdf5 \ --output_file /datasets/mimic-manager-success/static_generated_smoke.hdf5 \ --enable_cameras \ --mimic \ --external_callback isaaclab_arena.environments.isaaclab_interop.environment_registration_callback \ --task gr1_open_microwave ``` Result: exited successfully with `2/2 (100.0%)` successful demonstrations. Multiple workers completed before the one-trial stop condition was observed. - Replayed the first episode from the documented sequential GR1 recorded dataset through the annotation lifecycle on CPU. The smoke script initialized and reset the extracted success term and evaluated it after each action. Result: `ANNOTATION_SUCCESS_RESULT=True`. - Ran the Arena task-success regression file: ```bash /isaac-sim/python.sh -m pytest -q isaaclab_arena/tests/test_task_success_from_progress.py ``` Result: `15 passed`. - Ran the Isaac Lab success-term tests: ```bash /isaac-sim/python.sh -m pytest -q source/isaaclab_mimic/test/test_success_term.py ``` Result: `3 passed`. - Ran `pre-commit run --all-files`: all hooks passed. --------- Signed-off-by: Qian Lin <qianl@nvidia.com> Signed-off-by: Peter Du <peterd@nvidia.com>
Reset Isaac Lab's thread-local stage as well as the stage attached to Kit. USD-only authoring can leave a stage active without a SimulationContext; resetting only Kit lets the next environment inherit old geometry. Use the public stage utilities and cover teardown without a simulation context in a regression test. This also fixes the order-dependent failure of the settling test following the DROID USD authoring test. Signed-off-by: Peter Du <peterd@nvidia.com>
🤖 Isaac Lab-Arena Review BotSummaryThis PR replays 46 commits merged to
No non-CAP code from the original PRs is missing. The new teardown change uses Findings🔵 PR description. The body is still the empty template. For a 47-commit restore, could it list the PRs left out (#1241, #1257, #1263, #1267, #1270, #1274, #1281, #1283, #1295, #1296, #1299, #1315, #1324, #1325, #1328, #1331, #1334) and the 8 commits without PR numbers that rework CAP PRs into core-only changes (e.g. #1261 → "Add composite predicates…", #1262 → "Add YAML environment configuration overrides", #1310 → "Add decorator registration…")? Reviewers could then trace each commit back to its original review. 🔵 Merge method. Could this be merged with rebase or a merge commit rather than squash? A squash would fold 47 commits from different authors into one, and per-PR history, 🟡 Infra. VerdictShip it (once CI is green and the description is filled in) |
|
| nested_target = concrete_type if construct_structured else current_value | ||
| _materialize_targets(nested_target, payload, path=path, construct_structured=construct_structured, strict=strict) |
There was a problem hiding this comment.
Extending an existing CouplerProxyCfg.entries list with a plain mapping passes current_value=None for the new entry. This code then calls _materialize_targets(None, ...), which raises TypeError at dataclasses.fields(NoneType) and stops environment setup. Construct missing entries from concrete_type instead of trying to inspect a missing instance.
| if isinstance(self.initial_pose, PosePerEnv): | ||
| return EventTermCfg( | ||
| func=set_deformable_object_pose_per_env, | ||
| mode="reset", | ||
| params={"asset_cfg": SceneEntityCfg(self.name), "pose_list": self.initial_pose.poses}, |
There was a problem hiding this comment.
set_initial_velocity is ignored when a deformable uses PosePerEnv. This event passes only the poses, and set_deformable_object_pose_per_env calls the nodal helper without a velocity. The helper writes zero velocity on every reset, even after the caller configured nonzero motion. Pass initial_velocity through this reset path, or reject that combination explicitly.
| def _progress_criteria(record: dict[str, Any]) -> dict[str, dict[str, Any]]: | ||
| criteria_by_name = _progress(record).get("criteria_by_name") | ||
| return criteria_by_name if isinstance(criteria_by_name, dict) else {} |
There was a problem hiding this comment.
Previously saved episode files contain progress.objectives and events with objective, but the new readers accept only criteria_by_name and criteria_name. The file loader still accepts those older files unchanged, so their task details and funnels disappear without a warning. Normalize both formats when reading, while keeping the new names for newly written records.
Summary
Short description of the change (max 50 chars)
Detailed description