Skip to content

Migrate Franka tasks to the shared asset and add minimal presets - #8170

Merged
kellyguo11 merged 32 commits into
isaac-sim:developfrom
maxkra15:maximiliank/franka-reach-drawer-upstream
Oct 2, 2026
Merged

kellyguo11 merged 32 commits into
isaac-sim:developfrom
maxkra15:maximiliank/franka-reach-drawer-upstream

Conversation

@maxkra15

@maxkra15 maxkra15 commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Description

Migrate maintained Franka tasks, contributed configs, benchmarks, and controller examples to the shared FRANKA_PANDA_CFG. Standard tasks use full primitive arm/gripper collisions with explicit Physics=physx or Physics=mujoco payloads.

Add FRANKA_MINIMAL_CFG and expose its gripper-only collision scope through preset() in the existing Franka Reach and rigid Lift configs. Select it with presets=minimal, including Reach OSC, while retaining task-local controller and actuator tuning. Backend selection stays independent of collision scope. No additional task ID or environment-page section is added.

Preserve task-local actuator tuning, correct absolute Reach action mapping and OSC actuator handling, and support differential-IK action offsets in the zero agent. Reset clearance ignores disabled colliders, and cuRobo excludes robot geometry from world obstacles even with custom ignore lists. Legacy compatibility configs remain available; maintained tasks no longer use them.

Lift reset/curriculum and continuous Reach tracking improvements are isolated in dependent #8171. No USD packages, checkpoints, or new dependencies are added to this source PR.

Breaking: migrated tasks change robot dynamics and collision scope. Existing checkpoints require requalification; use explicit legacy configs where the old contract is needed.

Validation

At 25ad1ecfc, based on current develop (ae5d4e397):

  • 72 focused tests passed on HORDE; formatting, changelog checks, and warning-free documentation build passed.
  • 28 rollouts used fresh, SHA-256-verified public-S3 checkpoints, with no checkpoint lookup cache and zero non-finite actions.
  • Reach/OSC, Drawer, and non-camera Soft/Cloth/Cable exceeded 90% task-level success.
  • Rigid Lift is not fully qualified by this asset-only change: Newton averaged 55.2% across seeds 40–42. The obsolete Newton IK-Rel artifact also remains unqualified.

CI fixes at 90960d451: excluded the robot from cuRobo world obstacles, and prevented randomized RL resets during static cable golden capture. The frontend checks, all six cuRobo tests, Isaac Sim RTX and active legacy kitless cable goldens passed; existing goldens and thresholds were unchanged. Formatting, changelog checks, and the warning-free docs build passed.

Full CI is being rerun. The controlled speed comparison remains pending; no performance multiplier or complete checkpoint qualification is claimed.

The separate ovstage-only cable renderer check also fails on unmodified develop; it is post-merge-only and is not part of the PR CI gate.

Release backport

  • Backport this pull request to the active release branch after it merges into develop

Merged latest upstream develop (7c05e96ca). Audited every test added or changed by this PR and removed 14 redundant cases: broad asset inventories, repeated preset combinations, comparisons of copied defaults, and a synthetic USD-variant test. Kept regression coverage for action normalization and offsets, zero-agent holds, disabled collider filtering, OSC solver limits, backend payloads surviving IK overrides, and deterministic cable golden capture.

Latest local validation: 128 focused tests passed across the six affected suites. Nine temporary production mutations failed on the intended retained assertions; production files were restored. Formatting and changelog checks passed. Earlier validation included minimal Reach, Reach OSC, and Lift random-agent rollouts and a clean warning-free documentation build. The final test pruning does not affect rendered docs.

@github-actions github-actions Bot added documentation Improvements or additions to documentation isaac-lab Related to Isaac Lab team labels Sep 29, 2026
@maxkra15
maxkra15 marked this pull request as ready for review September 29, 2026 20:56
@maxkra15
maxkra15 requested a review from a team September 29, 2026 20:56
@maxkra15

Copy link
Copy Markdown
Contributor Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 29, 2026
@greptile-apps

greptile-apps Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

[Medium risk] Migrates robot asset references and updates action processing.

This PR is not ready to merge until its prerequisite Franka asset definitions and exports are available.

Findings

  1. P1 Franka asset imports fail ▶

Summary

This PR moves Franka Reach, Drawer, contributed tasks, examples, and the render benchmark toward a shared flat asset, and adds an affine offset to differential-IK actions.

  • Backend and collider choices are configured for the core Reach and Drawer tasks.
  • The reviewed head cannot load its newly referenced Franka configurations until the prerequisite asset change is present.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Franka asset exports] --> B[Task and example imports]
  B --> C[Backend and collider presets]
  C --> D[Scene and controller setup]
Loading

Reviews (1) · Last reviewed commit: "Scope shared-asset tests to migrated rig..."

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

The flat-asset migration and additive differential-IK action offset are generally coherent, but backend-specific Physics variants are not selected in the controller tutorials or several contributed stack/lift configurations. The Reach task also changes success termination and reward behavior without a clear changelog entry. Two localized style inconsistencies should be cleaned up.

  • Design and architecture: Backend selection is correctly kept at configuration boundaries for Reach, Drawer, cabinet IK, Pour, and the render benchmark, and cabinet IK preserves the parent variant mapping across robot replacement. However, the dual-backend controller tutorials and contributed stack/lift configurations adopt the variant-bearing flat asset without applying the same backend-keyed Physics selection, leaving those paths architecturally inconsistent with the rest of the migration.
  • API: DifferentialInverseKinematicsActionCfg.offset is an additive API with a compatibility-preserving zero default. It is materialized as an owned batched tensor, applied as raw * scale + offset before clipping, exposed through the IO descriptor, documented in the action-term contract, and covered by a changelog fragment. The reviewed task-facing surfaces also include backend and collider variants, migrated prim-path expressions, actuator keys, and the new arm_collisions preset.
  • Compatibility and deprecation: Breaking changes: Isaac-Reach-Franka and Isaac-Reach-Franka-OSC no longer terminate on success and no longer provide the previous weight-10 success reward. This changes episode behavior and return scale, while the major changelog fragment currently discusses the asset, collision defaults, checkpoint requalification, and action/controller contracts without naming the success-term removal. The flat-asset dynamics and gripper-only collision default are documented, and the existing diffik_abs alias retains a targeted deprecation warning and functional replacement path.
  • Implementation: The affine action path through process_actions and set_command, OSC actuator-property preservation, Pour actuator migration, cabinet variant preservation, and render-benchmark variant resolution are internally consistent. The remaining implementation gap is that run_diff_ik.py, run_osc.py, and contributed stack/lift configurations use the flat asset without selecting mujoco versus physx according to the active backend, despite supporting more than one backend.
  • Style consistency: Import grouping, preset() usage, comments, and changelog placement generally follow adjacent patterns. Rename the cabinet IK local physics_variants to reflect that it preserves the complete robot variant mapping, including Colliders, and keep the short target_specs tuple entries consistently on one line rather than forcing unnecessary multi-line formatting.
  • Test quality: The new differential-IK action test owns the affine transform contract at the action-term boundary. The Reach preset tests cover asset identity, backend payloads, collision presets, normalized absolute actions, success tracking, and OSC actuator preservation; cabinet IK and render-benchmark tests cover backend-specific variants. Existing integration tests were also migrated deliberately between the flat and legacy plants. Coverage does not currently guard the missing backend variant selection in the controller tutorials or affected contributed stack/lift configurations.

Minor fixes needed. Posted 5 actionable findings inline.

Automated review; human maintainers own approval decisions.

Comment thread scripts/tutorials/05_controllers/run_diff_ik.py Outdated

@AntoineRichard AntoineRichard left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of NITs but looks good otherwise I think.

Comment thread scripts/tutorials/05_controllers/run_diff_ik.py
Comment thread source/isaaclab/test/envs/test_diffik_action_processing.py Outdated
@maxkra15 maxkra15 changed the title Migrate Franka Reach and Drawer to the main shared asset config Migrate Franka tasks to the shared asset and add minimal Reach Oct 1, 2026
@maxkra15

maxkra15 commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Oct 1, 2026
@maxkra15
maxkra15 requested a review from peterd-NV as a code owner October 1, 2026 17:47
@github-actions github-actions Bot added the isaac-mimic Related to Isaac Mimic team label Oct 1, 2026
@maxkra15

maxkra15 commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Oct 1, 2026
@maxkra15

maxkra15 commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Oct 1, 2026
@StafaH StafaH changed the title Migrate Franka tasks to the shared asset and add minimal Reach Migrate Franka tasks to the shared asset and add minimal presets Oct 1, 2026
@StafaH

StafaH commented Oct 1, 2026

Copy link
Copy Markdown
Collaborator

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Oct 1, 2026
@kellyguo11
kellyguo11 merged commit b7c0225 into isaac-sim:develop Oct 2, 2026
59 checks passed
isaaclab-bot Bot pushed a commit that referenced this pull request Oct 2, 2026
## Description

Migrate maintained Franka tasks, contributed configs, benchmarks, and
controller examples to the shared `FRANKA_PANDA_CFG`. Standard tasks use
full primitive arm/gripper collisions with explicit `Physics=physx` or
`Physics=mujoco` payloads.

Add `FRANKA_MINIMAL_CFG` and expose its gripper-only collision scope
through `preset()` in the existing Franka Reach and rigid Lift configs.
Select it with `presets=minimal`, including Reach OSC, while retaining
task-local controller and actuator tuning. Backend selection stays
independent of collision scope. No additional task ID or
environment-page section is added.

Preserve task-local actuator tuning, correct absolute Reach action
mapping and OSC actuator handling, and support differential-IK action
offsets in the zero agent. Reset clearance ignores disabled colliders,
and cuRobo excludes robot geometry from world obstacles even with custom
ignore lists. Legacy compatibility configs remain available; maintained
tasks no longer use them.

Lift reset/curriculum and continuous Reach tracking improvements are
isolated in dependent #8171. No USD packages, checkpoints, or new
dependencies are added to this source PR.

**Breaking:** migrated tasks change robot dynamics and collision scope.
Existing checkpoints require requalification; use explicit legacy
configs where the old contract is needed.

## Validation

At `25ad1ecfc`, based on current `develop` (`ae5d4e397`):

- 72 focused tests passed on HORDE; formatting, changelog checks, and
warning-free documentation build passed.
- 28 rollouts used fresh, SHA-256-verified public-S3 checkpoints, with
no checkpoint lookup cache and zero non-finite actions.
- Reach/OSC, Drawer, and non-camera Soft/Cloth/Cable exceeded 90%
task-level success.
- Rigid Lift is not fully qualified by this asset-only change: Newton
averaged 55.2% across seeds 40–42. The obsolete Newton IK-Rel artifact
also remains unqualified.

CI fixes at `90960d451`: excluded the robot from cuRobo world obstacles,
and prevented randomized RL resets during static cable golden capture.
The frontend checks, all six cuRobo tests, Isaac Sim RTX and active
legacy kitless cable goldens passed; existing goldens and thresholds
were unchanged. Formatting, changelog checks, and the warning-free docs
build passed.

Full CI is being rerun. The controlled speed comparison remains pending;
no performance multiplier or complete checkpoint qualification is
claimed.

The separate ovstage-only cable renderer check also fails on unmodified
`develop`; it is post-merge-only and is not part of the PR CI gate.

## Release backport

- [x] <!-- backport-active-release --> Backport this pull request to the
active release branch after it merges into `develop`

Merged latest upstream `develop` (`7c05e96ca`). Audited every test added
or changed by this PR and removed 14 redundant cases: broad asset
inventories, repeated preset combinations, comparisons of copied
defaults, and a synthetic USD-variant test. Kept regression coverage for
action normalization and offsets, zero-agent holds, disabled collider
filtering, OSC solver limits, backend payloads surviving IK overrides,
and deterministic cable golden capture.

Latest local validation: 128 focused tests passed across the six
affected suites. Nine temporary production mutations failed on the
intended retained assertions; production files were restored. Formatting
and changelog checks passed. Earlier validation included minimal Reach,
Reach OSC, and Lift random-agent rollouts and a clean warning-free
documentation build. The final test pruning does not affect rendered
docs.

---------

Co-authored-by: Mustafa Haiderbhai <mhaiderbhai@nvidia.com>

(cherry picked from commit b7c0225)
@isaaclab-bot

isaaclab-bot Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Backported to release/3.0.0 as 5aba1e9.

StafaH added a commit that referenced this pull request Oct 2, 2026
## Description

Improve Franka rigid Lift resets and motion regularization:

- Associate each of the eight cloned object geometries with its matching
finger opening and hand-frame orientation, independent of prototype
order.
- Apply aligned pre-grasps after the generic finger-width reset, cache
static reset data, and bound reset-bank prefill.
- Scale Franka motion penalties with success-driven ADR, avoid
terminating nominal arm-velocity excursions, and terminate non-finite
rigid-object states.

Restore continuous Franka Reach tracking without an early-success
termination or terminal success reward. Kuka reward/curriculum configs
remain unchanged. Asset, backend, actuator, Drawer, and deformable
migrations belong to #8170.

Updated against `develop` at `b7c022547` after #8170 merged. Minimal
Reach, Reach-OSC, and Lift use `presets=minimal`; there is no separate
minimal task config. The diff against `develop` contains only the 11
task behavior, test, and release-note files.

**Behavior change:** training and play use a mixed table/pregrasp reset
bank. Aligned pre-grasps have a 0.75 proposal probability before
rejection and bank sampling; these scores are not table-only pickup
success. For a separate table-only bank, set
`env.events.conditional_reset.params.terms.reset_object_to_target.params.probability=0`
before initialization.

## Validation

At `00387a373`, 60 focused Reach/Lift/preset tests and
formatting/changelog checks passed against current `develop`. Minimal
Reach (2 environments) and Lift (8 environments, reset bank reduced to
two states per group) each completed 10 random-agent steps with Newton
MJWarp; the Lift reset bank filled successfully. These are smoke checks,
not checkpoint requalification.

At `325bdd34e`: 79 focused tests, formatting/changelog checks, and a
warning-free documentation build passed. All 28 fresh-public-S3
checkpoint rollouts completed with zero non-finite actions.

Rigid Lift mean success: Isaac Sim PhysX 94.7%, Newton 83.5% (seeds
40–42), and OvPhysX 91.5% (seed 42). The same checkpoints are used in
#8170 and #8171; different reset distributions make this an MDP
comparison, not evidence of new policy training.

At `4f0c7ce8d`, the verified CI fixes from #8170 are included; 67
focused Lift/Reach/action tests and formatting/changelog checks passed.
Active Isaac Sim and legacy kitless cable golden tests now pass without
modifying goldens or thresholds. Full CI is being rerun.

**Remaining gates:** Newton's thin capsule scored 40.6%; OvPhysX's thin
plate scored 52.9%. The old Newton IK-Rel artifact remains unqualified.
Controlled speed measurements remain pending.

The separate ovstage-only cable renderer check also fails on unmodified
`develop`; it is post-merge-only and is not part of the PR CI gate.

## Release backport

- [x] <!-- backport-active-release --> Backport this pull request to the
active release branch after it merges into `develop`

---------

Co-authored-by: Mustafa Haiderbhai <mhaiderbhai@nvidia.com>
isaaclab-bot Bot pushed a commit that referenced this pull request Oct 2, 2026
## Description

Improve Franka rigid Lift resets and motion regularization:

- Associate each of the eight cloned object geometries with its matching
finger opening and hand-frame orientation, independent of prototype
order.
- Apply aligned pre-grasps after the generic finger-width reset, cache
static reset data, and bound reset-bank prefill.
- Scale Franka motion penalties with success-driven ADR, avoid
terminating nominal arm-velocity excursions, and terminate non-finite
rigid-object states.

Restore continuous Franka Reach tracking without an early-success
termination or terminal success reward. Kuka reward/curriculum configs
remain unchanged. Asset, backend, actuator, Drawer, and deformable
migrations belong to #8170.

Updated against `develop` at `b7c022547` after #8170 merged. Minimal
Reach, Reach-OSC, and Lift use `presets=minimal`; there is no separate
minimal task config. The diff against `develop` contains only the 11
task behavior, test, and release-note files.

**Behavior change:** training and play use a mixed table/pregrasp reset
bank. Aligned pre-grasps have a 0.75 proposal probability before
rejection and bank sampling; these scores are not table-only pickup
success. For a separate table-only bank, set
`env.events.conditional_reset.params.terms.reset_object_to_target.params.probability=0`
before initialization.

## Validation

At `00387a373`, 60 focused Reach/Lift/preset tests and
formatting/changelog checks passed against current `develop`. Minimal
Reach (2 environments) and Lift (8 environments, reset bank reduced to
two states per group) each completed 10 random-agent steps with Newton
MJWarp; the Lift reset bank filled successfully. These are smoke checks,
not checkpoint requalification.

At `325bdd34e`: 79 focused tests, formatting/changelog checks, and a
warning-free documentation build passed. All 28 fresh-public-S3
checkpoint rollouts completed with zero non-finite actions.

Rigid Lift mean success: Isaac Sim PhysX 94.7%, Newton 83.5% (seeds
40–42), and OvPhysX 91.5% (seed 42). The same checkpoints are used in
#8170 and #8171; different reset distributions make this an MDP
comparison, not evidence of new policy training.

At `4f0c7ce8d`, the verified CI fixes from #8170 are included; 67
focused Lift/Reach/action tests and formatting/changelog checks passed.
Active Isaac Sim and legacy kitless cable golden tests now pass without
modifying goldens or thresholds. Full CI is being rerun.

**Remaining gates:** Newton's thin capsule scored 40.6%; OvPhysX's thin
plate scored 52.9%. The old Newton IK-Rel artifact remains unqualified.
Controlled speed measurements remain pending.

The separate ovstage-only cable renderer check also fails on unmodified
`develop`; it is post-merge-only and is not part of the PR CI gate.

## Release backport

- [x] <!-- backport-active-release --> Backport this pull request to the
active release branch after it merges into `develop`

---------

Co-authored-by: Mustafa Haiderbhai <mhaiderbhai@nvidia.com>

(cherry picked from commit 34040cd)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

asset New asset feature or request documentation Improvements or additions to documentation isaac-lab Related to Isaac Lab team isaac-mimic Related to Isaac Mimic team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants