feat: port isaacgym example to isaaclab

[lvjonok]

Description

I have been researching reinforcement learning frameworks and possible integrations and one of the appealing choices nowadays is IsaacLab. However, I have found that there was no example on how one may use IsaacLab environments with CleanRL. Turns out the changes have not been significant, yet, it may help someone to bootstrap their work.

I have took ppo_continuous_action_isaacgym as a source and have changed IsaacGym into IsaacLab. Under the hood there are parameters that IsaacLab parses using argparse and they may interfere with CleanRL, so I have updated the script so one may pass both sets of parameters.

I am more than sure that I haven't made this PR ready-to-merge, but I am keen to update based on the comments. For instance, I think it would be wise to have IsaacGym vs IsaacLab training performance comparison. I think one of the limitations is python+torch version that IsaacLab requires. Although it was quite straightforward to install everything through pip, I bet the python version is only 3.10.

Types of changes

• Bug fix
• New feature
• New algorithm
• Documentation

Checklist:

• I've read the CONTRIBUTION guide (required).
• I have ensured pre-commit run --all-files passes (required).
• I have updated the tests accordingly (if applicable).
• I have updated the documentation and previewed the changes via mkdocs serve.
  • I have explained note-worthy implementation details.
  • I have explained the logged metrics.
  • I have added links to the original paper and related papers.

If you need to run benchmark experiments for a performance-impacting changes:

• I have contacted @vwxyzjn to obtain access to the openrlbenchmark W&B team.
• I have used the benchmark utility to submit the tracked experiments to the openrlbenchmark/cleanrl W&B project, optionally with --capture_video.
• I have performed RLops with python -m openrlbenchmark.rlops.
  • For new feature or bug fix:
    • I have used the RLops utility to understand the performance impact of the changes and confirmed there is no regression.
  • For new algorithm:
    • I have created a table comparing my results against those from reputable sources (i.e., the original paper or other reference implementation).
  • I have added the learning curves generated by the python -m openrlbenchmark.rlops utility to the documentation.
  • I have added links to the tracked experiments in W&B, generated by python -m openrlbenchmark.rlops ....your_args... --report, to the documentation.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
cleanrl	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 17, 2025 5:55am