EXport and dePLOY Reinforcement Learning policies.
The core idea lies in a "self-contained" export approach: Rather than exporting only the neural network policy, this tool captures the entire environment logic, including observation generation and action processing, into a single ONNX file. By tracing Torch operations from the simulation environment, the exporter embeds the computational layers required to transform raw robot state interfaces into policy inputs and policy outputs into executable commands.
By encapsulating the environment's computation graph within the model file itself, this library minimizes operational effort and maximizes confidence that a policy will behave identically in simulation and on physical hardware.
Authors: Dario Bellicoso, Annika Wollschläger
- Environment Exporting: Export RL environments and policies from simulation frameworks in a self-contained ONNX file.
- C++ Controller with ONNX Runtime Integration: Deploy trained policies using ONNX Runtime for real-time policy execution
- Multi-Framework Support: Built-in support for IsaacLab with extensible framework integration
control/: C++ controller library with ONNX Runtime integrationpython/exploy/: Python exporter package for policy and environment exportexamples/exporter_scripts/: Usage examples for supported frameworksexamples/controller/: Usage examples for control developmentdocs/: Documentation source files
Exploy's documentation is available at bdaiinstitute.github.io/exploy. To get started with Exploy's core concepts, refer to the following guides:
- Exporter — Step-by-step guide to
exporting an RL environment and policy to a self-contained ONNX file using
exploy.exporter.core. - Controller — Step-by-step guide to deploying a trained policy on a robot using the C++ controller with ONNX Runtime integration.
We use Pixi to build this repository. See the Pixi installation guide for setup instructions.
git clone https://github.com/bdaiinstitute/exploy.git
cd exployExploy is split into two packages: the Exporter package developed in Python and the Controller package developed in C++.
To use the exporter in your project, install it with pip:
pip install -e .The command above installs the core implementation of the exporter. This repository also provides
support for exporter integrations with environments developed in IsaacLab. You can install
the additional dependency with:
pip install -e .[isaaclab]The easiest way to consume the library is via add_subdirectory:
cmake_minimum_required(VERSION 3.20)
project(my_robot_controller LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
# Path to the exploy project root
set(EXPLOY_ROOT_DIR "/path/to/exploy" CACHE PATH "Exploy root directory")
# Pull in the exploy shared library (disables its tests in your build)
set(BUILD_TESTING OFF CACHE BOOL "" FORCE)
add_subdirectory(
"${EXPLOY_ROOT_DIR}/control"
"${CMAKE_CURRENT_BINARY_DIR}/exploy_control"
EXCLUDE_FROM_ALL
)
add_executable(my_controller main.cpp)
target_link_libraries(my_controller PRIVATE exploy)See examples/controller/ for a complete working example.
pixi installpixi run -e controller configure
pixi run -e controller build# Python tests
pixi run -e core test
# C++ tests
pixi run -e controller testStart with the documentation to learn the core workflow for exporting and
deploying a task. If you have an NVIDIA GPU, you can also run an IsaacLab exporter example with
pixi run -e isaaclab export-isaaclab.
This project uses semantic versioning (MAJOR.MINOR.PATCH). The current version is specified in pixi.toml.
Releases are published using GitHub Releases. Version tags follow the format vX.Y.Z.
All dependencies are managed through Pixi and specified in pixi.toml with version constraints.
We use pre-commit hooks to ensure code quality. To set up pre-commit hooks:
# Install pre-commit hooks
pixi run -e python pre-commit install
# Run pre-commit on all files (optional)
pixi run -e python pre-commit run --all-filesTasks are run through Pixi and defined in pixi.toml. Common tasks include:
pixi run -e core test: Run Python tests with pytest
pixi run -e core lint-python: Check Python code with ruffpixi run -e core format-python: Format Python code with ruff
pixi run -e controller configure: Run CMake configurationpixi run -e controller build: Build the C++ librarypixi run -e controller test: Run C++ tests with CTestpixi run -e controller format-cpp: Format C++ code with clang-format
pixi run -e isaaclab export-isaaclab: Export a policy for an existing IsaacLab task
This project is under light maintenance. No feature development is guaranteed, but if you have bug reports and/or pull requests that fix bugs, expect an RAI maintainer to respond within a few weeks.
We welcome your contributions!
To contribute:
- Report bugs and suggest improvements by opening an issue.
- Submit pull requests for code changes or documentation updates.
- Follow the project's code style and testing guidelines (see comments and existing code for reference).
- Ensure your changes pass all tests and pre-commit checks.
All contributions are reviewed by project maintainers before merging. Thank you for helping improve this project!
This project is licensed under the MIT License - see the LICENSE file for details.
Copyright (c) 2026 Robotics and AI Institute LLC dba RAI Institute
If you use this work in your research or project, please consider citing it using the 'Cite this repository' button in the sidebar, or using:
@misc{exploy2026,
author = {Dario Bellicoso, Annika Wollschläger},
title = {Exploy: EXport and dePLOY Reinforcement Learning policies},
month = {March},
year = {2026},
url = {https://github.com/bdaiinstitute/exploy}
}