【Robotics】 Create Quadruped Robot RL FootStep Training Environment In IsaacLab

# 0. Breif

This document is used to give a whole description about how a beginner of robotics can create a training environment of `Quadruped Robot` in isaaclab.

The LLM and VLM models, like deepseek-v3, gpt-4o, etc. have been used to help me to search the useful documents and help me to understand the steps of each part.

# 1. Introduction

First it is important to know that nvidia gives us a whole toolchain to support the developers to create a `REAL2SIM2REAL` (which means create a sim environment with the real captured sensor data, and than training in the simulation environment, at last deploye the training model to the physical robot) environment to create out own robot.

We need to understand the relationship about omniverse, isaacsim, isaacgym(deprecated), isaaclab, etc.

* Containment Summary

```
NVIDIA Omniverse
└── Isaac Sim               ← is a simulator built using Omniverse tech
    └── used by Isaac Lab   ← Isaac Lab wraps Isaac Sim (and Gym) for RL
Isaac Gym (deprecated)          ← standalone, not part of Omniverse
└── used by Isaac Lab       ← Isaac Lab can use Isaac Gym (optionally)
```

``` mermaid
graph TD
    Omniverse --> IsaacSim
    IsaacSim --> IsaacLab
    IsaacGym --> IsaacLab
    IsaacLab --> RL_Frameworks

IsaacSim --> PhysX
    IsaacGym --> PhysX
    IsaacGym --> CUDA

class Omniverse platform
    class IsaacSim module
    class IsaacLab wrapper
    class IsaacGym legacy
    class RL_Frameworks ext

classDef platform fill:#F9F,stroke:#333,stroke-width:2;
    classDef module fill:#BBF,stroke:#333;
    classDef wrapper fill:#BFB,stroke:#333;
    classDef legacy fill:#FCC,stroke:#666,stroke-dasharray: 5 5;
    classDef ext fill:#EEE,stroke:#444;

```

``` mermaid
graph TD
    A[NVIDIA Omniverse] --> B[Isaac Sim]
    A --> C[Isaac Lab]
    C --> B
    C --> D[Isaac Gym]
    D --> E["Isaac Gym Preview Release - Deprecated"]

B --> F[USD-based Simulation Engine]
    B --> G[PhysX / RTX / ROS2 Integration]
    D --> H[GPU-based Physics for RL]

C --> I[RL + Robotics Benchmarking]
    C --> J[Interface Layer: Gym & Sim]

class A omniverse
    class B isaac
    class C isaaclab
    class D isaacgym
    class E deprecated
    class F,G,H tech
    class I,J usecase

classDef omniverse fill:#FFD700,stroke:#333,stroke-width:2;
    classDef isaac fill:#FFDD99,stroke:#333;
    classDef isaaclab fill:#FFB347,stroke:#333;
    classDef isaacgym fill:#FF9999,stroke:#333;
    classDef deprecated fill:#ccc,stroke:#666,stroke-dasharray: 5 5;
    classDef tech fill:#CDEFFF,stroke:#666;
    classDef usecase fill:#D0FFD0,stroke:#666;

```

A lot of works used isaacgym or the old version of isaaclab, which is not easy to understand the tutorial. What I want to do is to use the latest version of all things to create a good framework to demonstrate that I have understand how to make the quadruped robot walk. (The Feynman Study Method)

# 2 Try the Isaaclab

* Computer: 
	GPU RTX4090
	CPU 12th Gen Intel(R) Core(TM) i9-12900K
	MEM 128GB

* ubuntu 24.04, nvidia driver 570.124.06, cuda 12.8

* python 3.10.16

* isaacsim 4.5, isaaclab 2.1 ( master branch)

## 2.1 Install IsaacSim, IsaacLab

This link https://isaac-sim.github.io/IsaacLab/main/source/setup/installation/pip_installation.html#installing-isaac-lab (Annexe 3) is used to install isaaclab. It is really convinient and fluent.

### 2.2.1 Try IsaacSim

#### 2.2.1.1 Prepare Omniverse
* Install Omniverse Cache and launch it. Go to http://localhost:3080/ , which is used to optimise the data transformer.

![](/media/202504/2025-04-13_201300_4735390.44859937935308014.png)

* install nuleus server

![](/media/202504/2025-04-13_201714_7960130.006092374758115304.png)
![](/media/202504/2025-04-13_201838_9546340.8832501963087199.png)

* install nuleus navigater. This app helps your to navigate all the elements. If cannot connect the localhost server please check if some services stacked on http://localhost:3080/ 
![](/media/202504/2025-04-13_202014_7627120.7040012757365172.png)

#### 2.2.1.2 IsaacSim

* Launch IsaacSim

`./isaac-sim.sh`

![](/media/202504/2025-04-13_114642_2566250.23998748868377173.png)

* Try Assets
https://docs.isaacsim.omniverse.nvidia.com/latest/installation/install_faq.html#isaac-sim-setup-assets-content-pack

![](/media/202504/2025-04-13_114733_8459930.5028925001459694.png)

`There are three tabs for the tutorials, all three perform the same actions and reach the same outcome. Go through the full page under the same tab to learn about each workflow. Toggle between tabs to compare the different workflows.`

* Quickstart with Isaac Sim
https://docs.isaacsim.omniverse.nvidia.com/latest/introduction/quickstart_isaacsim.html

* Quickstart with a Robot
https://docs.isaacsim.omniverse.nvidia.com/latest/introduction/quickstart_isaacsim_robot.html

* IsaacSim WebRTC

* IsaaSim ROS

* IsaacSim Foxglove

### 2.2.2 Try IsaacLab

## 2.2 To understand the training of robot

See this video (Annexe 1), very clear!

* Structure
![](/media/202504/2025-04-13_115911_3213440.48625410562415883.png)

* Reward
![](/media/202504/2025-04-13_115953_8241190.9108773692786811.png)

### 2.2.1 Actuator, Joint, Motor

* USD, URDF, and MJX (MJX-XML / MJXL)

| Format        | Full Name                        | Used By                          | Purpose                                                                 |
|---------------|----------------------------------|----------------------------------|-------------------------------------------------------------------------|
| **USD**       | Universal Scene Description      | NVIDIA Omniverse, Isaac Sim      | Rich, scalable 3D scene format for robotics, VFX, games                |
| **URDF**      | Unified Robot Description Format | ROS (Robot Operating System)     | XML format for describing robot geometry and kinematics               |
| **MJXL / MJX**| MuJoCo XML / MJX-XML             | DeepMind’s MuJoCo                | XML format for physics-based simulation of robots and environments     |

```
         +----------+                   +------+
         |  URDF    |----[converter]--->| USD  |  --> Isaac Sim, Omniverse
         +----------+                   +------+
               |
               |----[urdf2mjcf]---+
               v                 |
          +--------+            |
          |  MJX   | <----------+
          +--------+     (can also write MJX by hand for MuJoCo)

```

### 2.2.2 RSL_RL

### 2.2.3 isaac_berkeley_humanoid

### 2.2.4 Quadruped Robot

[【附件】rl-video-step-68000.mp4](/media/attachment/2025/04/rl-video-step-68000.mp4)

[【附件】model_2900.pt](/media/attachment/2025/04/model_2900.pt)

[【附件】rl-video-step-28000.mp4](/media/attachment/2025/04/rl-video-step-28000.mp4)

[【附件】model_1000.pt](/media/attachment/2025/04/model_1000.pt)

[【附件】rl-video-step-4000.mp4](/media/attachment/2025/04/rl-video-step-4000.mp4)

[【附件】model_100.pt](/media/attachment/2025/04/model_100.pt)

# Annexe
* 1. https://www.youtube.com/watch?v=xwOaStX0mxE (a video about `How to build Humanoid: NVIDIA Isaac Lab, how to walk`)
* 2. https://www.nvidia.com/gtc/?ncid=ref-inpa-172249 (Nvidia GTC about robotics)
* 3. https://isaac-sim.github.io/IsaacLab/main/source/setup/installation/pip_installation.html#installing-isaac-lab (how to install isaaclab)
* 4. https://github.com/HybridRobotics/isaac_berkeley_humanoid (a task that could be used in isaaclab of how to training and running a humanoid robot)
* 5. https://docs.isaacsim.omniverse.nvidia.com/latest/installation/install_faq.html#isaac-sim-setup-assets-content-pack (isaacsim assets)
* 6. https://www.myactuator.com/ (actuator helps to understand the parameters of actuator)
* 7. https://www.youtube.com/@Skyentific (this teacher is great! Just follow him)
* 8. https://www.bilibili.com/video/BV1mEmxYkEa3?spm_id_from=333.788.videopod.sections&vd_source=c551ef751ac5907298e9c711d193f9cd (Isaac Sim examples)
* 9. https://www.bilibili.com/video/BV1rooaYVEk8/?spm_id_from=333.337.search-card.all.click&vd_source=94db5c5542d93967c5ec55fc3755b309