PythonAI算法
使用 ISAAC Lab 训练机器人行走
ISAAC Lab 机器人训练指南涵盖环境配置、机器人注册及强化学习训练全流程。首先需搭建 Ubuntu 22.04 系统并安装 NVIDIA 驱动、CUDA、PyTorch 及 Anaconda 环境。随后通过 Isaac Sim 导入机器人 USD 文件，编写参数配置文件与奖励函数脚本完成机器人注册。最后利用 RSL-RL 或 skrl 框架进行策略训练与测试，支持平坦与崎岖地形场景。
ISAAC Lab 机器人训练指南涵盖环境配置、机器人注册及强化学习训练全流程。首先需搭建 Ubuntu 22.04 系统并安装 NVIDIA 驱动、CUDA、PyTorch 及 Anaconda 环境。随后通过 Isaac Sim 导入机器人 USD 文件，编写参数配置文件与奖励函数脚本完成机器人注册。最后利用 RSL-RL 或 skrl 框架进行策略训练与测试，支持平坦与崎岖地形场景。
参考官方文档进行安装，建议 /home 与 /usr 的硬盘容量均不少于 200G。
根据自身显卡型号与操作系统，选择对应的显卡驱动，建议选择 550.xxx.xxx 版本的显卡驱动，按照官方教程进行安装即可。安装完成后在终端输入 nvidia-smi，若出现以下信息则表示驱动安装成功：
Thu Jun 5 15:49:51 2025 +-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.14 Driver Version: 550.54.14 CUDA Version: 12.4 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| 0 NVIDIA GeForce RTX 4070 ... Off | 00000000:01:00.0 On | N/A |
| 30% 58C P2 130W / 245W | 6085MiB / 12282MiB | 80% Default |
+-----------------------------------------+------------------------+----------------------+
根据前面的 nvidia 显卡驱动的信息显示，选择对应的 cuda 版本进行安装。可以在 NVIDIA 的 CUDA Toolkit Archive 中，根据本机的操作系统找到对应版本的安装包。
具体安装步骤：
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin
sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/12.4.0/local_installers/cuda-repo-ubuntu2204-12-4-local_12.4.0-550.54.14-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu2204-12-4-local_12.4.0-550.54.14-1_amd64.deb
sudo cp /var/cuda-repo-ubuntu2204-12-4-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cuda-toolkit-12-4
安装后设置环境变量：
echo 'export PATH=/usr/local/cuda-12.4/bin:$PATH' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda-12.4/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
echo 'export CUDA_HOME=/usr/local/cuda-12.4' >> ~/.bashrc
source ~/.bashrc
验证 CUDA 安装是否成功，可以在终端输入以下命令：
nvcc --version
如果安装成功，会显示 CUDA 的版本信息。
前往 NVIDIA 的 cuDNN Archive 下载对应版本的 cuDNN。选择与 CUDA 版本相匹配的 cuDNN 版本。
找到下载的 cudnn 的 deb 文件所在目录，打开终端，执行下述命令：
sudo dpkg -i cudnn-local-repo-ubuntu2204-xxxxxxxxxxxxxx.deb
sudo cp /var/cudnn-local-repo-ubuntu2204-9.10.1/cudnn-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cudnn-cuda-12
根据前面下载的 cuda 版本，在 pytorch 官网找到对应的 pytorch 安装指令，在自己的 python 环境中执行以下命令安装 pytorch：
conda install pytorch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 pytorch-cuda=12.4-c pytorch -c nvidia
参考 Anaconda 官方安装教程。
可以参考 Isaac Sim 官方安装教程，以下是简要步骤：
conda create -n env_isaaclab python=3.10
conda activate env_isaaclab
pip install --upgrade pip
conda install pytorch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 pytorch-cuda=12.4-c pytorch -c nvidia
pip install 'isaacsim[all,extscache]==4.5.0' --extra-index-url https://pypi.nvidia.com
首次运行 isaacsim 需要较长时间。第一次运行将提示用户接受 NVIDIA 软件许可协议。要接受 EULA，请在出现以下消息提示时回复：
By installing or using Isaac Sim, I agree to the terms of NVIDIA SOFTWARE LICENSE AGREEMENT (EULA) in https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-software-license-agreement
Do you accept the EULA? (Yes/No): Yes
在 isaacsim 的 官方网站 下载 4.5 版本的 isaacsim 及 isaacsim 的资产包，资产包为 92G。
按照官方教程进行资产配置。此外，使用 vscode 进入 isaacsim 的库文件目录，在全局搜索中搜索默认的资产目录：
https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/4.5
全部替换成自己的资产目录：
https://your-asset-directory/Isaac/4.5
可以通过 git 拉取 isaaclab 的代码库，或者从官方 github 页面下载 zip 包，以下是通过 git 拉取的命令：
git clone [email protected]:isaac-sim/IsaacLab.git
安装后进入 isaaclab 目录，执行以下命令安装相关依赖：
./isaaclab.sh --install
# or "./isaaclab.sh -i"
验证安装是否成功，在 isaaclab.sh 所在目录执行：
./isaaclab.sh -p scripts/tutorials/00_sim/create_empty.py
可以通过 solidworks 导出机器人的 urdf 文件，然后打开 isaacsim，点击左上角的 File -> Import -> Import URDF，选择导出的 urdf 文件，导入后会生成一个 usd 文件，保存到指定目录。若是移动机器人，记得选择可动基座。若导入时卡住或者报错，检查 urdf 的完整性，检查 stl 路径，若 package://... 的路径格式不可行，建议替换成相对或绝对路径。
cd ~/isaaclab/source/isaaclab_tasks/isaaclab_tasks/manager_based/locomotion/velocity/config
复制已有的 h1 文件夹，重命名为你的机器人名称，比如这里使用 op3，创建一个 op3.py 作为 op3 机器人的参数文件。当前的目录结构应如下，按需求补充缺失脚本：
isaaclab_tasks
└── manager_based
└── locomotion
└── velocity
├── config
│ └── op3
│ └── agents
│ ├── __init__.py
│ ├── rsl_rl_ppo_cfg.py
│ ├── skrl_flat_ppo_cfg.yaml
│ ├── skrl_rough_ppo_cfg.yaml
│ ├── __init__.py
│ ├── op3.py
│ ├── flat_env_cfg.py
│ ├── rough_env_cfg.py
│ ├── check_op3.py
│ ├── velocity_env_cfg.py
│ └── op3_velocity_env_cfg.py
机器人参数文件：
机器人参数文件 op3.py 的内容如下：
import isaaclab.sim as sim_utils
from isaaclab.actuators import ImplicitActuatorCfg
from isaaclab.assets.articulation import ArticulationCfg
from isaaclab.utils.assets import ISAAC_NUCLEUS_DIR
OP3_CFG = ArticulationCfg(
spawn=sim_utils.UsdFileCfg(
usd_path=f"xxx/op3.usd", # 替换为你前面导出的机器人 usd 文件路径
activate_contact_sensors=True,
rigid_props=sim_utils.RigidBodyPropertiesCfg(
disable_gravity=False,
retain_accelerations=False,
linear_damping=0.0,
angular_damping=0.0,
max_linear_velocity=20.0,
max_angular_velocity=20.0,
max_depenetration_velocity=10.0,
),
articulation_props=sim_utils.ArticulationRootPropertiesCfg(
enabled_self_collisions=True,
solver_position_iteration_count=4,
solver_velocity_iteration_count=4,
),
),
init_state=ArticulationCfg.InitialStateCfg(
pos=(0.0, 0.0, 0.28), # 机器人初始位置，具体高度可由脚本 check_op3.py 查看
joint_pos={
"head_pan": 0.0,
"head_tilt": 0.0,
".*_hip_yaw": 0.0,
".*_hip_roll": 0.0,
".*_hip_pitch": 0.0,
".*_knee": 0.0,
".*_ank_pitch": 0.0,
".*_ank_roll": 0.0,
".*_sho_pitch": 0.0,
".*_sho_roll": 0.0,
".*_el": 0.0,
},
joint_vel={".*": 0.0},
),
soft_joint_pos_limit_factor=0.7,
actuators={
"legs": ImplicitActuatorCfg(
joint_names_expr=[".*_hip_yaw", ".*_hip_roll", ".*_hip_pitch", ".*_knee", ".*_ank_pitch", ".*_ank_roll"],
effort_limit_sim=20,
velocity_limit_sim=20.0,
stiffness={
".*_hip_yaw": 100.0,
".*_hip_roll": 100.0,
".*_hip_pitch": 100.0,
".*_knee": 100.0,
".*_ank_pitch": 100.0,
".*_ank_roll": 100.0,
},
damping={
".*_hip_yaw": 20.0,
".*_hip_roll": 20.0,
".*_hip_pitch": 20.0,
".*_knee": 20.0,
".*_ank_pitch": 20.0,
".*_ank_roll": 20.0,
},
),
"arms": ImplicitActuatorCfg(
joint_names_expr=[".*_sho_pitch", ".*_sho_roll", ".*_el"],
effort_limit_sim=20,
velocity_limit_sim=20.0,
stiffness={
".*_sho_pitch": 100.0,
".*_sho_roll": 100.0,
".*_el": 100.0,
},
damping={
".*_sho_pitch": 20.0,
".*_sho_roll": 20.0,
".*_el": 20.0,
},
),
"head": ImplicitActuatorCfg(
joint_names_expr=["head_pan", "head_tilt"],
effort_limit_sim=20,
velocity_limit_sim=20.0,
stiffness={"head_pan": 100.0, "head_tilt": 100.0},
damping={"head_pan": 20.0, "head_tilt": 20.0},
),
},
)
机器人基座高度检查脚本：
检查机器人基座高度 check_op3.py 的内容如下，可以先在 op3.py 中给机器人一个较高的初始基座高度，然后在这个脚本的输出中查看实际落地时机器人的基座高度，以调整机器人初始基座高度：
import argparse
import torch
from isaaclab.app import AppLauncher
parser = argparse.ArgumentParser(description="This script demonstrates how to simulate bipedal robots.")
AppLauncher.add_app_launcher_args(parser)
args_cli = parser.parse_args()
app_launcher = AppLauncher(args_cli)
simulation_app = app_launcher.app
import isaaclab.sim as sim_utils
from isaaclab.assets import Articulation
from isaaclab.sim import SimulationContext
# 改为你的机器人的参数配置脚本：
from op3 import OP3_CFG
def design_scene(sim: sim_utils.SimulationContext) -> tuple[list, torch.Tensor]:
"""Designs the scene."""
cfg = sim_utils.GroundPlaneCfg()
cfg.func("/World/defaultGroundPlane", cfg)
cfg = sim_utils.DomeLightCfg(intensity=2000.0, color=(0.75, 0.75, 0.75))
cfg.func("/World/Light", cfg)
origins = torch.tensor([[0.0, -1.0, 0.0], [0.0, 0.0, 0.0], [0.0, 1.0, 0.0],]).to(device=sim.device)
hr2 = Articulation(OP3_CFG.replace(prim_path="/World/G1"))
robots = [hr2]
return robots, origins
def run_simulator(sim: sim_utils.SimulationContext, robots: list[Articulation], origins: torch.Tensor):
"""Runs the simulation loop."""
sim_dt = sim.get_physics_dt()
sim_time = 0.0
count = 0
while simulation_app.is_running():
if count % 1000 == 0:
sim_time = 0.0
count = 0
for index, robot in enumerate(robots):
joint_pos, joint_vel = robot.data.default_joint_pos, robot.data.default_joint_vel
robot.write_joint_state_to_sim(joint_pos, joint_vel)
root_state = robot.data.default_root_state.clone()
root_state[:, :3] += origins[index]
robot.write_root_pose_to_sim(root_state[:, :7])
robot.write_root_velocity_to_sim(root_state[:, 7:])
robot.reset()
print(">>>>>>> Reset!")
for robot in robots:
robot.set_joint_position_target(robot.data.default_joint_pos.clone())
robot.write_data_to_sim()
sim.step()
sim_time += sim_dt
count += 1
for robot in robots:
robot.update(sim_dt)
root_pos = robot.data.root_pos_w # 获取机器人根位置
print(f"Robot height (z): {root_pos[:, 2].item():.3f}") # 打印 z 坐标（高度）
def main():
sim_cfg = sim_utils.SimulationCfg(dt=0.005, device=args_cli.device, gravity=[0.0, 0.0, -9.81],)
sim = SimulationContext(sim_cfg)
sim.set_camera_view(eye=[3.0, 0.0, 2.25], target=[0.0, 0.0, 1.0])
robots, origins = design_scene(sim)
sim.reset()
print("[INFO]: Setup complete...")
run_simulator(sim, robots, origins)
if __name__ == "__main__":
main()
simulation_app.close()
速度环境配置文件 op3_velocity_env_cfg.py 的内容如下，可以先复制原有的 velocity_env_cfg.py，然后对照着修改，需要修改的地方有注释说明：
# Copyright (c) 2022-2025, The Isaac Lab Project Developers.
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause
import math
from dataclasses import MISSING
import isaaclab.sim as sim_utils
from isaaclab.assets import ArticulationCfg, AssetBaseCfg
from isaaclab.envs import ManagerBasedRLEnvCfg
from isaaclab.managers import CurriculumTermCfg as CurrTerm
from isaaclab.managers import EventTermCfg as EventTerm
from isaaclab.managers import ObservationGroupCfg as ObsGroup
from isaaclab.managers import ObservationTermCfg as ObsTerm
from isaaclab.managers import RewardTermCfg as RewTerm
from isaaclab.managers import SceneEntityCfg
from isaaclab.managers import TerminationTermCfg as DoneTerm
from isaaclab.scene import InteractiveSceneCfg
from isaaclab.sensors import ContactSensorCfg, RayCasterCfg, patterns
from isaaclab.terrains import TerrainImporterCfg
from isaaclab.utils import configclass
from isaaclab.utils.assets import ISAAC_NUCLEUS_DIR, ISAACLAB_NUCLEUS_DIR
from isaaclab.utils.noise import AdditiveUniformNoiseCfg as Unoise
import isaaclab_tasks.manager_based.locomotion.velocity.mdp as mdp
from isaaclab.terrains.config.rough import ROUGH_TERRAINS_CFG
# isort: skip
from isaaclab.managers import ActionTermCfg
@configclass
class MySceneCfg(InteractiveSceneCfg):
"""Configuration for the terrain scene with a legged robot."""
# ground terrain
terrain = TerrainImporterCfg(
prim_path="/World/ground",
terrain_type="generator",
terrain_generator=ROUGH_TERRAINS_CFG,
max_init_terrain_level=5,
collision_group=-1,
physics_material=sim_utils.RigidBodyMaterialCfg(
friction_combine_mode="multiply",
restitution_combine_mode="multiply",
static_friction=1.0,
dynamic_friction=1.0,
),
visual_material=sim_utils.MdlFileCfg(
mdl_path=f"{ISAACLAB_NUCLEUS_DIR}/Materials/TilesMarbleSpiderWhiteBrickBondHoned/TilesMarbleSpiderWhiteBrickBondHoned.mdl",
project_uvw=True,
texture_scale=(0.25, 0.25),
),
debug_vis=False,
)
robot: ArticulationCfg = MISSING
height_scanner = RayCasterCfg(
prim_path="{ENV_REGEX_NS}/Robot/robotis_op3/base", # 将机器人 usd 放入 isaacsim，查看自己机器人的 base_link 名称，将这里的 robotis_op3/base 替换为自己的机器人名称/base_link 的 link 名称
offset=RayCasterCfg.OffsetCfg(pos=(0.0, 0.0, 20.0)),
attach_yaw_only=True,
pattern_cfg=patterns.GridPatternCfg(resolution=0.1, size=[1.6, 1.0]),
debug_vis=False,
mesh_prim_paths=["/World/ground"],
)
contact_forces = ContactSensorCfg(prim_path="{ENV_REGEX_NS}/Robot/robotis_op3/.*", history_length=3, track_air_time=True) # 将 robotis_op3 替换为自己的机器人名称
sky_light = AssetBaseCfg(
prim_path="/World/skyLight",
spawn=sim_utils.DomeLightCfg(
intensity=750.0,
texture_file=f"{ISAAC_NUCLEUS_DIR}/Materials/Textures/Skies/PolyHaven/kloofendal_43d_clear_puresky_4k.hdr",
),
)
@configclass
class CommandsCfg:
"""Command specifications for the MDP."""
"""设定机器人在世界坐标系下跟踪期望的线速度和角速度"""
base_velocity = mdp.UniformVelocityCommandCfg(
asset_name="robot", # 作用对象（机器人名）
resampling_time_range=(10.0, 10.0), # 多少秒重新采样一次目标命令（这里固定为 10 秒）
rel_standing_envs=0.02, # 站立环境的比例（通常用于 curriculum 或特殊奖励）
rel_heading_envs=1.0, # 有 heading 命令的环境比例（1.0 表示全部环境都用 heading 命令）
heading_command=True, # 是否启用 heading 命令（目标朝向）
heading_control_stiffness=0.5, # 朝向控制的刚度（影响朝向跟踪的'紧迫感'）
debug_vis=True, # 是否可视化目标命令
ranges=mdp.UniformVelocityCommandCfg.Ranges(
lin_vel_x=(-1.0, 1.0), # 目标 x 方向线速度采样范围（前后）
lin_vel_y=(-1.0, 1.0), # 目标 y 方向线速度采样范围（左右）
ang_vel_z=(-1.0, 1.0), # 目标 z 轴角速度采样范围（旋转）
heading=(-math.pi, math.pi), # 目标朝向采样范围（弧度，-π到π）
),
)
@configclass
class ActionsCfg:
"""Action specifications for the MDP."""
joint_pos = mdp.JointPositionActionCfg(
asset_name="robot",
joint_names=[".*"],
scale=0.5,
use_default_offset=True
)
@configclass
class ObservationsCfg:
"""Observation specifications for the MDP."""
@configclass
class PolicyCfg(ObsGroup):
"""Observations for policy group."""
base_lin_vel = ObsTerm(func=mdp.base_lin_vel, noise=Unoise(n_min=-0.1, n_max=0.1))
base_ang_vel = ObsTerm(func=mdp.base_ang_vel, noise=Unoise(n_min=-0.2, n_max=0.2))
projected_gravity = ObsTerm(
func=mdp.projected_gravity,
noise=Unoise(n_min=-0.05, n_max=0.05),
)
velocity_commands = ObsTerm(func=mdp.generated_commands, params={"command_name": "base_velocity"})
joint_pos = ObsTerm(func=mdp.joint_pos_rel, noise=Unoise(n_min=-0.01, n_max=0.01))
joint_vel = ObsTerm(func=mdp.joint_vel_rel, noise=Unoise(n_min=-1.5, n_max=1.5))
actions = ObsTerm(func=mdp.last_action)
height_scan = ObsTerm(
func=mdp.height_scan,
params={"sensor_cfg": SceneEntityCfg("height_scanner")},
noise=Unoise(n_min=-0.1, n_max=0.1),
clip=(-1.0, 1.0),
)
def __post_init__(self):
self.enable_corruption = True
self.concatenate_terms = True
policy: PolicyCfg = PolicyCfg()
@configclass
class EventCfg:
"""Configuration for events."""
physics_material = EventTerm(
func=mdp.randomize_rigid_body_material,
mode="startup",
params={
"asset_cfg": SceneEntityCfg("robot", body_names=".*"),
"static_friction_range": (0.8, 0.8),
"dynamic_friction_range": (0.6, 0.6),
"restitution_range": (0.0, 0.0),
"num_buckets": 64,
},
)
add_base_mass = EventTerm(
func=mdp.randomize_rigid_body_mass,
mode="startup",
params={
"asset_cfg": SceneEntityCfg("robot", body_names="base"), # 将 base 替换为自己的机器人 base_link 的 link 名称
"mass_distribution_params": (-5.0, 5.0),
"operation": "add",
},
)
base_external_force_torque = EventTerm(
func=mdp.apply_external_force_torque,
mode="reset",
params={
"asset_cfg": SceneEntityCfg("robot", body_names="base"), # 将 base 替换为自己的机器人 base_link 的 link 名称
"force_range": (0.0, 0.0),
"torque_range": (-0.0, 0.0),
},
)
reset_base = EventTerm(
func=mdp.reset_root_state_uniform,
mode="reset",
params={
"pose_range": {"x": (-0.5, 0.5), "y": (-0.5, 0.5), "yaw": (-3.14, 3.14)},
"velocity_range": {
"x": (-0.5, 0.5),
"y": (-0.5, 0.5),
"z": (-0.5, 0.5),
"roll": (-0.5, 0.5),
"pitch": (-0.5, 0.5),
"yaw": (-0.5, 0.5),
},
},
)
reset_robot_joints = EventTerm(
func=mdp.reset_joints_by_scale,
mode="reset",
params={"position_range": (0.5, 1.5), "velocity_range": (0.0, 0.0)},
)
push_robot = EventTerm(
func=mdp.push_by_setting_velocity,
mode="interval",
interval_range_s=(10.0, 15.0),
params={"velocity_range": {"x": (-0.5, 0.5), "y": (-0.5, 0.5)}},
)
@configclass
class RewardsCfg:
"""Reward terms for the MDP."""
track_lin_vel_xy_exp = RewTerm(
func=mdp.track_lin_vel_xy_exp,
weight=1.0,
params={"command_name": "base_velocity", "std": math.sqrt(0.25)}
)
track_ang_vel_z_exp = RewTerm(
func=mdp.track_ang_vel_z_exp,
weight=0.5,
params={"command_name": "base_velocity", "std": math.sqrt(0.25)}
)
# -- penalties
lin_vel_z_l2 = RewTerm(func=mdp.lin_vel_z_l2, weight=-2.0)
ang_vel_xy_l2 = RewTerm(func=mdp.ang_vel_xy_l2, weight=-0.05)
dof_torques_l2 = RewTerm(func=mdp.joint_torques_l2, weight=-1.0e-5)
dof_acc_l2 = RewTerm(func=mdp.joint_acc_l2, weight=-2.5e-7)
action_rate_l2 = RewTerm(func=mdp.action_rate_l2, weight=-0.01)
feet_air_time = RewTerm(
func=mdp.feet_air_time,
weight=0.125,
params={
"sensor_cfg": SceneEntityCfg("contact_forces", body_names=".*ank_roll_link"),
"command_name": "base_velocity",
"threshold": 0.5,
},
# 将.*ank_roll_link 替换为自己的机器人脚部关节 link 名称
)
# undesired_contacts = RewTerm(# func=mdp.undesired_contacts,# weight=-1.0,# params={"sensor_cfg": SceneEntityCfg("contact_forces", body_names=".*THIGH"), "threshold": 1.0},# ) # 暂不使用
flat_orientation_l2 = RewTerm(func=mdp.flat_orientation_l2, weight=0.0)
dof_pos_limits = RewTerm(func=mdp.joint_pos_limits, weight=0.0)
@configclass
class TerminationsCfg:
"""Termination terms for the MDP."""
time_out = DoneTerm(func=mdp.time_out, time_out=True)
base_contact = DoneTerm(
func=mdp.illegal_contact,
params={"sensor_cfg": SceneEntityCfg("contact_forces", body_names="base"), "threshold": 1.0},
# 将 base 替换为自己的机器人 base_link 的 link 名称
)
@configclass
class CurriculumCfg:
"""Curriculum terms for the MDP."""
terrain_levels = CurrTerm(func=mdp.terrain_levels_vel)
@configclass
class LocomotionVelocityRoughEnvCfg(ManagerBasedRLEnvCfg):
"""Configuration for the locomotion velocity-tracking environment."""
scene: MySceneCfg = MySceneCfg(num_envs=4096, env_spacing=2.5)
observations: ObservationsCfg = ObservationsCfg()
actions: ActionsCfg = ActionsCfg()
commands: CommandsCfg = CommandsCfg()
rewards: RewardsCfg = RewardsCfg()
terminations: TerminationsCfg = TerminationsCfg()
events: EventCfg = EventCfg()
curriculum: CurriculumCfg = CurriculumCfg()
def __post_init__(self):
"""Post initialization."""
self.decimation = 4
self.episode_length_s = 20.0 # simulation settings
self.sim.dt = 0.005
self.sim.render_interval = self.decimation
self.sim.physics_material = self.scene.terrain.physics_material
self.sim.physx.gpu_max_rigid_patch_count = 10 * 2 ** 15
if self.scene.height_scanner is not None:
self.scene.height_scanner.update_period = self.decimation * self.sim.dt
if self.scene.contact_forces is not None:
self.scene.contact_forces.update_period = self.sim.dt
if getattr(self.curriculum, "terrain_levels", None) is not None:
if self.scene.terrain.terrain_generator is not None:
self.scene.terrain.terrain_generator.curriculum = True
else:
if self.scene.terrain.terrain_generator is not None:
self.scene.terrain.terrain_generator.curriculum = False
复杂地面环境参数配置脚本
复杂地面环境配置文件 rough_env_cfg.py 的内容如下：
# Copyright (c) 2022-2025, The Isaac Lab Project Developers.
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause
from isaaclab.managers import RewardTermCfg as RewTerm
from isaaclab.managers import SceneEntityCfg
from isaaclab.utils import configclass
import isaaclab_tasks.manager_based.locomotion.velocity.mdp as mdp
from isaaclab_tasks.manager_based.locomotion.velocity.op3_velocity_env_cfg import LocomotionVelocityRoughEnvCfg, RewardsCfg
from op3 import OP3_CFG # 导入前面编写的机器人参数配置脚本
import random
@configclass
class OP3Rewards(RewardsCfg):
# 可以将全文的 OP3 替换为自己机器人的每次，后面不再提起这点
"""Reward terms for the MDP."""
"""机器人死亡时的惩罚"""
termination_penalty = RewTerm(func=mdp.is_terminated, weight=-200.0)
lin_vel_z_l2 = None
"""奖励机器人在机器人自身的朝向坐标系下跟踪期望的 xy 线速度"""
track_lin_vel_xy_exp = RewTerm(
func=mdp.track_lin_vel_xy_yaw_frame_exp,
weight=1.0,
params={"command_name": "base_velocity", "std": 0.5},
)
"""奖励机器人在世界坐标系下跟踪期望的 z 轴角速度"""
track_ang_vel_z_exp = RewTerm(
func=mdp.track_ang_vel_z_world_exp,
weight=1.0,
params={"command_name": "base_velocity", "std": 0.5}
)
"""奖励双足交替抬起 (步态),鼓励有步态的行走，在飞行测试时，令 weight=0 即可"""
feet_air_time = RewTerm(
func=mdp.feet_air_time_positive_biped,
weight=0.25,
params={
"command_name": "base_velocity",
"sensor_cfg": SceneEntityCfg("contact_forces", body_names=".*ank_roll_link"),
"threshold": 0.4,
},
# 将.*ank_roll_link 替换为自己的机器人脚部关节 link 名称
)
"""惩罚脚在地面滑动 (非理想步态)"""
feet_slide = RewTerm(
func=mdp.feet_slide,
weight=-0.25,
params={
"sensor_cfg": SceneEntityCfg("contact_forces", body_names=".*ank_roll_link"),
"asset_cfg": SceneEntityCfg("robot", body_names=".*ank_roll_link"),
},
# 将.*ank_roll_link 替换为自己的机器人脚部关节 link 名称
)
# Penalize ankle joint limits
"""惩罚踝关节超出关节极限"""
dof_pos_limits = RewTerm(
func=mdp.joint_pos_limits,
weight=-1.0,
params={"asset_cfg": SceneEntityCfg("robot", joint_names=".*_ank_roll")}
) # 将.*_ank_roll 替换为自己的机器人脚部关节 joint 名称
"""惩罚髋关节 (hip_yaw, hip_roll,hip_yaw) 偏离默认值"""
joint_deviation_hip = RewTerm(
func=mdp.joint_deviation_l1,
weight=-0.2,
params={"asset_cfg": SceneEntityCfg("robot", joint_names=[".*_hip_roll", ".*_hip_pitch", ".*_hip_yaw"])
} # 将.*_hip_roll, .*_hip_pitch, .*_hip_yaw 替换为自己的机器人髋关节 joint 名称
"""惩罚膝关节 (knee) 偏离默认值"""
# 对应：Rough-OP3-train
@configclass
class OP3RoughEnvCfg(LocomotionVelocityRoughEnvCfg):
rewards: OP3Rewards = OP3Rewards()
def __post_init__(self):
super().__post_init__()
# Scene
self.scene.robot = OP3_CFG.replace(prim_path="{ENV_REGEX_NS}/Robot")
if self.scene.height_scanner:
self.scene.height_scanner.prim_path = "{ENV_REGEX_NS}/Robot/robotis_op3/base" # 将 robotis_op3/base 替换为自己的机器人名称/base_link 的 link 名称
self.events.push_robot = None
self.events.add_base_mass = None
self.events.reset_robot_joints.params["position_range"] = (1.0, 1.0)
self.events.base_external_force_torque.params["asset_cfg"].body_names = [".*base"] # 将 base 替换为自己的机器人 base_link 的 link 名称
self.events.reset_base.params = {
"pose_range": {"x": (-0.5, 0.5), "y": (-0.5, 0.5), "yaw": (-3.14, 3.14)},
"velocity_range": {
"x": (0.0, 0.0),
"y": (0.0, 0.0),
"z": (0.0, 0.0),
"roll": (0.0, 0.0),
"pitch": (0.0, 0.0),
"yaw": (0.0, 0.0),
},
}
self.terminations.base_contact.params["sensor_cfg"].body_names = [".*base"]
self.rewards.undesired_contacts = None
self.rewards.flat_orientation_l2.weight = -1.0
self.rewards.dof_torques_l2.weight = 0.0
self.rewards.action_rate_l2.weight = -0.005
self.rewards.dof_acc_l2.weight = -1.25e-7
self.commands.base_velocity.ranges.lin_vel_x = (0.5, 1.0) # 前后速度范围
self.commands.base_velocity.ranges.lin_vel_y = (0.0, 0.0) # 左右速度范围
self.commands.base_velocity.ranges.ang_vel_z = (-1.0, 1.0) # 旋转速度范围
self.terminations.base_contact.params["sensor_cfg"].body_names = ".*base"
"""对应：Rough-OP3-Play"""
@configclass
class OP3RoughEnvCfg_PLAY(OP3RoughEnvCfg):
def __post_init__(self):
super().__post_init__()
self.scene.num_envs = 1
self.scene.env_spacing = 2.5
self.episode_length_s = 40.0
self.scene.terrain.max_init_terrain_level = None
if self.scene.terrain.terrain_generator is not None:
self.scene.terrain.terrain_generator.num_rows = 5
self.scene.terrain.terrain_generator.num_cols = 5
self.scene.terrain.terrain_generator.curriculum = False
# 与 OP3RoughEnvCfg 的 commands 对应：
self.commands.base_velocity.ranges.lin_vel_x = (1.0, 1.0)
self.commands.base_velocity.ranges.lin_vel_y = (0.0, 0.0)
self.commands.base_velocity.ranges.ang_vel_z = (-1.0, 1.0)
self.commands.base_velocity.ranges.heading = (0.0, 0.0)
self.observations.policy.enable_corruption = False
self.events.base_external_force_torque = None
self.events.push_robot = None
平整地面环境参数配置脚本
平地环境配置文件 flat_env_cfg.py 的内容如下：
# Copyright (c) 2022-2025, The Isaac Lab Project Developers.
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause
from isaaclab.utils import configclass
from .rough_env_cfg import OP3RoughEnvCfg # 一样的，将脚本中的'OP3'替换为自己的机器人名称，其他的不用改
@configclass
class OP3FlatEnvCfg(OP3RoughEnvCfg):
def __post_init__(self):
super().__post_init__()
self.scene.terrain.terrain_type = "plane"
self.scene.terrain.terrain_generator = None
self.scene.height_scanner = None
self.observations.policy.height_scan = None
self.curriculum.terrain_levels = None
self.rewards.feet_air_time.weight = 1.0
self.rewards.feet_air_time.params["threshold"] = 0.6
class OP3FlatEnvCfg_PLAY(OP3FlatEnvCfg):
def __post_init__(self) -> None:
super().__post_init__()
self.scene.num_envs = 1
self.scene.env_spacing = 2.5
self.observations.policy.enable_corruption = False
self.events.base_external_force_torque = None
self.events.push_robot = None
在 init.py 中注册机器人训练环境，内容如下，只需要把 OP3 替换为自己的机器人名称即可：
# Copyright (c) 2022-2025, The Isaac Lab Project Developers.
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause
import gymnasium as gym
from . import agents
### Register Gym environments.
""" 地形：崎岖地形 (rough terrain),有地形生成器和难度课程。
用途：标准训练环境，适合训练机器人在复杂地形上行走。
奖励、终止、观测等：完整，适合正式训练。
"""
gym.register(
id="Rough-OP3-train",
entry_point="isaaclab.envs:ManagerBasedRLEnv",
disable_env_checker=True,
kwargs={
"env_cfg_entry_point": f"{__name__}.rough_env_cfg:OP3RoughEnvCfg",
"rsl_rl_cfg_entry_point": f"{agents.__name__}.rsl_rl_ppo_cfg:OP3RoughPPORunnerCfg",
"skrl_cfg_entry_point": f"{agents.__name__}:skrl_rough_ppo_cfg.yaml",
},
)
""" 地形：崎岖地形，但用于'Play'模式。
区别：
环境数量：1，更适合测试和可视化。
地形课程关闭，地形数量减少，内存占用低。
随机扰动/推搡等事件关闭，更稳定。
观测扰动关闭，便于观察真实表现。
用途：用于演示、可视化、调试和模型评估。
"""
gym.register(
id="Rough-OP3-Play",
entry_point="isaaclab.envs:ManagerBasedRLEnv",
disable_env_checker=True,
kwargs={
"env_cfg_entry_point": f"{__name__}.rough_env_cfg:OP3RoughEnvCfg_PLAY",
"rsl_rl_cfg_entry_point": f"{agents.__name__}.rsl_rl_ppo_cfg:OP3RoughPPORunnerCfg",
"skrl_cfg_entry_point": f"{agents.__name__}:skrl_rough_ppo_cfg.yaml",
},
)
""" 地形：平地 (flat terrain),无地形生成器。
区别：
无地形难度课程，地形始终为平面。
无高度扫描观测，观测量减少。
奖励参数适配平地。
用途：适合在平地上训练，便于对比和基础能力训练。
"""
gym.register(
id="Flat-OP3-train",
entry_point="isaaclab.envs:ManagerBasedRLEnv",
disable_env_checker=True,
kwargs={
"env_cfg_entry_point": f"{__name__}.flat_env_cfg:OP3FlatEnvCfg",
"rsl_rl_cfg_entry_point": f"{agents.__name__}.rsl_rl_ppo_cfg:OP3FlatPPORunnerCfg",
"skrl_cfg_entry_point": f"{agents.__name__}:skrl_flat_ppo_cfg.yaml",
},
)
""" 地形：平地，Play 模式。
区别：
环境数量少，适合测试。
无扰动、无观测噪声，便于可视化和调试。
用途：平地上的演示、可视化、调试和模型评估。
"""
gym.register(
id="Flat-OP3-Play",
entry_point="isaaclab.envs:ManagerBasedRLEnv",
disable_env_checker=True,
kwargs={
"env_cfg_entry_point": f"{__name__}.flat_env_cfg:OP3FlatEnvCfg_PLAY",
"rsl_rl_cfg_entry_point": f"{agents.__name__}.rsl_rl_ppo_cfg:OP3FlatPPORunnerCfg",
"skrl_cfg_entry_point": f"{agents.__name__}:skrl_flat_ppo_cfg.yaml",
},
)
rsl_rl_ppo_cfg.py:
# Copyright (c) 2022-2025, The Isaac Lab Project Developers.
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause
from isaaclab.utils import configclass
from isaaclab_rl.rsl_rl import RslRlOnPolicyRunnerCfg, RslRlPpoActorCriticCfg, RslRlPpoAlgorithmCfg
@configclass
class OP3RoughPPORunnerCfg(RslRlOnPolicyRunnerCfg):
# 将'OP3'替换为自己的机器人名称即可，其他参数可不改
num_steps_per_env = 24
max_iterations = 3000
save_interval = 50
experiment_name = "OP3_rough"
empirical_normalization = False
policy = RslRlPpoActorCriticCfg(
init_noise_std=1.0,
actor_hidden_dims=[512, 256, 128],
critic_hidden_dims=[512, 256, 128],
activation="elu",
)
algorithm = RslRlPpoAlgorithmCfg(
value_loss_coef=1.0,
use_clipped_value_loss=True,
clip_param=0.2,
entropy_coef=0.01,
num_learning_epochs=5,
num_mini_batches=4,
learning_rate=1.0e-3,
schedule="adaptive",
gamma=0.99,
lam=0.95,
desired_kl=0.01,
max_grad_norm=1.0,
)
@configclass
class OP3FlatPPORunnerCfg(OP3RoughPPORunnerCfg):
def __post_init__(self):
super().__post_init__()
self.max_iterations = 1000
self.experiment_name = "OP3_flat"
self.policy.actor_hidden_dims = [128, 128, 128]
self.policy.critic_hidden_dims = [128, 128, 128]
skrl_flat_ppo_cfg.yaml 和 skrl_rough_ppo_cfg.yaml:
skrl_flat_ppo_cfg.yaml 和 skrl_rough_ppo_cfg.yaml 的内容中，只需要把 OP3 替换为自己的机器人名称即可，其他参数可不改。
skrl_flat_ppo_cfg.yaml 的内容如下：
seed: 42
# Models are instantiated using skrl's model instantiator utility
# https://skrl.readthedocs.io/en/latest/api/utils/model_instantiators.html
models:
separate: False
policy:
# see gaussian_model parameters
class: GaussianMixin
clip_actions: False
clip_log_std: True
min_log_std: -20.0
max_log_std: 2.0
initial_log_std: 0.0
network:
- name: net
input: STATES
layers: [128, 128, 128]
activations: elu
output: ACTIONS
value:
# see deterministic_model parameters
class: DeterministicMixin
clip_actions: False
network:
- name: net
input: STATES
layers: [128, 128, 128]
activations: elu
output: ONE
# Rollout memory
# https://skrl.readthedocs.io/en/latest/api/memories/random.html
memory:
class: RandomMemory
memory_size: -1 # automatically determined (same as agent:rollouts)
# PPO agent configuration (field names are from PPO_DEFAULT_CONFIG)
# https://skrl.readthedocs.io/en/latest/api/agents/ppo.html
agent:
class: PPO
rollouts: 24
learning_epochs: 5
mini_batches: 4
discount_factor: 0.99
lambda: 0.95
learning_rate: 1.0e-03
learning_rate_scheduler: KLAdaptiveLR
learning_rate_scheduler_kwargs:
kl_threshold: 0.01
state_preprocessor: null
state_preprocessor_kwargs: null
value_preprocessor: null
value_preprocessor_kwargs: null
random_timesteps: 0
learning_starts: 0
grad_norm_clip: 1.0
ratio_clip: 0.2
value_clip: 0.2
clip_predicted_values: True
entropy_loss_scale: 0.01
value_loss_scale: 1.0
kl_threshold: 0.0
rewards_shaper_scale: 1.0
time_limit_bootstrap: False
# logging and checkpoint
experiment:
directory: "OP3_flat"
experiment_name: ""
write_interval: auto
checkpoint_interval: auto
# Sequential trainer
# https://skrl.readthedocs.io/en/latest/api/trainers/sequential.html
trainer:
class: SequentialTrainer
timesteps: 24000
environment_info: log
skrl_rough_ppo_cfg.yaml 的内容如下：
seed: 42
# Models are instantiated using skrl's model instantiator utility
# https://skrl.readthedocs.io/en/latest/api/utils/model_instantiators.html
models:
separate: False
policy:
# see gaussian_model parameters
class: GaussianMixin
clip_actions: False
clip_log_std: True
min_log_std: -20.0
max_log_std: 2.0
initial_log_std: 0.0
network:
- name: net
input: STATES
layers: [512, 256, 128]
activations: elu
output: ACTIONS
value:
# see deterministic_model parameters
class: DeterministicMixin
clip_actions: False
network:
- name: net
input: STATES
layers: [512, 256, 128]
activations: elu
output: ONE
# Rollout memory
# https://skrl.readthedocs.io/en/latest/api/memories/random.html
memory:
class: RandomMemory
memory_size: -1 # automatically determined (same as agent:rollouts)
# PPO agent configuration (field names are from PPO_DEFAULT_CONFIG)
# https://skrl.readthedocs.io/en/latest/api/agents/ppo.html
agent:
class: PPO
rollouts: 24
learning_epochs: 5
mini_batches: 4
discount_factor: 0.995
lambda: 0.95
learning_rate: 1.0e-03
learning_rate_scheduler: KLAdaptiveLR
learning_rate_scheduler_kwargs:
kl_threshold: 0.01
state_preprocessor: null
state_preprocessor_kwargs: null
value_preprocessor: null
value_preprocessor_kwargs: null
random_timesteps: 0
learning_starts: 0
grad_norm_clip: 1.0
ratio_clip: 0.2
value_clip: 0.2
clip_predicted_values: True
entropy_loss_scale: 0.01
value_loss_scale: 1.0
kl_threshold: 0.0
rewards_shaper_scale: 1.0
time_limit_bootstrap: False
# logging and checkpoint
experiment:
directory: "OP3_rough"
experiment_name: ""
write_interval: auto
checkpoint_interval: auto
# Sequential trainer
# https://skrl.readthedocs.io/en/latest/api/trainers/sequential.html
trainer:
class: SequentialTrainer
timesteps: 72000
environment_info: log
在完成前面的注册机器人训练环境的所有操作后，进入 isaaclab 目录，在终端执行训练语句：
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Flat-OP3-train --max_iterations 3000 --seed 42 --headless
这里的训练框架为 rsl-rl，你也可以使用 isaaclab/scripts/reinforcement_learning/... 路径下的其他训练框架，显然这里的训练环境就是前面注册的 Flat-OP3-train，你也可以使用复杂地面环境 Rough-OP3-train。
此外，关于训练时填入的参数说明可以通过下面语句获取：
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Flat-OP3-train -h
usage: train.py [-h] [--video] [--video_length VIDEO_LENGTH] [--video_interval VIDEO_INTERVAL] [--num_envs NUM_ENVS] [--task TASK] [--seed SEED] [--max_iterations MAX_ITERATIONS] [--distributed] [--experiment_name EXPERIMENT_NAME] [--run_name RUN_NAME] [--resume RESUME] [--load_run LOAD_RUN] [--checkpoint CHECKPOINT] [--logger {neptune,tensorboard,wandb}] [--log_project_name LOG_PROJECT_NAME] [--headless] [--livestream {0,1,2}] [--enable_cameras] [--device DEVICE] [--verbose] [--info] [--experience EXPERIENCE] [--kit_args KIT_ARGS] Train an RL agent with RSL-RL.
options:
-h, --help show this help message and exit
--video Record videos during training.
--video_length VIDEO_LENGTH Length of the recorded video (in steps).
--video_interval VIDEO_INTERVAL Interval between video recordings (in steps).
--num_envs NUM_ENVS Number of environments to simulate.
--task TASK Name of the task.
--seed SEED Seed used for the environment
--max_iterations MAX_ITERATIONS RL Policy training iterations.
--distributed Run training with multiple GPUs or nodes.
rsl_rl: Arguments for RSL-RL agent.
--experiment_name EXPERIMENT_NAME Name of the experiment folder where logs will be stored.
--run_name RUN_NAME Run name suffix to the log directory.
--resume RESUME Whether to resume from a checkpoint.
--load_run LOAD_RUN Name of the run folder to resume from.
--checkpoint CHECKPOINT Checkpoint file to resume from.
--logger {neptune,tensorboard,wandb} Logger module to use.
--log_project_name LOG_PROJECT_NAME Name of the logging project when using wandb or neptune.
app_launcher arguments: Arguments for the AppLauncher. For more details, please check the documentation.
--headless Force display off at all times.
--livestream {0,1,2} Force enable livestreaming. Mapping corresponds to that for the `LIVESTREAM` environment variable.
--enable_cameras Enable camera sensors and relevant extension dependencies.
--device DEVICE The device to run the simulation on. Can be "cpu", "cuda", "cuda:N", where N is the device ID
--verbose Enable verbose-level log output from the SimulationApp.
--info Enable info-level log output from the SimulationApp.
--experience EXPERIENCE The experience file to load when launching the SimulationApp. If a empty string is provided, the experience file is determined based on the headless flag. If a relative path is provided, it is resolved relative to the `apps` folder in Isaac Sim and Isaac Lab (in that order).
--kit_args KIT_ARGS Command line arguments for Omniverse Kit as a string separated by a space delimiter. Example usage: --kit_args "--ext-folder=/path/to/ext1 --ext-folder=/path/to/ext2"
在完成训练后，你会发现在 ~/isaaclab/logs/rsl_rl/OP3_flat 路径下有很多 .pt 文件，这些文件就是训练过程中保存的强化学习网络模型参数文件，你可以使用这些参数文件来测试机器人的行走能力。
比如，我前面训练了 3000 轮，那我就选取最后一次获取的 .pt 文件 ~/model_2999.pt，在终端执行下面的测试语句：
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/play.py --task Flat-OP3-Play --checkpoint /home/suiyi/isaacsim/isaaclab/logs/rsl_rl/OP3_flat/2025-06-05_19-31-40/model_2999.pt
微信公众号「极客日志」，在微信中扫描左侧二维码关注。展示文案：极客日志 zeeklog
使用加密算法（如AES、TripleDES、Rabbit或RC4）加密和解密文本明文。 在线工具，加密/解密文本在线工具，online
生成新的随机RSA私钥和公钥pem证书。 在线工具，RSA密钥对生成器在线工具，online
基于 Mermaid.js 实时预览流程图、时序图等图表，支持源码编辑与即时渲染。 在线工具，Mermaid 预览与可视化编辑在线工具，online
解析常见 curl 参数并生成 fetch、axios、PHP curl 或 Python requests 示例代码。 在线工具，curl 转代码在线工具，online
将字符串编码和解码为其 Base64 格式表示形式即可。 在线工具，Base64 字符串编码/解码在线工具，online
将字符串、文件或图像转换为其 Base64 表示形式。 在线工具，Base64 文件转换器在线工具，online