U0env 🐟

Underwater Robot Simulation, Data Collection, and Evaluation Framework

U0env is a simulation and evaluation framework for underwater tasks, built on the Stonefish physics engine and ROS Noetic. It supports multi-task data collection, LeRobot dataset conversion, and closed-loop evaluation with Vision-Language-Action (VLA) models.

✨ Features

• 🌊 Realistic underwater simulation based on Stonefish physics engine
• 🤖 BlueROV2 + Alpha5 manipulator + Robotiq gripper integration
• 📊 20+ underwater manipulation tasks (grasping, navigation, scanning, etc.)
• 📦 Automated data collection with batch processing and parallel execution
• 🔄 One-click conversion to LeRobot dataset format
• 🧪 Closed-loop evaluation with configurable metrics
• 🎯 MoveIt-based motion planning support

📁 Project Structure

u0env/
├── cpp_env/                    # C++ simulation environment (Stonefish)
├── ros_ws/                     # ROS workspace
│   └── src/
│       ├── stonefish_ros/      # Stonefish ROS interface (submodule)
│       ├── stonefish_bluerov2/ # BlueROV2 simulation package
│       ├── description_alpha/  # Alpha5 manipulator descriptions
│       └── description_robotiq/# Robotiq gripper descriptions
├── tools/
│   ├── dataprocess/            # Data processing & conversion scripts
│   ├── datavisualize/          # Visualization utilities
│   └── evalmetrics/            # Evaluation metrics
├── Stonefish/                  # Stonefish physics engine (submodule)
├── batch_run.sh                # Unified batch experiment script
├── build_all.sh                # Build script
└── exp_setting.csv             # Task configuration

🚀 Getting Started

This project depends on Stonefish and its ROS interface stonefish_ros. Please follow the steps below to complete the environment setup and compilation.

1. Initialize Submodules

First, after cloning this repository, you need to initialize and update the Stonefish, stonefish_ros and lerobot submodules:

git submodule update --init Stonefish
git submodule update --init ros_ws/src/stonefish_ros
git submodule update --init tools/lerobot

2. Create Environment and Install Dependencies

Install Miniconda or Miniforge, and install robostack-noetic:

# To avoid accidental violation of the Anaconda ToS, remove the defaults channel:
conda config --remove channels defaults

## Optional
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
conda config --set channel_priority strict

# Create a ros-noetic desktop environment
conda create -n ros_env -c conda-forge -c robostack-noetic ros-noetic-desktop
# Activate the environment
conda activate ros_env
# Add the robostack channel to the environment
conda config --env --add channels robostack-noetic
# Installing tools for local development
conda install -c conda-forge ros-dev-tools

# Test
# First terminal
conda activate ros_env
roscore
# Second terminal
conda activate ros_env
rviz

Install stonefish, stonefish_ros and project dependencies:

conda activate ros_env
conda install -c conda-forge glm sdl2 freetype
conda install -c robostack-noetic -c conda-forge \
  ros-noetic-ros-control \
  ros-noetic-perception \
  ros-noetic-moveit
pip install trimesh fast-simplification pandas matplotlib

3. Generate Alpha5 Description Files

The Reach Alpha5 manipulator description files are derived from the proprietary alpha_description package. You need to download it first and then run the generation script.

3.1 Download alpha_description

First, obtain the relevant license from Reach Robotics, then download alpha which contains the alpha_description package, and place it at:

ros_ws/src/description_alpha/origin/alpha_description/

The directory should contain: config/, meshes/, xacro/, CMakeLists.txt, package.xml, LICENSE.

3.2 Run the generation script

conda activate ros_env
python tools/dataprocess/pkg_install.py

This will generate:

• ros_ws/src/description_alpha/config/ - Controller configurations
• ros_ws/src/description_alpha/meshes/ - STL mesh files
• ros_ws/src/description_alpha/xacro/ - Xacro description files
• ros_ws/src/stonefish_bluerov2/data/alpha5/ - OBJ mesh files for simulation

3.3 Generate URDF file

After running the script above, generate the URDF file manually:

conda activate ros_env
export ROS_PACKAGE_PATH=<project_root>/ros_ws/src:$ROS_PACKAGE_PATH
mkdir -p ros_ws/src/description_alpha/urdf
xacro ros_ws/src/description_alpha/xacro/alpha.config.xacro \
  use_fake_hardware:=true use_sim:=false \
  > ros_ws/src/description_alpha/urdf/alpha_robotiq_hand-e.urdf

Note: Replace <project_root> with the actual path to the u0env project root.

4. Build the Project and Testing

Change directory to the project directory and run:

./build_all.sh

Use the following commands to run the simulation environment in a terminal with graphical interface support (e.g., MobaXterm):

Optional test: run simulation without ROS

cd build/mysim_build
./FishSim

Optional test: run simulation through ROS

conda activate ros_env
cd ros_ws
source devel/setup.bash
roslaunch stonefish_bluerov2 cpp_env.launch

📊 Data Collection

Collect pkl Data

Task configurations are defined in exp_setting.csv. Run data collection with the unified batch script:

conda activate ros_env
cd ros_ws
source devel/setup.bash
cd ..

# Collect training set for all tasks
./batch_run.sh --mode collect --split train

# Collect test set for all tasks
./batch_run.sh --mode collect --split test

# Collect a specific task using task_code in exp_setting.csv, e.g., pick_pipe0_shallow
./batch_run.sh --mode collect --split train --task pick_pipe0_shallow

# Debug mode (only 1 episode per task)
./batch_run.sh --mode collect --split train --debug

# Run with 4 parallel (better less than 5) for all modes (collect/eval)
./batch_run.sh --mode collect --split train --parallel 4

For more options, run ./batch_run.sh --help.

Convert to LeRobot Dataset Format

Create a separate conda environment and install lerobot:

conda deactivate
cd tools/lerobot/
conda create -y -n lerobot python=3.10
conda activate lerobot
pip install -e .
pip install scipy

Convert the collected data to LeRobot dataset format:

python tools/dataprocess/data_process.py --src /path/to/dataset/origin --dst /path/to/dataset/lerobot

For more details, run:

python tools/dataprocess/data_process.py --help

🧪 Evaluation

Run Evaluation with Automatic Recording

First, start the VLA model inference service. Refer to section "7. Inference" in the u0model. Parallel evaluation requires launching the corresponding number of VLA models; with default non-parallel mode, only a single model needs to be started.

Run evaluation:

conda activate ros_env
cd ros_ws
source devel/setup.bash
cd ..

# Evaluate all tasks
./batch_run.sh --mode eval

# Evaluate a specific task with 4 parallel episodes
./batch_run.sh --mode eval --task pick_pipe0_shallow --parallel 4

# Debug mode (only 1 episode per task)
./batch_run.sh --mode eval --debug

For more options, run ./batch_run.sh --help.

Compute Evaluation Metrics

# Compute metrics for all tasks
python tools/evalmetrics/calculate_metrics.py --eval-dir ./dataset/eval --no-visualize

# Compute metrics with visualization
python tools/evalmetrics/calculate_metrics.py --eval-dir ./dataset/eval

# Compute metrics for specific tasks
python tools/evalmetrics/calculate_metrics.py --eval-dir ./dataset/eval --task-codes pick_pipe0_shallow scan_ship_ancient

Results are saved to <eval-dir>/metrics/ as CSV and JSON.

📄 License

This project is licensed under the Apache License 2.0.

Note: The alpha_description package (including mesh files, URDF/xacro descriptions, and configuration files) is proprietary software owned by Reach Robotics and is not covered by the Apache 2.0 license of this project. Users must independently obtain the appropriate license from Reach Robotics before downloading or using those materials. The generated derivative files (e.g., converted OBJ meshes) are also subject to the original license terms.

⚠️ Disclaimer

This is an independent research project and is not affiliated with, endorsed by, or maintained by Reach Robotics. All trademarks and product names mentioned in this project (including but not limited to Alpha5, BlueROV2) are the property of their respective owners. Please refer to the Alpha repository for software and documentation.

📝 Citation

If you use this work in your research, please cite:

@misc{gu2025usimu0visionlanguageactiondataset,
      title={USIM and U0: A Vision-Language-Action Dataset and Model for General Underwater Robots},
      author={Junwen Gu and Zhiheng Wu and Pengxuan Si and Shuang Qiu and Yukai Feng and Luoyang Sun and Laien Luo and Lianyi Yu and Jian Wang and Zhengxing Wu},
      year={2025},
      eprint={2510.07869},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2510.07869},
}

🙏 Acknowledgments

• Stonefish — Underwater physics simulation engine
• stonefish_ros — ROS interface for Stonefish
• LeRobot — Hugging Face LeRobot framework
• RoboStack — Conda-based ROS distribution