unitree-g1/phases/phase-3-groot-wbc-deployment.md

Phase 3: GR00T-WBC Deployment on GB10

Date: 2026-02-14 Goal: Deploy and validate GR00T-WBC whole-body control framework on Dell Pro Max GB10 for G1 humanoid simulation

---

Summary

Successfully deployed NVIDIA's GR00T-WholeBodyControl on the GB10 (aarch64, Ubuntu 24.04, Blackwell GPU). Identified and fixed 4 critical blockers. Achieved working simulation with walking robot controlled via keyboard. Established remote visualization via NoMachine and web-based MJPEG streaming.

Key Achievements

1. GR00T-WBC running in simulation — Both Balance and Walk ONNX policies loaded and executing at 50 Hz
2. Walking robot — WASD keyboard control verified: forward, backward, strafe, rotate
3. Performance validated — 3.5 ms/iteration (17.5% of 20ms budget) in sync mode
4. Remote visualization — NoMachine virtual desktop + web MJPEG viewer
5. 4 critical bugs identified and fixed on aarch64 platform

Critical Fixes

Fix 1: CycloneDDS aarch64 Buffer Overflow (ROOT CAUSE)

• File: unitree_sdk2py/core/channel_config.py
• Problem: <Tracing> XML section triggers glibc FORTIFY_SOURCE buffer overflow specifically on ARM64
• Discovery: Isolated via Domain(0) (works) vs Domain(0, config_with_tracing) (crashes) vs Domain(0, config_without_tracing) (works)
• Fix: Remove <Tracing> section from both ChannelConfigHasInterface and ChannelConfigAutoDetermine
• Impact: Without this fix, nothing in the Unitree SDK works on aarch64

Fix 2: ROS2 Python Package Path

• File: .venv/lib/python3.12/site-packages/ros2.pth (created)
• Problem: ROS2 packages at /opt/ros/jazzy/lib/python3.12/site-packages/ not visible in venv
• Fix: Create .pth file with that path
• Impact: sensor_msgs, std_msgs, rclpy all accessible

Fix 3: ROS2 Shared Library Path

• Problem: rclpy native libraries (librcl_action.so etc.) not found
• Fix: export LD_LIBRARY_PATH=/opt/ros/jazzy/lib:$LD_LIBRARY_PATH

Fix 4: Sync Mode Sim Thread Check

• File: run_g1_control_loop.py line 211
• Problem: Code checks env.sim.sim_thread.is_alive() but in sync mode no thread exists
• Fix: Add not config.sim_sync_mode guard to the check

Architecture Findings

Pre-trained ONNX Policies

• Balance: 516-dim obs → 15-dim action (lower body joint targets)
• Walk: Same dimensions, velocity-conditioned locomotion
• Training: PPO in Isaac Lab, domain randomization, zero-shot sim-to-real (exact recipe not published)
• Not mocap-based: Command-conditioned (velocity tracking), not motion imitation

Keyboard Control Architecture

• KeyboardDispatcher (raw) uses sshkeyboard library — fails in remote terminals
• ROSKeyboardDispatcher subscribes to /keyboard_input topic — works via separate publisher
• Dispatchers call handle_keyboard_button() on registered listeners (policy, env, e-stop)
• Walk policy: ]=enable, w/s=fwd/back (+/-0.2 m/s per press), a/d=strafe, q/e=rotate, z=zero

Visualization Challenges (Headless GB10)

Attempted 5 approaches for remote visualization:

Approach	Result	Issue
x11vnc on physical display :0	Black screen	No monitor connected, GPU output disabled
x11vnc on Xvfb :99	Static frame	OpenGL renders to own framebuffer, x11vnc reads X11 pixmap
NoMachine physical shadow	Black screen	Shadowing black physical display
NoMachine virtual desktop	Works but GLFW freezes	GLFW passive viewer thread stalls after seconds
Web MJPEG streaming	Best solution	Offscreen renderer → JPEG → HTTP, no X11 dependency

DDS/ROS2 Coexistence

• CycloneDDS system (0.10.4) and ROS2 (0.10.5) are ABI-incompatible
• FastRTPS is the default RMW in ROS2 Jazzy — avoids CycloneDDS version conflict
• Second ChannelFactory init from BaseSimulator fails gracefully (try/except)
• CYCLONEDDS_URI environment variable controls Unitree SDK's DDS config

Software Installed on GB10

Package	Purpose
GR00T-WBC	Whole-body control framework
NoMachine 9.3.7	Remote desktop (NX protocol)
Sunshine 2025.924	Game streaming (NVENC) — installed but Moonlight client unstable
x11vnc	VNC server — limited use (can't capture OpenGL)
Xvfb	Virtual framebuffer for headless rendering
openbox	Lightweight window manager for Xvfb

Files Created/Modified on GB10

File	Action	Purpose
channel_config.py	Modified	Removed Tracing XML (buffer overflow fix)
.venv/.../ros2.pth	Created	ROS2 Python path
run_g1_control_loop.py	Modified	Sync mode sim thread guard
launch_sim.sh	Created	Convenience launcher
launch_with_web_viewer.py	Created	Launcher with web MJPEG streaming
web_viewer.py	Created	MuJoCo offscreen → MJPEG HTTP server
/tmp/keysender.py	Created	ROS keyboard publisher for remote control
/etc/gdm3/custom.conf	Modified	Auto-login enabled
/etc/x11vnc.pass	Modified	VNC password set
Firewall (ufw)	Modified	Opened ports 5900, 4000, 47984-47990, 8080

Next Steps

1. Get web viewer working — test launch_with_web_viewer.py for smooth browser-based visualization
2. Connect mocap — webcam → pose estimation → upper body teleop commands
3. Network bridge to G1 — connect GB10 to robot's 192.168.123.0/24 subnet
4. Deploy to real robot — switch --interface sim to --interface real
5. Push recovery testing — benchmark stock vs. GR00T-WBC push robustness
6. Custom RL training — use Isaac Lab to train custom policies with perturbation curriculum