--- id: g1-integration title: "Unitree G1 Robot Integration" status: proposed source_sections: "Cross-referenced from git/unitree-g1 context system + Dell Pro Max GB10 context" related_topics: [connectivity, ai-workloads, ai-frameworks, multi-unit-stacking] key_equations: [] key_terms: [unitree-g1, jetson-orin-nx, offboard-compute, cyclonedds, dds, teleoperation, isaac-lab, lerobot] images: [] examples: [] open_questions: - "DDS latency over Wi-Fi between GB10 and G1 under realistic conditions" - "Optimal LLM size for real-time task planning (latency vs. capability tradeoff)" - "Isaac Lab on GB10 vs. dedicated x86 workstation for RL training throughput" - "Docker container availability for CycloneDDS 0.10.2 on ARM64 DGX OS" - "Power-efficient always-on inference mode for persistent LLM service" --- # Unitree G1 Robot Integration Architecture and use cases for pairing the Dell Pro Max GB10 as an offboard AI companion to the Unitree G1 humanoid robot. ## 1. Why Integrate? The G1's development computer (Jetson Orin NX 16GB, 100 TOPS) is excellent for real-time control but severely limited for large-scale AI. The GB10 removes that ceiling. | Capability | G1 Orin NX | GB10 | Factor | |---|---|---|---| | AI compute | 100 TOPS | 1,000 TFLOPS (FP4) | ~10,000x | | Memory for models | 16 GB | 128 GB unified | 8x | | Max LLM (quantized) | ~7B | ~200B | ~30x | | CUDA architecture | sm_87 (Orin) | sm_121 (Blackwell) | 2 gens newer | | Storage | 32 GB eMMC (loco) | 2-4 TB NVMe | 60-125x | The GB10 acts as an **offboard AI brain** — handling reasoning, planning, training, and heavy inference that the Orin NX cannot. ## 2. Connection Architecture ``` ┌──────────────────────────┐ ┌───────────────────────────┐ │ Unitree G1 │ │ Dell Pro Max GB10 │ │ │ │ │ │ ┌─────────────────┐ │ │ 128 GB unified memory │ │ │ Locomotion CPU │ │ │ 1 PFLOP FP4 │ │ │ RK3588 │ │ │ DGX OS (Ubuntu 24.04) │ │ │ 192.168.123.161 │ │ │ │ │ │ 500 Hz control │ │ │ ┌──────────────────┐ │ │ └──────┬───────────┘ │ │ │ LLM Server │ │ │ │ CycloneDDS │ │ │ (llama.cpp/ │ │ │ ┌──────┴───────────┐ │ Wi-Fi 6/7 │ │ TensorRT-LLM/ │ │ │ │ Development CPU │◄──┼──────────────┼──►│ Ollama) │ │ │ │ Jetson Orin NX │ │ or 10GbE │ └──────────────────┘ │ │ │ 192.168.123.164 │ │ │ │ │ │ 100 TOPS │ │ │ ┌──────────────────┐ │ │ └──────────────────┘ │ │ │ Training Server │ │ │ │ │ │ (Isaac Lab / │ │ │ Sensors: │ │ │ PyTorch / │ │ │ - D435i (RGB-D) │ │ │ LeRobot) │ │ │ - MID360 (LiDAR) │ │ └──────────────────┘ │ │ - 6-axis IMU │ │ │ │ - 4-mic array │ │ ┌──────────────────┐ │ │ - Joint encoders (500Hz)│ │ │ VLM Server │ │ │ │ │ │ (vision-language) │ │ └──────────────────────────┘ └───────────────────────────┘ ``` ### Connection Options | Method | Bandwidth | Latency | Best For | |--------|-----------|---------|----------| | Wi-Fi (G1 Wi-Fi 6 ↔ GB10 Wi-Fi 7) | ~1 Gbps theoretical | 5-50 ms variable | Untethered operation, API calls | | 10GbE Ethernet (GB10 RJ45) | 10 Gbps | <1 ms | Lab/tethered, sensor streaming | | QSFP (via switch) | 200 Gbps | <1 ms | Multi-robot + GB10 cluster (T4) | **Recommended:** Wi-Fi for mobile operation; 10GbE for lab/training workflows where the robot is stationary or tethered. ### Network Configuration The G1 uses a fixed 192.168.123.0/24 subnet. The GB10 must join this subnet or use a router/bridge. **Option A — Direct subnet join (simplest):** ```bash # On GB10, configure 10GbE or Wi-Fi interface sudo nmcli con add type ethernet ifname eth0 con-name g1-net \ ip4 192.168.123.100/24 ``` **Option B — Dual-network (GB10 keeps internet access):** - GB10's 10GbE on 192.168.123.0/24 (to G1) - GB10's Wi-Fi on home network (internet access for model downloads, updates) **Option C — G1 joins GB10's network:** - Configure G1's Wi-Fi to connect to the same network as GB10 - G1's internal subnet (192.168.123.0/24) remains separate for internal DDS ## 3. Use Cases ### 3a. LLM-Based Task Planning The G1's `learning-and-ai.md` lists "LLM-based task planning integration (firmware v3.2+)" as preliminary. The GB10 can serve as the LLM backend. **Architecture:** ``` User (voice/text) → G1 mic array → STT on Orin NX → HTTP/gRPC to GB10 → LLM (Llama 70B+) generates plan → Plan sent back to G1 → G1 executes via locomotion policy + manipulation ``` **GB10 serves an OpenAI-compatible API** via llama.cpp or Ollama: ```bash # On GB10: start LLM server ./llama-server --model ~/models/llama-70b-q4.gguf \ --host 0.0.0.0 --port 30000 --n-gpu-layers 99 # On G1 Orin NX: call the API curl http://192.168.123.100:30000/v1/chat/completions \ -d '{"model":"llama","messages":[{"role":"user","content":"Walk to the kitchen table and pick up the red cup"}]}' ``` **Recommended models:** | Model | Size | Speed (est.) | Task Planning Quality | |-------|------|---------|----------------------| | Llama 3.2 3B | 3B | ~100 tok/s | Basic commands | | Nemotron-3-Nano 30B | 3B active | Fast | Good (built-in reasoning) | | Llama 3.1 70B (Q4) | 70B | ~15-20 tok/s | Strong | | Llama 3.1 70B (Q8) | 70B | ~10-15 tok/s | Strong, higher quality | For task planning, 1-5 second response time is acceptable — the robot doesn't need instant responses for high-level commands. ### 3b. Vision-Language Models Stream camera frames from the G1's Intel RealSense D435i to the GB10 for analysis by large VLMs that can't run on the Orin NX. **Use cases:** - Scene understanding ("describe what you see") - Object identification ("find the red cup on the table") - Spatial reasoning ("is the path ahead clear?") - Anomaly detection ("does anything look unusual?") **Data path:** ``` G1 D435i (1920x1080 @ 30fps) → RealSense SDK on Orin NX → JPEG compress → HTTP POST to GB10 → VLM inference → JSON response back to G1 ``` **Bandwidth:** A 1080p JPEG frame is ~100-500 KB. At 5 fps sampling: ~2.5 MB/s (~20 Mbps). Well within Wi-Fi capacity. ### 3c. RL Policy Training Train locomotion and manipulation policies on the GB10 using Isaac Lab or MuJoCo, then deploy to the G1. **Why GB10 over the Orin NX:** - Isaac Lab runs thousands of parallel environments — needs GPU memory - GB10's 128 GB unified memory holds large batches + simulation state - Blackwell GPU (6,144 CUDA cores, sm_121) accelerates physics simulation - CUDA 13.0 + PyTorch container `nvcr.io/nvidia/pytorch:25.11-py3` **Workflow:** ``` 1. GB10: Train policy in Isaac Lab (unitree_rl_lab, G1-29dof config) 2. GB10: Validate in MuJoCo (sim2sim cross-validation) 3. GB10 → G1: Transfer trained model file (.pt / .onnx) 4. G1 Orin NX: Deploy via unitree_sdk2_python (low-gain start) 5. G1: Gradual gain ramp-up with tethered safety testing ``` **Key: Both systems are ARM64.** Model files trained on GB10 (ARM) deploy directly to Orin NX (ARM) without architecture-related issues. ### 3d. Imitation Learning Data Processing Process teleoperation demonstration data collected on the G1 using larger models on the GB10. **Workflow:** ``` 1. G1: Collect demonstrations via XR teleoperation (Vision Pro / Quest 3) 2. Transfer data to GB10 (SCP over network) 3. GB10: Train behavior cloning / diffusion policy (LeRobot, 128GB memory) 4. GB10: Validate in simulation 5. Transfer model to G1 for deployment ``` The GB10's 128 GB memory enables training larger LeRobot policies and processing more demonstration episodes than the Orin NX's 16 GB allows. ### 3e. Multi-Modal Interaction Combine the G1's sensor suite with GB10's compute for rich interaction: ``` G1 Microphones (4-mic array) → Speech-to-Text (Orin NX: Whisper small) → Text to GB10 → LLM reasoning → Response text → TTS on Orin NX → G1 Speaker (5W stereo) G1 Camera (D435i) → Frames to GB10 → VLM → Situational awareness G1 LiDAR (MID360) → Point cloud to GB10 → Semantic mapping ``` ### 3f. Simulation Environment Run full physics simulation of the G1 on the GB10: - **MuJoCo:** CPU-based simulation for policy validation - **Isaac Lab:** GPU-accelerated parallel environments for training - **Isaac Gym:** Legacy GPU simulation (unitree_rl_gym) The GB10 can run these natively (ARM64 NVIDIA stack) while the G1 operates in the real world. Sim2Real transfer between GB10 simulation and real G1 hardware. ## 4. Latency Budget | Operation | Latency | Acceptable? | |-----------|---------|-------------| | G1 locomotion control loop | 2 ms (500 Hz) | N/A — stays on-robot | | DDS internal (loco ↔ dev) | 2 ms | N/A — stays on-robot | | Wi-Fi round-trip (G1 ↔ GB10) | 5-50 ms | Yes for planning, No for control | | 10GbE round-trip (G1 ↔ GB10) | <1 ms | Yes for most tasks | | LLM inference (70B Q4) | 1-5 seconds | Yes for task planning | | VLM inference (per frame) | 0.5-2 seconds | Yes for scene understanding | **Critical rule:** Real-time control (500 Hz locomotion, balance, joint commands) MUST stay on the G1's locomotion computer. The GB10 handles only high-level reasoning with relaxed latency requirements. ## 5. Communication Protocol Options ### Option A: REST/gRPC API (Simplest) GB10 runs an HTTP API server (llama.cpp, Ollama, or custom FastAPI). G1's Orin NX sends requests and receives responses. - **Pros:** Simple, stateless, easy to debug, OpenAI-compatible - **Cons:** Higher latency per request, no streaming sensor data - **Best for:** Task planning, VLM queries, one-shot inference ### Option B: CycloneDDS (Native G1 protocol) Install CycloneDDS 0.10.2 on the GB10, making it appear as another node on the G1's DDS network. - **Pros:** Native integration, low latency, pub/sub for streaming data - **Cons:** More complex setup, must match exact DDS version (0.10.2) - **Best for:** Streaming sensor data, continuous perception, tight integration ```bash # On GB10: Install CycloneDDS (must be 0.10.2 to match G1) pip install cyclonedds==0.10.2 # Subscribe to G1 state # (requires unitree_sdk2 IDL definitions) ``` ### Option C: ROS 2 Bridge Both systems support ROS 2 (G1 via unitree_ros2, GB10 via standard Ubuntu packages). - **Pros:** Rich ecosystem, visualization (RViz), bag recording - **Cons:** Additional overhead, ROS 2 setup complexity - **Best for:** Research workflows, multi-sensor fusion, SLAM ## 6. What NOT to Use the GB10 For - **Real-time joint control** — Too much latency over network. Locomotion stays on RK3588. - **Balance/stability** — 500 Hz control loop cannot tolerate network jitter. - **Safety-critical decisions** — Emergency stop must be on-robot, not dependent on network. - **Field deployment brain** — GB10 requires wall power (280W). It's a lab/indoor companion only. ## 7. Hardware Compatibility Notes | Aspect | G1 | GB10 | Compatible? | |--------|-----|------|-------------| | CPU architecture | ARM (A76/A55, Orin ARM) | ARM (Cortex-X925/A725) | Yes | | CUDA | sm_87 (Orin) | sm_121 (Blackwell) | Model files portable | | Linux | Custom (kernel 5.10 RT) | Ubuntu 24.04 (kernel 6.8) | Yes (userspace) | | Python | 3.x on Orin NX | 3.x on DGX OS | Yes | | DDS | CycloneDDS 0.10.2 | Not pre-installed (installable) | Yes (with setup) | | Docker | Available on Orin NX | Pre-installed + NVIDIA Runtime | Yes | | Network | Wi-Fi 6, Ethernet | Wi-Fi 7, 10GbE, QSFP | Yes | **ARM alignment is a significant advantage.** Both systems are ARM-native, eliminating cross-architecture issues with model files, shared libraries, and container images. ## 8. Cost Context | Component | Price | Role | |-----------|-------|------| | G1 EDU Standard | $43,500 | Robot platform | | G1 EDU Ultimate B | $73,900 | Robot + full DOF + tactile | | Dell Pro Max GB10 (2TB) | $3,699 | Offboard AI brain | | Dell Pro Max GB10 (4TB) | $3,999 | Offboard AI brain | The GB10 adds <10% to the cost of a G1 EDU while multiplying AI capability by orders of magnitude. Even at the $13,500 base G1 price, the GB10 is a ~27% add-on that transforms the AI capabilities. ## 9. Quick Start (Proposed) 1. **Network:** Connect GB10 to G1's subnet (Wi-Fi or Ethernet to 192.168.123.100) 2. **LLM server:** Start llama.cpp or Ollama on GB10 (port 30000 or 12000) 3. **Test connectivity:** `curl http://192.168.123.100:30000/v1/models` from G1 Orin NX 4. **First integration:** Simple Python script on Orin NX that sends voice-to-text to GB10 LLM, receives plan, prints it 5. **Iterate:** Add VLM, sensor streaming, RL training as needed ## Key Relationships - GB10 compute: [[gb10-superchip]], [[ai-frameworks]], [[ai-workloads]] - GB10 networking: [[connectivity]] - G1 reference: `git/unitree-g1/context/` (separate knowledge base)