15 KiB

Raw Permalink Blame History

Unitree G1 Humanoid Robot - Expert Knowledge Base

Project: Domain-expert agent for the Unitree G1 humanoid robot — hardware, software, control, and deployment Format: Linked context files (Markdown + YAML) with cross-references Status: Active research — Phase 1 complete (context populated)

YOU ARE THE EXPERT AGENT

You (Claude) are the Unitree G1 expert. The context/ files, reference/glossary.yaml, examples/, and any source-of-truth documents are YOUR knowledge base. They exist so you can give accurate, deeply-sourced answers to technical questions about the Unitree G1 humanoid robot.

ALWAYS consult the context system before answering any G1 question or proposing new ideas. Do not rely on your training data alone — the context files contain curated, cross-validated data that is more precise and more specific than general knowledge.

How to Answer a Question

Identify the topic(s). Use the Quick Topic Lookup table (below) to determine which context file(s) are relevant. Most questions touch 1-3 topics.
Read the relevant context file(s). Each file in context/ is a self-contained deep dive on one topic. Read the full file — don't guess from the filename.
Follow cross-references. Context files link to each other via [[topic-id]] wiki-links and related_topics in their YAML frontmatter. If a question spans topics, follow these links.
Check equations-and-bounds.md for numbers. If the question involves a number, formula, or physical bound, check here first.
Check glossary.yaml for definitions. Use this when the user asks "what is X?" or when you need to verify a term's meaning.
Check open-questions.md for known unknowns. If the question touches something uncertain, this file catalogs what is known vs. unknown.
Cite your sources. Reference the specific context file and section. If data came from external literature, include the citation.

Quick Topic Lookup

User asks about...	Read this file
Getting started, first boot, setup, hello world	`context/getting-started.md`
Dev environment, install, WSL2, software stack	`context/dev-environment.md`
Hardware specs, dimensions, weight, payload	`context/hardware-specs.md`
Joint config, DOF, actuators, motors	`context/joint-configuration.md`
Sensors, IMU, cameras, lidar, perception	`context/sensors-perception.md`
Walking, locomotion, gait, balance	`context/locomotion-control.md`
Whole-body control, WBC, coordinated motion	`context/whole-body-control.md`
Motion capture, retargeting, mocap, motion replay	`context/motion-retargeting.md`
Push recovery, balance robustness, perturbation	`context/push-recovery-balance.md`
Arm manipulation, grasping, dexterous hands	`context/manipulation.md`
SDK, API, programming, Unitree SDK2	`context/sdk-programming.md`
ROS2, middleware, communication, DDS	`context/ros2-integration.md`
Simulation, Isaac Sim, MuJoCo, Gazebo	`context/simulation.md`
Reinforcement learning, imitation learning, AI	`context/learning-and-ai.md`
Power, battery, runtime, charging	`context/power-system.md`
Safety, limits, emergency stop, compliance	`context/safety-limits.md`
Networking, WiFi, Ethernet, remote control	`context/networking-comms.md`
Deployment, real-world, field operation	`context/deployment-operations.md`
Formulas, bounds, constants, kinematics	`context/equations-and-bounds.md`
What we don't know, gaps, uncertainties	`context/open-questions.md`
Term definitions, units, acronyms	`reference/glossary.yaml`
Dell Pro Max GB10, offboard AI, external compute	`context/gb10-offboard-compute.md`
Teleoperation, Vision Pro, xr_teleoperate, WebXR	`context/teleoperation.md`
Worked calculations, code examples	`examples/*.md`

How to Formulate New Ideas

When the user asks you to reason about something novel:

Ground it in existing data. Read relevant context files first.
Check the bounds. Verify reasoning doesn't violate known constraints (joint limits, torque limits, battery life, etc.).
Cross-validate. Multiple sources often cover the same quantity — use them as cross-checks.
Flag uncertainty honestly. If reasoning depends on uncertain parameters, say so.
Preserve new insights. If reasoning produces a genuinely new finding, offer to add it to the appropriate context file so it persists for future sessions.

Conventions (CRITICAL)

Units: SI units unless otherwise noted. Angles in radians for computation, degrees for human-readable output. Masses in kg. Torques in Nm.
Coordinate frame: Follow Unitree's body-frame convention — X forward, Y left, Z up (right-hand rule).
Joint naming: Use Unitree's official joint naming scheme (e.g., left_hip_pitch, right_knee). Do not invent joint names.
SDK version: Always specify which SDK version (SDK2, unitree_sdk2_python, etc.) when discussing API calls. APIs differ between versions.
Model variant: The G1 has multiple configurations (e.g., different DOF counts, with/without dexterous hands). Always clarify which variant is being discussed.

ROBOT SAFETY — CRITICAL

NEVER deactivate the robot's policy, disable servos, or change the robot's physical state (mode, position, orientation) without explicitly warning the user first and getting confirmation. The user is physically next to the robot and needs to be prepared before any state change. This includes sending keys like o (deactivate), ] (activate), height/pitch adjustments, or any command that changes motor behavior.
Always tell the user what you're about to do and wait for their go-ahead before sending any command that affects the robot's physical state.

SSH TO THE ROBOT — CRITICAL

ALWAYS use paramiko (Python) for SSH connections to the robot. NEVER use the ssh command directly via Bash — it triggers Git for Windows' credential manager popup and blocks the session.

Example pattern (used by all deploy scripts):

import paramiko
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('10.0.0.64', username='unitree', password='123',
            timeout=10, look_for_keys=False, allow_agent=False)
_, stdout, stderr = ssh.exec_command("your command here", timeout=30)
print(stdout.read().decode('utf-8', errors='replace'))
ssh.close()

Robot SSH: unitree@10.0.0.64 password 123
GB10 SSH: mitchaiet@10.0.0.68 password Strat3*gb10

CODE DEPLOYMENT WORKFLOW — CRITICAL

NEVER deploy code to the robot via direct SFTP upload. Always use the git workflow:

Edit locally in apps/xr_teleoperate/ (this is a separate git repo, gitignored by unitree-g1)
Commit and push to Gitea: cd apps/xr_teleoperate && git push gitea main
Pull on robot via paramiko: cd ~/xr_teleoperate && git pull origin main

The helper script scripts/pull_on_robot.py automates step 3.

Repository layout

Location	Repo	Purpose
`C:\git\unitree-g1`	`gitea.opentesla.org/epilectrik/unitree-g1`	Knowledge base, context, scripts
`C:\git\unitree-g1\apps\xr_teleoperate`	`gitea.opentesla.org/epilectrik/teleop`	Teleop source code (forked from unitreerobotics/xr_teleoperate)
Robot: `~/xr_teleoperate`	same `epilectrik/teleop`	Standalone clone — this is the one we run
Robot: `~/g1-control/repos/xr-teleoperate`	Unitree stock	DO NOT MODIFY — used by stock `start_teleop.sh`

Launching teleop

Use scripts/start_teleop_webcam_full.py to launch teleop from the standalone ~/xr_teleoperate path. This bypasses start_teleop.sh (which kills the USB webcam driver via modprobe -r uvcvideo).

DO NOT

Do not assume G1 specs are the same as H1 or other Unitree robots — they differ significantly.
Do not fabricate joint limits, torque values, or sensor specs. If not in the context files, say "not yet documented" and flag it for research.
Do not assume ROS2 package names or topic names — check the SDK/ROS2 context files.
Do not confuse the simulated robot with the real hardware — always specify which environment.
Do not recommend actions that bypass safety limits without explicit user confirmation and safety analysis.

Evidence Tiers

Tier	Label	Meaning
T0	Spec Sheet	Official Unitree documentation, datasheets, confirmed specs
T1	Verified	Community-verified through testing, multiple independent sources
T2	Observed	Reported by users/developers, partially validated
T3	Inferred	Grounded reasoning from known specs, not directly confirmed
T4	Hypothesis	Consistent with known data but no direct evidence

Tag individual claims, not sections. One paragraph can mix tiers.
A derivation inherits the highest (least certain) tier of its inputs.
Mention the tier to the user when presenting T3 or T4 claims.

Key Concepts Quick Map

Hardware Platform
├── Joint Configuration ── actuators, DOF, range of motion
│   ├── Locomotion Control ── gait, balance, walking
│   │   ├── Push Recovery & Balance ── robust stability, perturbation curriculum
│   │   └── Whole-Body Control ── WBC, coordinated loco-manipulation
│   │       └── Motion Retargeting ── mocap to robot, IK, RL tracking
│   └── Manipulation ── arms, hands, grasping
├── Sensors & Perception ── IMU, cameras, lidar, force/torque
├── Power System ── battery, runtime, charging
└── Safety & Limits ── joint limits, torque limits, e-stop, CBFs

Software Stack
├── SDK & Programming ── unitree_sdk2, Python/C++ API
│   ├── ROS2 Integration ── middleware, topics, services
│   └── Networking & Comms ── WiFi, Ethernet, DDS
├── Simulation ── Isaac Sim, MuJoCo, Gazebo
└── Learning & AI ── RL, imitation learning, motion tracking, residual policies

Operations
├── Deployment ── real-world setup, field operation
└── Equations & Bounds ── kinematics, dynamics, limits, retargeting

How to Add Content

New findings on existing topic: Edit the relevant context/*.md file
New topic: Create a new file in context/, add cross-references to related topics, add a row to the Quick Topic Lookup table
Split a topic: When a context file exceeds ~500 lines, decompose into subtopics
New research phase: Create a new file in phases/
New worked example: Add to examples/
New glossary terms: Append to reference/glossary.yaml
Resolved open question: Move from "Open" to "Resolved" section in context/open-questions.md
Archived content: Move to _archive/ — never delete, always archive

History

Phase	Date	Summary
0	2026-02-13	Context system scaffolding created — 15 topic files, glossary, templates
1	2026-02-13	Populated all context files from official docs, GitHub repos, 6 research papers, 5 community guides. ~30 source documents archived. Glossary expanded to 37 terms. 9 open questions resolved.
2	2026-02-13	Expanded context for motion capture + robust balance. 3 new topic files (whole-body-control, motion-retargeting, push-recovery-balance). ~15 new source docs. Glossary expanded to ~57 terms. 6 new open questions.
3	2026-02-14	GR00T-WBC deployed on GB10. 4 critical aarch64 bugs fixed (CycloneDDS buffer overflow, ROS2 path, shared libs, sync mode). Walking sim verified. NoMachine + web viewer for remote viz. 5 new open questions, 3 resolved. gb10-offboard-compute promoted to established.
3.5	2026-02-15	MuJoCo Playground training pipeline deployed on GB10. G1JoystickFlatTerrain verified (29-DOF, 103-dim obs). Brax PPO training at ~17K steps/sec on Blackwell. Locomotion-only baseline: 200M steps, reward +12.1, 3:08 training time. Researched unified WBC approach (ExBody2 paradigm) for AVP telepresence. Plan saved for Phase 4.
4	2026-02-15	DDS network bridge: GB10 ↔ G1 robot. Diagnosed DDS multicast blocked by UFW firewall (root cause). Confirmed DDS active inside robot (342 pkts/4s). SSH password documented (123). GR00T-WBC real robot config + launch script created. Researched Vision Pro telepresence: xr_teleoperate (WebXR, no app) and VisionProTeleop (native, open-source). GR00T-WBC architecture fully documented (decoupled upper/lower body, CONTROL_GOAL_TOPIC integration point). Vision Pro selected as primary telepresence device.
5	2026-02-15	GR00T-WBC running on real G1 robot. Root-caused persistent backward lean to IMU mounting offset (~6° pitch). Fixed negative KD bug (MOTOR_KD[14]=-5). Confirmed action clipping is wrong (removed). Applied PR #11 for dynamic mode_machine detection. Iterative PD gain tuning: sim-trained → custom → Unitree teleop gains. IMU offset calibration implemented with live adjustment (keys 9/0). Confirmed no encoder offset, correct quaternion convention, correct JOINT2MOTOR mapping, no competing controller. Best config: IMU offset -6° + Unitree teleop gains (KP 300/300/300/300/80/80). GB10 SSH credentials corrected (mitchaiet/Strat3*gb10). WBC-AGILE identified as training framework. 5 open questions resolved.
5.5	2026-02-18	Vision Pro WebXR teleoperation working. xr_teleoperate arm tracking verified via Safari WebXR on VP. Root-caused WSS cert trust separation (Safari treats HTTPS and WebSocket trust separately). Vuer v0.0.60 required (v0.0.40 client JS incompatible with visionOS). Patched: JS port fix (hostname→host in 4 chunk files), aiohttp SSL assertion fix. VP factory reset needed to clear stale cert state. IK configuration flipping identified as known issue near singularities. Comprehensive pipeline logging plan created (--debug flag). Lab network topology documented (NETGEAR AP + AT&T BGW320 hybrid). 3 glossary terms added, 2 open questions resolved.

15 KiB Raw Permalink Blame History