This repository implements teleoperation of the **Unitree humanoid robot** using **Apple Vision Pro**.
This repository implements teleoperation of the **Unitree humanoid robot** using **XR Devices** ( such as Apple Vision Pro、 PICO 4 Ultra Enterprise or Meta Quest 3 ).
Here are the currently supported robots,
Here are the currently supported robots,
@ -75,13 +76,23 @@ Here are the currently supported robots,
<imgsrc="https://oss-global-cdn.unitree.com/static/9871e3bac4c54140b0839c68baf48a4a_1872x929.png"alt="Watch the Document"style="width: 75%;">
</a>
</a>
</p>
</p>
# 1. 📦 Prerequisites
# 1. 📦 Prerequisites
We tested our code on Ubuntu 20.04 and Ubuntu 22.04, other operating systems may be configured differently.
We tested our code on Ubuntu 20.04 and Ubuntu 22.04, other operating systems may be configured differently.
@ -125,7 +136,7 @@ In the Ubuntu system's `~/.bashrc` file, the default configuration is: `PS1='${d
# 2. ⚙️ TeleVision and Apple Vision Pro configuration
# 2. ⚙️ Configuration
## 2.1 📥 basic
## 2.1 📥 basic
@ -138,7 +149,9 @@ In the Ubuntu system's `~/.bashrc` file, the default configuration is: `PS1='${d
## 2.2 🔌 Local streaming
## 2.2 🔌 Local streaming
**Apple** does not allow WebXR on non-https connections. To test the application locally, we need to create a self-signed certificate and install it on the client. You need a ubuntu machine and a router. Connect the Apple Vision Pro and the ubuntu **Host machine** to the same router.
**2.2.1 Apple Vision Pro**
does not allow WebXR on non-https connections. To test the application locally, we need to create a self-signed certificate and install it on the client. You need a ubuntu machine and a router. Connect the Apple Vision Pro and the ubuntu **Host machine** to the same router.
We have tried using hand tracking for teleoperation on the PICO 4 Ultra Enterprise and Meta-Quest 3.
The system specifications of PICO 4 Ultra Enterprise:
> System Version: 5.12.6.U; Android version number: 14; Software version number: c000_cf01_bv1.0.1_sv5.12.6_202412121344_sparrow_b4244_user; browser version: [4.0.28 beta version](https://github.com/vuer-ai/vuer/issues/45#issuecomment-2674918619)
The system specifications of Meta-Quest 3:
> System version: 49829370066100510; Version: 62.0.0.273.343.570372087; Runtime version: 62.0.0.269.341.570372063; OS version: SQ3A.220605.009.A1.
For more configuration steps, please refer to the [issue](https://github.com/unitreerobotics/avp_teleoperate/issues/32).
## 2.3 🔎 Unit Test
## 2.3 🔎 Unit Test
This step is to verify that the environment is installed correctly.
This step is to verify that the environment is installed correctly.
@ -265,7 +294,7 @@ First, **Operator B** needs to perform the following steps:
1. Modify the `img_config` image client configuration under the `if __name__ == '__main__':` section in `~/avp_teleoperate/teleop/teleop_hand_and_arm.py`. It should match the image server parameters you configured on PC2 in Section 3.1.
1. Modify the `img_config` image client configuration under the `if __name__ == '__main__':` section in `~/avp_teleoperate/teleop/teleop_hand_and_arm.py`. It should match the image server parameters you configured on PC2 in Section 3.1.
2. Choose different launch parameters based on your robot configuration
2. Choose different launch parameters based on your robot configuration. Here are some example commands:
```bash
```bash
# 1. G1 (29DoF) Robot + Dex3-1 Dexterous Hand (Note: G1_29 is the default value for --arm, so it can be omitted)
# 1. G1 (29DoF) Robot + Dex3-1 Dexterous Hand (Note: G1_29 is the default value for --arm, so it can be omitted)
@ -273,12 +302,18 @@ First, **Operator B** needs to perform the following steps:
# 3. H1_2 Robot (Note: The first-generation Inspire Dexterous Hand is currently only supported in the H1_2 branch. Support for the Main branch will be added later.)
3. If the program starts successfully, the terminal will pause at the final line displaying the message: "Please enter the start signal (enter 'r' to start the subsequent program):"
3. If the program starts successfully, the terminal will pause at the final line displaying the message: "Please enter the start signal (enter 'r' to start the subsequent program):"
silencht
11 months ago