Reads server host/port from start screen inputs and auto-connects
if not already connected when entering AR mode.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Server: reads all 3 cameras from Isaac Sim SHM, encodes JPEG
(head Q85, wrists Q70), sends with 1-byte camera ID prefix.
Head at ~10fps, wrist cameras at ~5fps for bandwidth control.
Godot: teleop_client parses camera ID byte and emits per-camera
signals. Three webcam quads arranged in a row (left|head|right)
in front of the user in AR mode.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Hand tracking laser pointer with pinch-to-click gesture for menu interaction
- Meta Quest passthrough (alpha blend + XR_FB_PASSTHROUGH extension)
- Stationary webcam quad (moved from XRCamera3D to XROrigin3D)
- Gaze-activated exit balls (4 balls: exit AR, 2 reserved, quit app)
- Gaze laser beam and ball growth visual feedback
- Spinning G1 3D model flanking the start screen menu
- Recenter support via pose_recentered signal
- Exit AR mode returns to CONFIG phase with UI repositioning
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Two-phase startup: CONFIG phase (VR start screen) -> AR phase (passthrough)
- Start screen with server host/port input, connect button, and launch AR button
- VR UI pointer with controller ray-pointing and hand poke interaction
- Body tracking visualization with colored spheres (spine, head, arms)
- Hand tracking with camera-based tracking (no controllers required)
- Android flavor manifest for Quest hand/body tracking permissions
- Upgrade OpenXR Vendors plugin from v3.1.2 to v4.3.0-stable
- Export presets updated for Godot 4.6.1 Meta plugin format
- Build scripts for Godot 4.6.1 with vendor plugin setup
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Godot 4.3 project for Quest 3 that replaces the browser-based WebXR
teleoperation with a native app using XR_FB_body_tracking (70 joints).
Key advantage: chest-relative wrist tracking eliminates head-position
subtraction artifacts and decouples body rotation from arm control.
Godot app: body_tracker.gd, teleop_client.gd, webcam_display.gd
Robot server: teleop_server.py (WebSocket, replaces Vuer ~900→~250 lines)
Drop-in wrapper: native_tv_wrapper.py (compatible with teleop_hand_and_arm.py)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
melancholytron
melancholytron
Joe DiPrima