New Open-Source Project Lets Snap Spectacles AR Headsets Control Real Robots in Real-Time
Summary
A new open-source project called Spectacles Dimensional OS now lets users control real Unitree robots in real-time using Snap Spectacles AR headsets, featuring manual spatial navigation and an AI-powered voice command mode backed by OpenAI.
Key Points
- A new open-source project called Spectacles Dimensional OS enables Snap Spectacles AR headsets to control and visualize Unitree GO2 and G1 robots in real-time via a WebSocket bridge connected to Dimensional OS, a robot operating system.
- Two spatial control modes are available: Manual Mode, which lets users physically aim a direction marker to navigate the robot, and Agent Mode, currently in development, which accepts voice commands processed by an OpenAI-powered AI agent to autonomously direct the robot.
- The system uses AprilTag-based spatial registration to align the AR world frame with the robot's physical position, supports LiDAR obstacle visualization, and communicates over a versioned JSON WebSocket protocol running on port 8787 across a shared local WiFi network.