NVIDIA Launches Open Source XR AI Beta, Bringing Real-Time Visual and Voice Intelligence to AR Glasses and XR Headsets
Summary
NVIDIA launches an open source XR AI beta platform enabling developers to build real-time visual and voice AI agents for AR glasses and XR headsets, powered by GPU-accelerated services including Cosmos, Nemotron, and NeMo Agent Toolkit.
Key Points
- NVIDIA XR AI is now publicly available in beta as an open source library that enables developers to build intelligent agents for AR glasses and XR headsets, connecting devices to GPU-accelerated AI services for real-time visual and voice interaction.
- The platform uses a modular architecture combining NVIDIA Cosmos for visual grounding, Nemotron models for language reasoning, Model Context Protocol for enterprise data connectivity, and frameworks like NVIDIA NeMo Agent Toolkit for agent orchestration, supporting flexible multi-user and multi-agent deployments.
- Developers can get started by cloning the public beta repository, running sample agents with live camera and microphone streams, integrating enterprise tools via MCP servers, and optionally adding NVIDIA CloudXR for rendered 3D spatial experiences.