Meta’s research group has published InteractiveWorld, an open-source simulator where one AI plays a person walking around a house and another plays the wearable assistant watching through that person’s eyes. The code sits in the facebookresearch repository on GitHub, alongside a paper whose first author is Qinsi Wang. Assistants that live in glasses will be judged on what happens in the room, and this is a way to test that before anyone wears one.
The loop is simple to describe. The assistant gets nothing but the wearer’s view through their own eyes, plus the conversation. It gives guidance, a second model acts as the human and turns that guidance into simulator commands, and the 3D world answers with the next view. An animated body walks, grabs, opens, pours, cuts, sits and picks up the phone across 44 homes and venues taken from Stanford’s BEHAVIOR-1K library. The README counts 36 families of action.
Scoring is what separates it from a chat benchmark. A task counts as passed once the finished world satisfies each stated goal, so a can of soda is on the coffee table or it is not. Nobody grades the assistant’s prose.
The bundled test, InteractiveBench, freezes 120 tasks into six categories of 20 apiece: Finding, Guidance, Planning, Decision, Accessibility and Prevention. Any model can sit in the assistant’s chair, whether a local open-source one served through vLLM or Ollama or a hosted one from OpenAI, Anthropic or Google. According to the repository, the paper used Gemini 3.1 Pro as the human for every assistant it tested. The README publishes no leaderboard scores, so how current models fare is a question the paper has to answer.
Running it is not a laptop job. The team tests on an Nvidia A100 with 80 GB of memory, a single simulator worker needs roughly 4 GB on the GPU, and the install needs around 100 GB of disk. Setup bakes 6,831 household objects into usable assets, a job the README puts at two minutes when eight workers share it. Teams can also write their own tasks as a single JSON file, and the release tree is checked for tampering before every benchmark run so scores stay comparable.
The license limits who benefits. InteractiveWorld is released under CC BY-NC 4.0, and the human body model and walking-motion data come with their own non-commercial research terms and require registration. Academic labs can build on it freely. A company shipping a glasses product would need to read those terms closely before using it for anything beyond looking.
Any startup pitching a glasses assistant can now be asked for a score on those 120 tasks, and that answer will say more than a polished demo video.
Reported from the facebookresearch InteractiveWorld repository on GitHub; the page carries no publication date.
