Google’s Genie 4 Builds Interactive Worlds With Sound — and Testers Are Using It for Film

October 28 — Google DeepMind on Tuesday launched its latest world model, Genie 4, currently in research preview. The model can generate the most realistic interactive virtual worlds yet, and users can type prompts during a simulation to change the scene. It also generates real-time audio synchronised with the virtual world, covering music, ambient sound and interactive dialogue.

Genie 4 arrives months after the AI video startup Runway released its second world model, GWM Worlds 2, an interactive model that responds dynamically to user input and also supports audio generation. At the same time the world-model field is starting to consolidate: World Labs, founded by Fei-Fei Li, was acquired by AMD, and Amazon is in talks to buy Decart.

Launching Genie 4 is Google’s latest move to stay ahead in a fiercely competitive field.

More than a year ago Google released the previous model, Genie 3, also as a research preview. Besides a big jump in visual realism and the new audio capability, Genie 4 supports free-form editing — adjusting the weather or camera angle in real time, or adding people and objects. It can keep a scene coherent for up to two minutes, twice as long as the previous model, with better physics and more lifelike virtual characters.

Integration with Google’s Gemini model has also deepened: Gemini acts as the director, planning the steps and checking frames, while Genie 4 generates the visual content. In one experiment tied to the Gemini robotics project, the Gemini 4 Argon model broke a task into steps and Genie 4 simulated it in real time. The experiment demonstrated a workable approach: validating scenarios in simulation before running robot tests in the real world.

“We will increasingly see these hybrid world models, where Gemini writes code and text, Genie 4 reads that in real time and simulates reality at different levels of abstraction,” Google research director Tejas Kulkarni said at a briefing.

Like other world models, its main commercial scenarios include games, interactive learning and robotics. One imagined case for interactive learning: a user steps into a painting of a historical scene and talks to the people in it about the events recorded there.

But “trusted testers” — the first users able to try Google’s frontier products — found a potential new use: film prototyping.

“The value of real-time is most obvious in situations where you need to maintain a coherent creative thread — brainstorming, storyboarding, all the stages that need ideas to come quickly,” Google DeepMind engineer Diego Rivas said at a media briefing.

Google has also released Veo, an AI video generation model that turns text, image or video prompts into high-definition clips with synchronised audio, and which is steadily gaining popularity among film and television creators.

Rivas added that users may in future use Gemini Omni, a multimodal model, to turn prototype material into production assets, with Gemini generating video from images, audio and text. But he noted that these capabilities are still some way from shipping.

Throughout the briefing Rivas stressed that Genie 4 is still early and that what the final product will feel like is undecided. The team also acknowledged weaknesses: the longer a simulation runs, the more details and objects drift, and the team has not found a reliable way to keep audio accurate while also making text in the virtual world legible.

Genie 4 is now available as a research preview to trusted testers. Over the coming weeks it will roll out to Google AI Ultra subscribers and to the Genie project — Google’s tool for generating interactive virtual worlds from text or image prompts — and early partners using the Gemini API and Google AI Studio will also get access.