Meta has announced the launch of Movie Gen, a GenAI model that’s capable of creating video content out of text prompts. Apart from that, it is also capable of adding objects to existing clips and even generating background music and sound effects for them. Meta claims that Movie Gen outperforms similar models in the industry, like OpenAI’s Sora, but the company hasn’t made it accessible to the public yet.
Meta calls the launch of Movie Gen part of its third wave of generative AI work, with the first one being its Make-A-Scene series of models meant for creating image, audio, video, and 3D animation and the second being Llama Image foundation models. Movie Gen combines the modalities of all previous AI models and enables fine-grained control for its users.
Movie Gen has four capabilities. It can generate realistic videos using text prompts. Videos of up to 16 seconds can be created at the rate of 16fps. The second way it can be used is for generating personalised videos, which means it can take a photograph of a person and add it to a clip that’s generated using a text prompt. Movie Gen is also capable of video editing of existing clips, which means changes such as background or style modifications can be made as well. Its fourth capability is audio generating. Based on text prompts, Movie Gen can create high-quality and high-fidelity audio of up to 45 seconds, including ambient sound, sound effects (Foley), and instrumental background music.
Meta’s statement announcing the launch of this tool reads, “As we continue to improve our models and move toward a potential future release, we’ll work closely with filmmakers and creators to integrate their feedback. By taking a collaborative approach, we want to ensure we’re creating tools that help people enhance their inherent creativity in new ways they may have never dreamed would be possible. Imagine animating a ‘day in the life’ video to share on Reels and editing it using text prompts, or creating a customized animated birthday greeting for a friend and sending it to them on WhatsApp. With creativity and self-expression taking charge, the possibilities are infinite.”
In February this year, OpenAI surprised the world with the launch of Sora, its own video generation platform. It can be used to generate realistic videos based on user prompts, creating scenes with multiple characters, emotions, and detailed environments. Like Meta’s Movie Gen, Sora too is not available to the public.