New video AI also makes music

Eight months after ChatGPT developer OpenAI introduced Sora, a pretty impressive text-to-video AI, Meta is now upping the ante. Like Sora, Movie Gen is an AI model that creates short videos from text instructions – but meta-AI can do even more.

Movie Gen: Meta’s video AI can do a lot

With Movie Gen, you can not only create new AI videos, but also edit existing moving image material afterwards. With simple instructions, scenes can be changed, for example new ones can be inserted or objects can be replaced.

Videos can also be created from simple photos, as The Verge writes. In a meta example, for example, a person depicted in a passport photo is shown as an acting person in a video.

AI tool generates suitable music

What’s even more exciting is that Movie Gen delivers appropriate sounds or music for the videos – which are also supposed to be AI-generated. Meta shows vehicles here with engine noises and rock music, exploding New Year’s Eve rockets or orchestral sounds to natural scenes.

Given the high video quality and the variety of editing options, the mood in the film and video industry in particular is anything but good. Because the examples presented look very realistic. There is also the option of having the appropriate accompanying music created.

Meta, Movie Gen’s video model offers 30 billion parameters, and its audio model offers 13 billion parameters. To put it into perspective: Meta’s Llama AI has 405 billion parameters.

So far, text-to-video AI can create videos up to 16 seconds long. The resolution is 768 by 768 pixels, but can be brought to HD quality (1,080p) via upsampling.

Market launch of Movie Gen uncertain

Metas Movie Gen cannot yet be tried out by normal users. The video AI is currently only available to internal developers and a few selected beta users.

And it will probably stay that way for a while. Meta product manager Chris Cox explained via Threads that the company is not yet ready to release the AI ​​tool. And it will stay that way for a while.

Reason: The whole thing is still expensive and it takes too long for the videos to be created. It is therefore unclear when Movie Gen will come. That shouldn’t happen any time soon. Because OpenAI’s Sora is also over half a year after its presentation and is not yet available.