Runway, the video-focused artificial intelligence (AI) firm, introduced a new video generation model on Monday. Dubbed Gen-4, it is an image-to-video generation model which succeeds the company’s Gen-3 Alpha AI model. It comes with several improvements, including consistency in characters, locations, and objects across scenes, as well as controllable real-world physics. Runway claims that the new AI model also comes with higher prompt adherence, and it can retain the style, mood, and cinematic elements of the scene with simple commands.
Runway Introduces Gen-4 Image-to-Video Generation Model
In a post on X (formerly known as Twitter), the official handle of Runway announced the release of the new video model. Gen-4 is currently rolling out to the company’s pair tiers as well as enterprise clients. There is no word on when it might be available to the free tier. “Gen-4 is a significant step forward for fidelity, dynamic motion and controllability in generative media,” the post added.
The successor to the Gen-3 Alpha model comes with several enhancements to offer image and video generation with consistent styles, subjects, locations, and more. The company also posted several short films made entirely using the Gen-4 video generation model.
In a blog post, the company detailed the new capabilities. Runway says that, with just one reference image, the AI model can generate consistent characters across different lighting conditions, locations, and camera angles. The same is said to be true for objects. Users can provide a reference image of an object, and it can be placed in any location or condition while ensuring consistency. Runway says this enables users to generate videos for narrative-based content and product shoots using the same image reference.
By providing a text description alongside the reference image, the AI model can generate a scene from different angles, including close-ups and wide-angle side profiles, capturing even the details missing in the reference. Another area where the company claims Gen-4 excels is understanding of real-world physics and motion.
When subjects in a video interact with the environment, the AI model ensures that real-world physics and realistic motion are added. This was also seen in the demonstration videos shared by the company, where water makes a realistic splash and moving bushes generate a lifelike movement.
The company, however, did not reveal the dataset used to train the AI model for the dynamic and high-fidelity outputs. This is interesting, given that the company is currently facing a lawsuit against artists and rival generative AI companies that claim that Runway trains its models on copyrighted material without permission.
