Categories

Five people smiling and laughing closely together, facing the camera with bright expressions.

AI video editing using Runway: Part Four

The video generation results are in...

We continue our series introducing one of the world’s most popular online generative AI video toolkits, as we create a short promotional video for a brand new immersive experience.


The AI tools and models used in this series

We’ll use runway.ml to generate video clips and assemble them into a sequence, and we’ll use additional AI tools to help with initial ideation and to create text prompts to use at runway to help generate high quality video clips.

The main steps in the project:

  1. Generate an overview for an ‘immersive experience’ promotional video using perplexity.ai
  2. Generate still images using the flux-1.1-pro model at replicate.com to use as ‘first frame’ prompts at runway
  3. Generate short video clips using runway’s Gen-4 model
  4. Use runway’s online video editor to sequence the promotional video
  5. Use the speech-02-hd AI model at replicate.com to generate a realistic-sounding voiceover
  6. Use udio.com to generate a background music track
  7. Add text titles and export the video file


Part 4: Generate additional video clips for the short promo video using runway

Below are the six ‘prompt image’ stills generated using flux pro 1.1 at replicate.com, and the six video clips generated using Runway Gen-4 (using each still image as a starting frame). These videos are as-downloaded direct from Runway – They have not been edited.

Clip 1: Still Image prompt

People wearing VR headsets walk through a room with colorful digital projections and futuristic graphics on the walls and ceiling.

Clip 1: Runway Gen-4 5 second video clip

Clip 2: Still Image prompt

A person reaches toward a glowing sphere in a vibrant, neon-lit cityscape surrounded by digital screens and people.

Clip 2: Runway Gen-4 5 second video clip

Clip 3: Still Image prompt

A crowd enjoys a music event with colorful, animated lights and musical notes filling the vibrant atmosphere.

Clip 3: Runway Gen-4 5 second video clip

Clip 4: Still Image prompt

Five people smiling and laughing closely together, facing the camera with bright expressions.

Clip 4: Runway Gen-4 5 second video clip

Clip 5: Still Image prompt

A digital sign displays "FU:7R" in red lights amid smoke and sparks.

Clip 5: Runway Gen-4 5 second video clip

Clip 6: Still Image prompt

The words "Reality Remixed" in illuminated marquee letters with burst light effects in the background.

Clip 6: Runway Gen-4 5 second video clip


Trying Runway’s upscaling option

As mentioned in part three of this series, a short clip can be upscaled from 720p resolution to 4k for an additional 10 credits. We won’t upscale every clip, but to see if this option is worthwhile we upscaled the ‘faces’ clip.

‘Faces’ clip upscaled to 4k and downloaded from Runway:

Here’s a close-up crop of part of the central female’s face. A grab from a 720p video frame is on the left and from the upscaled 4k video on the right:

The ‘4k’ version shows more definition, though this could have been achieved with a small amount of sharpening in a regular video editing application.


In part five of this series we’ll use Runway’s online video editor to sequence the generated video clips into a promo!

A colorful logo with the word "Pixels" in bold, white, and orange letters, above ".cool" in red cursive, outlined with rainbow layers and yellow rays in the background.