Welcome! This month we’re introducing one of the world’s most popular generative AI toolsets, runway.ml, and using it to generate a set of short video clips to create a promotional video for a conference. The series continues below.
Project overview
As generative AI develops at speed, we’re delving into the apps and tools currently available to help with ‘real’ creative projects. In this case, we’re using the video generation tools available at runway.ml to create short video clips to edit together to create a short promotional video for an upcoming conference. We’ll use chatGPT to suggest a script, then use runway’s text-to-speech tool to create a realistic-sounding voiceover.
Note that this is not an explainer series, though we do detail the main steps to follow at runway.ml.
Using runway’s AI video clip generator: Creating camera clips with defined camera movement
For the next 3 clips in our planned video sequence (clips 3,4 and 5) we’ll use runway’s ‘camera motion’ option to incorporate camera movement into generated clips.
The next clip is clip 3, where we’ll request a shot of a large modern conference hall with stands.
1: We start by adding our prompt and using the ‘free preview’ option as before to generate possible start frames for the clip:
2: Select a prefered preview image then click the ‘Camera motion’ button as highlighted with the yellow circle below:
3: Use the sliders to add any prefered camera moves. Remember the movement will occur over a period of just 4 seconds. We’re adding a horizontal pan and a slight zoom to the shot, so have set ‘horizontal’ to 3 and ‘zoom’ to 1:
4: Click ‘Save’ to apply these settings, then click the ‘Generate 4s’ button to create the clip. You can see the rendered video clip in the section below.
We repeated the above process for clips 4 and 5, (a large auditorium, then a small group of people enjoying catching up) using the text prompts and camera motion settings below:
The generated video clips (clips 3, 4 and 5)
You’re welcome to download these clips to use in your own video / generative AI projects.
Clip 3: An expo floor with white and blue accents
Clip 4: A darkened auditorium with a large central display screen
Clip 5: Four people interacting positively over coffee and bagels.
Issues / errors observed
We expect there to be visible issues with generated video clips right now, though many such ‘glitches’ will be eradicated as the AI models develop. Think of these issues as part of the fun!
- The people in clip 3 exhibit warping and shapeshifting, though this is not distracting as the star of the clip is the exhibition hall itself.
- The people in clip 5 are a little too ‘airbrushed’ and exhibit overly-perfect shiny teeth, but this is a better result than a lot of modern ‘filters’ achieve when fed with real footage!
Join us for part five of this series when we use runway’s ‘image + description’ option to apply specific changes to an image to generate a short video.
