We continue our series introducing one of the world’s most popular online generative AI video production toolkits, as we create a short video for a brand new immersive experience.
The AI tools and models used in this series
We’ll use runway.ml to generate video clips and assemble them into a sequence, and we’ll use additional AI tools to help with initial ideation and to create text prompts to use at runway to help generate high quality video clips.
The main steps in the project:
- Generate an overview for an ‘immersive experience’ promotional video using perplexity.ai
- Generate still images using the flux-1.1-pro model at replicate.com to use as ‘first frame’ prompts at runway
- Generate short video clips using runway’s latest Gen-4 model
- Use runway’s online video editor to sequence the promotional video
- Use the speech-02-hd AI model at replicate.com to generate a realistic-sounding voiceover
- Use udio.com to generate a background music track
- Add text titles and export the video file
Part 7: Trim video clips and generate a music track
If you’ve missed the previous parts of this series, here’s the short promo video for a new immersive experience, as saved at the end of part six. It includes five of the original AI-generated video clips and the AI-generated voiceover:
Trim existing video clips in Runway
Hover over the start of a clip and drag your mouse to the right to trim from the start of a clip, or hover over the end of a clip and drag your mouse to the left to trim in from the end. Below, we’ve trimmed off the last second of the first video clip in the sequence:
Work along the timeline making adjustments to the remainder of the clips as needed.
Note: This method of editing a video sequence by manually trimming each clip and moving them around on the timeline is not a ‘professional’ workflow. In a professional environment a 3-point-editing workflow will be used.
Use AI to generate a music track
Runway does not offer the ability to generate music from scratch, so we’re going to use udio.com, one of the most popular online music generation toolkits. Signing up is free and you receive an initial allocation of free credits to try it out. We used udio.com at the end of 2024 to generate our festive song ‘Digital Christmas Dream’:
Feel free to play back the full festive creation 🙂
We start by entering our own text prompt – Can Udio create something usable from such a generic starting point?
Of the two generated 30-second tracks, ‘Celestial Tapestry’ could perhaps be used as part of a background sound mix:
To improve the text prompt we returned to perplexity.ai to continue our conversation about generating a promo video, and asked it to create a detailed prompt to use with Udio:
udio.com will accept a detailed text prompt:
Here are the two 30 second ‘songs’ generated by Udio at the first time of asking:
Chasing Horizons
Towards tomorrow
‘Towards tomorrow’ has the right vibe to go with the promo video. This music track has been imported into the existing runway video project:
Runway includes basic audio options to set the volume, speed and add a fade in/out:
As a voice narration track is also present, the music drowns it out. The music volume needs to ‘duck’ as the voice narration plays, rather than remaining at a single level all the way through…
Add keyframes to the audio track
Click the white ‘keyframes’ icon near the start of the music track (It looks like a double diamond):
The keyframe view opens. Place the playhead at the start of the music track:
In the audio properties panel on the right, set the VOLUME to be ZERO(We’re going to set the music to fade in, so it starts to start at zero):
Move the playhead one second to the right:
This time, set the volume to 100:
Repeat this process to ‘dip’ the music as required when the voiceover is playing. The music keyframe track ends up looking like this:
Add a text track to the sequence
To add text titles to a video project (or in this case a text ‘watermark’) click the ‘text’ button on the left. Enter the text to display then edit text properties using the panel on the right. Here’s how the ‘final’ timeline looks including the text track, video track, narration track and music track:
Export the sequence as a video file
Exporting a video from Runway is simple. Click the ‘Export’ button at the top right of the window then select to export in 720p or 1080p. (To export in 4k, an upgraded paid account is required).
In the final part of this series we’ll view the completed promo video and review the pros and cons of AI video generation using runway.ml in June 2025
