The latest advancements in AI art have taken over the internet. Whether it’s realistic images, Ghibli Studio anime, or toy action figures, it seems like many people are enjoying the ability to instantly make their vision come to life. One thing that hasn’t gotten as much attention is the fact that there are also tools out there which let you take those AI-generated images and then transform them into videos.
In this quick guide, I’m going to show you just how you can do it. It’s worth noting that the method can actually be applied to transform any kind of image into a video. It’s not limited to solely AI art. So if you have some old photograph of yourself you can now make it “come to life.” Let your imagination run wild and have fun with it.
How to transform an image into a video: easy guide
First, let’s review the tools you’ll need to get this done. Then, I’ll walk you through a successful example and go over some tips you should keep in mind when you try it yourself.
Toolkit
For the actual transformation, you’ll only need one tool – Live Portrait.
However, depending on what exactly you’re looking to do, you might need additional tools as well.
For example:
- If you want to make a video of yourself talking as a cartoon or an action figure, then you’ll need an image generator to create that first.
- If you want to make a video of your cat talking in a human voice that’s not your voice, then you’ll need an AI voice generator.
You probably already have a preferred image generator but if you don’t, both Grok and the paid version of ChatGPT are what I would recommend – with a preference for the latter. For the voice generator, Eleven Labs is one of the industry leaders. They have a free plan, but alternatives exist as well.
The process
Go to Live Portrait and you’ll see a prompt to upload two files – a source portrait and a driving video:
The source portrait is the still image that will be turned into your video. For my example, I used a real photograph that I asked ChatGPT to convert into Studio Ghibli style.
The driving video provides the movement instructions and audio. Think of it as the engine under the hood of the image. I recorded mine against a white background to keep the focus on my face.
⚠️ Important: Make sure your driving video is a 1X1 square. Don’t upload wide frame or vertical videos.
After you upload both files, scroll down to the Animate section and click the Animate button. You’ll then need to wait a minute or two for Live Portrait to process everything.
When it’s done, you’ll see the generated video on the left. Separately, on the right, you’ll see another video with three side-by-side panels. This video shows your driving video, source portrait, and generated video all next to each other, so you can compare the input to the output.
Here’s how mine came out:
Not too bad, right?
But before you run off to create your own, stick with me for just another minute so you don’t make some of the same beginner mistakes I did. To some extent, you’ll only have one chance to get it right.
Here’s what you need to know…
Hugging Face free user limits
As you just saw, the process is relatively easy to follow. However, it’s also just as easy to mess it up and come out with a disappointing final product. And I say that from firsthand experience. It took me three tries before I was able to generate the video I shared above.
The problem with not getting it right the first time, is that Hugging Face – the platform providing a home for Live Portrait – has a very tight cap on free usage:
Basically, you can generate one video every 24 hours. So you need to make it count.
Though if you really like the tool, it’s also possible to sign up for the Hugging Face Pro plan for $9 per month and that will free you up to make more videos. Or, if you have some experience working with the Command Line, you can install it locally and use it that way (with some exceptions for Mac silicon users).
Assuming you want to keep things simple and stick with the free Hugging Face version, then make sure you follow the tips below.
Tips to generate a good video
Live Portrait has two main limitations. Understanding them is crucial to getting a good result.
- The tool is based entirely on facial movements. It’s not designed to handle anything below the shoulders. So you can’t move your hands around or jump up and down or do any kind of exaggerated bodily movements.
- It doesn’t have a great imagination to “fill in the gaps.” In practice, this means that the position of the head in the source portrait and in the driving video should stay roughly the same. Let me show you an example of my first attempt, where I didn’t realize this and what happened as a result:
As you can see, in the driving video, the position of my head strayed too far from the position of the source portrait and the tool wasn’t able to animate the source portrait to match. Instead it peeled the source portrait face off to reveal the driving video underneath.
In short, when you’re doing your own, avoid sharp movements that move your face out of the source portrait frame.
Final thoughts on Live Portrait
As with everything else related to AI these days, the developments are happening at a rapid pace. The original version of Live Portrait was only launched last summer and since then it’s already seen significant improvements.
Although the “face only” aspect is limiting, I suspect that it won’t be an issue for very long. TikTok’s parent company, ByteDance, just published a paper about a similar model they’ve been working on called DreamActor-M1 and it incorporates bodily movements in addition to facial expressions. It’s not available to try yet, but I’m sure it’s only a matter of months before we get to dabble with it (or its capabilities show up elsewhere).
Have you tried Live Portrait yet? What did you think? Do know of any other good image-to-video generators that are free and easy to use? Let me know in the comments.
…
Don’t forget to join our crash course on speeding up your WordPress site. Learn more below:
Or start the conversation in our Facebook group for WordPress professionals. Find answers, share tips, and get help from other WordPress experts. Join now (it’s free)!
