New anime AI program animates your art for you when you add a voice clip【Videos】

DeepAnime is the attest attempt to raise the heights of automated anime production.

Illustration and animation are both types of drawing, but they’re not entirely one and the same. Tokyo-based tech company AlgoAge is looking to blur the line, though, with its DeepAnime project.

AlgoAge’s primary focus is on deep learning and GANs (generative adversarial networks), which examine input data, such as visual images, in order to break down into comment parts and spot similarities and differentiating factors. This allows GANs to create new artwork that’s unique compared to the inputs, but still identifiable as something that would belong to the same category, such as a cute anime-style girl.

We’ve seen GANs used to develop new anime character designs (and Pokémon designs) in the past, but the goal for DeepAnime is to produce new variations of artwork of the same character. After the human artist inputs a single piece of still artwork, DeepAnime will automatically animate it with mouth movements and blinking eyes, as seen in the examples here.

What’s especially cool is that DeepAnime is able to automatically match up the mouth movements with supplied audio. Input both your artwork and a voice clip, and the program combines them into an anime scene with voiced dialogue.

▼ Video demonstration of DeepAnime

The results are impressive, though a little too rudimentary for a full-fledged professional anime series. However, even in its current form, DeepAnime could be a massive help for video game developers, especially independent creators. Being able to effortlessly transform still character portraits into animated, talking versions for dialogue scenes would save a great deal of time and money, two things small indie studios are often in short supply of.

Like many GANs, DeepAnime is a work in progress, and isn’t available for distribution just yet, but when it is, it could be an extremely helpful tool for talented illustrators who don’t have the funds to pay a team of animators, but still want to see, and hear, their creations come to life.

Source: PR Times via Otakomu
Images: PR Times
● Want to hear about SoraNews24’s latest articles as soon as they’re published? Follow us on Facebook and Twitter!

Credit: