Turn video into an animated cartoon

Michael F. Cohen with his colleagues at Microsoft Research has invented a way to turn digital video into an animated cartoon. They’ve also developed techniques to combine animation with a still picture.


To define more meaningful regions, the user outlines the shapes on keyframes in the video, such as the pants on the girl swinging. He does this on several keyframes. “We rely on the user to circle things like the girl’s pants. There’s different shading on the pants, and some stripes. We can’t group them automatically,” said Cohen. The system can then interpolate between the keyframes, maintaining smooth trajectories along the time dimension, without jerky transitions or the need to draw on each frame.


The more complex the movements, the more keyframes the program needs to make the transition smooth. However, even the 300 frame video of the girl swinging on the monkey bars only needs a keyframe every 10 or 15 frames. By blending in the original pixels, the cartoon ends up being mixed - a real girl swinging through a cartoon landscape, or a cartoon girl swinging through a real landscape. The possibilities are endless.


In the future, the team would like to improve the user interface and make it possible to create other animation styles. They’d also like the program to work better with hand-held cameras. Unfortunately, it won’t be on your computer store shelf anytime soon, though the researchers would like this eventually.