In the age of Deep Fakes (Reddit's ban of AI-Deepfakes was inevitable), we can now add Photo Wake-up by Chung-Y Weng, Brian Curless, and Ira Kemelmacher-Shlizerman. Photo Wake-Up seems innocuous but its technology is both amazing and a bit scary if imagined in nefarious hands.
It might be a wakeup call in its own right, both in terms of what this holds for the future of 3D asset creation and further proliferation of deep fakes the average net cruiser will never suspect.
Three intelligent and digitally enlightened individuals have managed to pull off an amazing bit of inventiveness by animating a character from a photograph.
The amazing thing is, this is not a flat character but a three-dimensional character that might well be a boon to AR/VR in the future.
Imagine a gallery where the subject walks out of a historical photograph and stands directly in front of you or converses with you about its meaning or maybe even the weather.
All this driven by Artificial Intelligence (AI).
The software: Photo Wakeup: https://grail.cs.washington.edu/projects/wakeup/
Paper Summary (https://arxiv.org/abs/1812.02246):
We present a method and application for animating a human subject from a single photo. E.g., the character can walk out, run, sit, or jump in 3D. The key contributions of this paper are: 1) an application of viewing and animating humans in single photos in 3D, 2) a novel 2D warping method to deform a posable template body model to fit the person's complex silhouette to create an animatable mesh, and 3) a method for handling partial self occlusions. We compare to state-of-the-art related methods and evaluate results with human studies. Further, we present an interactive interface that allows reposing the person in 3D, and an augmented reality setup where the animated 3D person can emerge from the photo into the real world. We demonstrate the method on photos, posters, and art.
That says quite a bit for a one paragraph summary. Technically it's an amazing fete so let's parse this information a little further.
1. An application for viewing and animating humans in single photos. Depending on the complexity of the animation this is an amazing thing in itself.
Just imagine all the development hours saved.
Just imagine all the production costs saved.
Just imagine all of the out-of-work character developers that choose the wrong fork in their career path.
2. A novel 2D warping method to deform a posable template body model to fit the person's complex silhouette to create an animatable mesh.
Some of us have spent years honing our ability to create characters. Bones, skinning, weighting and all those skills will one day be for naught. While it won't happen overnight -- or anytime soon for that matter -- this could be the writing on the wall for certain aspects of 3D work.
What's that? No machine can produce art? Some in the AI community will disagree with you.
Remember, AI is a game changer. It's not static … it is in a state of constant learning.
3: A method for handling partial self-occlusions. Anyone that has worked in AR/VR has run into this. Basically, it's like your body hiding your hand from the sensors. The body is occluding the hand. Just the fact that they are specifically addressing self-occlusion shows how serious the research is behind this effort. Self-occlusion is a real-world problem and the authors are attempting to remedy it.
This goes well beyond the concept of publishing something to prove a point versus research and development of a functioning piece of software.
Throw in the ability to misuse this software for deep fakes during these turbulent political times and you are stirring up a recipe for misinformation of catastrophic proportions.
Older generations that tend to believe what is “written” also tend to vote. The two together can make for some unsettling times.
M.D. McCallum, aka WarLord, is an international award-winning commercial graphics artist, 3D animator, published author, project director, and webmaster with a freelance career that spans over 20 years. Now retired, M.D. is currently working part-time on writing and select character development projects. You can learn more about MD on his website.