From Spider-Man to The Avengers: Achieving Photoreal Digital Actors
Dr. Paul Debevec
Signal and Image Processing Institute, Department of Electrical Engineering,
Institute for Creative Technologies, University of Southern California
http://www.ict.usc.edu/~debevec
Abstract:
Somewhere between 2001's "Final Fantasy" and 2008's "The Curious Case of Benjamin Button", digital actors in movies went from being strangely synthetic to compellingly realistic. I will present how a variety of techniques for facial scanning, reflectance measurement, image-based lighting, and data-driven animation have helped achieve these advances. In particular, I will present how the Light Stage systems developed at USC ICT can be used to digitally relight real faces and record skin-pore-accurate facial models, helping create and relight digital actors in movies such as "Superman Returns", "The Social Network", and "Avatar". For in depth examples, I will present the "Digital Emily" project done with Image Metrics, one of the first photoreal digital faces, and our latest result "Digital Ira" done with Activision, showing a new level of realism in a real-time interactive character. I will also cover how we have adapted the discrete Lighting of the Light Stages to a continuous sphere for scanning shiny objects such as sunglasses, and various forms of autostereoscopic 3D displays useful for displaying virtual characters and for 3D teleconferencing.
Biography:
Paul Debevec is a research professor in the University of Southern California's Viterbi School of Engineering and the Associate Director of Graphics Research at USC's Institute for Creative Technologies. Since his 1996 UC Berkeley Ph.D. Thesis, he has helped develop data-driven techniques for photorealistic computer graphics including image-based modeling and rendering, high dynamic range imaging, image-based lighting, appearance capture, and 3D displays. His short films including The Campanile Movie, Rendering with Natural Light, and Fiat Lux provided early examples of the virtual cinematography, HDR lighting techniques seen The Matrix trilogy and have become standard practice in visual effects. Debevec's Light Stage systems for photoreal facial scanning have contributed to groundbreaking digital character work in movies such as Spider-Man 2, Superman Returns, The Curious Case of Benjamin Button, Avatar, and The Avengers, earning him and his colleagues a 2010 Scientific and Engineering Award from the Academy of Motion Picture Arts and Sciences. He serves as Vice President of ACM SIGGRAPH and is a member of the Visual Effects Society, the Academy of Motion Picture Arts and Sciences, and the Academy's Science and Technology Council.