“We present a real-time deep learning framework for video-based facial performance capture—the dense 3D tracking of an actor’s face given a monocular video. Our pipeline begins with accurately capturing a subject using a high-end production facial capture pipeline based on multi-view stereo tracking and artist-enhanced animations. With 5–10 minutes of captured footage, we train a convolutional neural network to produce high-quality output, including self-occluded regions, from a monocular video sequence of that subject. Since this 3D facial performance capture is fully automated, our system can drastically reduce the amount of labor involved in the development of modern narrative-driven video games or films involving realistic digital doubles of actors and potentially hours of animated dialogue per character. We compare our results with several state-of-the-art monocular real-time facial capture techniques and demonstrate compelling animation inference in challenging areas such as eyes and lips.”
Related Content
Related Posts:
- Intel and Submer Advance Data Center Cooling Tech
- First-Hand Experience: Deep Learning Lets Amputee Control Prosthetic Hand, Video Games
- IBM Unveils World’s First 2 Nanometer Chip Technology, Opening a New Frontier for Semiconductors
- NXP Selects TSMC 5nm Process for Next Generation High Performance Automotive Platform
- First Battery-Free Bluetooth Sticker Sensor Tag Demonstrated at NRF
- IBM Unveils World’s First Integrated Quantum Computing System for Commercial Use
- This 3D-printed prosthetic hand combines speed and strength with simplicity
- A Flexible Arduino Prototype
- First in the World Graffiti Drone
- IBM has made the world’s smallest computer, and it’s just absurd