Biography
Justus Thies leads the Neural Capture & Synthesis Group at the Max Planck Institute for Intelligent Systems. He received his PhD from the University of Erlangen-Nuremberg in 2017 for his research on marker-less motion capturing of facial performances and its applications.
More recently, he focuses on neural image synthesis techniques that allow for video editing and creation. His work combines methods from the Computer Vision, the Machine Learning and the Computer Graphics field. He publishes regularly at top tier conferences with broad impact not only in the research community but also in public media.
He is particularly known for his work on facial reenactment and video manipulation (e.g., Face2Face).
Education and positions held
- 2017-2021:
- Visual Computing & Artificial Intelligence (Prof. Nießner), Technical University of Munich (Postdoc)
- 2014-2017:
- Computer Graphics (Prof. Greiner), University of Erlangen Nuremberg (PhD student)
- 2008-2014:
- Computer Science, University of Erlangen Nuremberg (Bachelor/Master)
Research Summary
The main theme of my work is to capture and to (re-)synthesize the real world using commodity hardware. It includes the modeling of the human body, tracking, as well as the reconstruction and interaction with the environment.
The digitization is needed for various applications in AR/VR as well as in movie (post-)production. Teleconferencing and working in VR is of high interest for many companies ranging from social media platforms to car manufacturer. It enables the remote interaction in VR, e.g., the inspection of 3D content like CAD models or scans from real objects. A realistic reproduction of appearances and motions is key for such applications.
Thus, my work is closely related to photo-realistic video synthesis and editing. The development of algorithms for photo-realistic creation or editing of image content comes with a certain responsibility, since the generation of photo-realistic imagery can be misused. That’s why I’m also working on the detection of synthetic or manipulated images and videos (Digital Multi-media Forensics).
Key publications
- J. Thies, M. Zollhöfer, M. Stamminger, C. Theobalt, and M. Nießner. Face2face: Real-time Face Capture and Reenactment of RGB Videos. In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 2016.
- A. Rössler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Nießner. FaceForensics++: Learning to Detect Manipulated Facial Images. ICCV, 2019.
- V. Sitzmann, J. Thies, F. Heide, M. Nießner, G. Wetzstein, and M. Zollhöfer. DeepVoxels: Learning Persistent 3D Feature Embeddings. In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 2019.
- J. Thies, M. Zollhöfer, and M. Nießner. Deferred Neural Rendering: Image Synthesis using Neural Textures. ACM Transactions on Graphics 2019 (TOG), 2019.
- J. Thies, M. Elgharib, A. Tewari, C. Theobalt, and M. Nießner. Neural Voice Puppetry: Audio-driven Facial Reenactment. In Proc. European Conference on Computer Vision, 2020.