Real-Time Facial Expression Cloning and Animation

---

This is the work for generating a real-time facial expression animation based on an individualized wireframe model avatar. The tracking and head pose detection (developed by Dr. Qing Ji) and animation (developed by Dr. Lijun Yin) in 3D space are conducted in real-time. The face animation is based only on 22 fiducial points of a performer. 

This is the collaborative work with Dr. Qiang Ji from Intelligent System Lab, Rensselaer Polytechnic Institute. 

           




• Facial expression cloing with real time face tracking and aniamtion




Real time tracking and animation

---

---

Face Modeling and Expression Analysis and Synthesis

---

A realistic face modeling and facial expression analysis and systhesis system is developed. The system consists of face topo-feature labeling, facial model adaptation, texture of interest detection and facial expression reconstruction.

• Video 1: Face expressions (original video)
• Video 2: Texture of interests (TOI)
• Video 3: Tracking the key points of the facial model
• Video 4: Face model adaptation using corse-to-fine physics-based method
• Video 5: Adapted model animation
• Video 6: Facial texture before normalization
• Video 7: Facial texture after normalization
• Video 8: Texture of interest (TOI) updated (transmitted)
• Video 9: Synthesis only using 1st frame texture (no texture update)
• Video 10: Synthesis with only eye-mouth texture update
• Video 11: Synthesis with only wrinkle texture update
• Video 12: Synthesis with all TOI texture update, but no blending
• Video 13: Synthesis with TOI texture update and blending
• Video 14: Synthesis with TOI texture update, and blending large nasolabial texture
• Video 15: Reconstructed face with background
• Video 16: MPEG1 (head, 79kb/s)
• Video 17: Motion JPEG (head, 81kb/s)
• Video 18: MPEG1 (mario, 90kb/s)
• Video 19: Motion JPEG (mario, 93kb/s)
  
  

---

Video Head TP+6: Right view

Video Head TP-6: Left view

Video Fit TP+6: Right view

Video Fit TP-6: Left view

Topographic Labeling: Front view

The material is based upon work supported by the NSF under grant 0414029. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation.

---

lijun@cs.binghamton.edu