This paper presents a face control system for 3D avatar with webcam using the Facemesh API for face tracking and Jeeliz-transfer API for eye tracking. 3D avatar face animation is developed as a responsive web application. It starts with face detection and tracking through the webcam. Face coordinate data is normalized to a vertical face view where the distance between the eyes and the level of the eyes are the same for each video frame. Then new face coordinates are calculated in both 2D and 3D to study the change of specific coordinates such as mouth shape and face shape. In addition, the coordinates of face structures in the video platform have also been added to the 3D model platform. Specific coordinates are studied to analyze distance changes to be applied in 3D avatar manipulation. The 3D models are designed and created in multiple blend-shapes or basic character facial features. However, due to the limitations of web browser-based 3D morphing, which Three.js is used for morph target displays, it allows only eight combination shapes to be displayed at the same time. Therefore, the required blending geometry must be pre-assembled. Blend-shape factors are based on an analysis of the coordinates of each moving face in order to eliminate the limitations. Our 3D facial animations with a webcam generate results of high quality, real-time and online simulation. Therefore, our work is a fundamental technology that can be applied to animate other 3D characters in blend-shape format.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.