FLAME-driven Monocular Head Avatar Reconstruction

Given several frontal videos of a person's head with corresponding tracked meshes from FLAME, the task is to re-animate the person with unseen FLAME expression codes and then render from both seen (blue) and unseen (orange) camera viewpoints. This requires reconstructing an animatable 3D head representation (=3D head avatar). The challenge is conducted on recordings from 5 different individuals. For each individual, 18 short facial performance sequences are provided for training while the remaining 4 sequences are hold-out. For the hold-out sequences, only the tracked FLAME meshes and the camera poses are known.

Please see the NeRSemble Benchmark Toolkit on how to obtain the data to participate in the benchmark and prepare a submission.

Update to the avatar task (2026)

New in v2 (2026): The FLAME tracking for creating and driving the avatar has been updated with better torso stability, better lip closure, and more accurate mouth tracking. Please ensure that you are using flame2023_tracking_v2 when preparing a submission (see the NeRSemble Benchmark Toolkit). The leaderboard has been refreshed to only include methods that use the updated FLAME tracking. Submissions to v1 of the benchmark task are closed but the v1 leaderboard can still be accessed.

Evaluation and Metrics

We evaluate the similarity beween the ground truth and generated RGB videos. Our image-wise evaluation metrics are peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and learned perceptual image patch similarity (LPIPS). Additionally, we measure temporal consistency with FovVideoVDP (JOD) that is sensitive to flickering, noise and other temporal artifacts. Finally, we employ two face-specific metrics: Average Keypoint Distance (AKD) measured in pixels with keypoints estimated from PIPNet, and cosine similarity (CSIM) of identity embeddings based on ArcFace. For each pair of generated and ground-truth videos, we compute these six metrics and average the result for all 3 hold-out cameras, 4 hold-out sequences, and for all 5 individuals. To save compute during metric evaluation, the face and image metrics are evaluated every 3rd frame.

Evaluation is carried out on GT images with resolution 512x512. The background is expected to be white.

Results

Methods PSNR SSIM LPIPS JOD AKD CSIM
PMD-GAvatar 24.769 0.850 0.117 6.801 2.845 0.886
Kirill Chemrov, Vladislav Korzun, Darya Zhukova, Grigory Samokhin, Oleg Sémery. Personalized Mesh-conditioned Diffusion for 3D Gaussian Avatar.
WAP4D 19.512 0.809 0.187 5.568 6.469 0.776
Paul Creavin, Thomas Creavin, Timur Levent Kesdogan. WAP4D: Wan Avatar Prior for 4D.

Please refer to the submission instructions before making a submission

Submit results