We introduce Relightable Gaussian Blendshapes for modeling photorealistic head avatars. Our method learns a base head model with a neutral expression and a set of expression blendshapes from a monocular video captured in uncontrolled lighting conditions. Both the neutral model and expression blendshapes are represented as 3D Gaussians, which encapsulate not only geometric parameters but also physically based rendering (PBR) material properties. We utilize an environmental map to represent unknown lighting, and jointly optimize the Gaussian model and environmental light parameters. Besides, we implement a PBR shader that supports discrete light integration and omnidirectional visibility in real time (within 1 ms), a capability not seen in previous works. Another key design is the optimization of a personalized deformable mesh alongside the Gaussian blendshapes to derive normals and visibility, which are intractable to extract directly from Gaussians. Our method achieves the computational efficiency of Gaussian blendshapes at 365fps while enabling relighting under arbitrary environmental lighting, surpassing state-of-the-art methods in both quality and efficiency.
Our method transforms a monocular video input into a relightable Gaussian blendshapes representation of a head avatar. This representation encompasses a neutral model \( B_0 \), a group of expression blendshapes \( B_1,B_2,...,B_K \), and the mouth interior model \( B_m \). All models are composited of Gaussians, with each Gaussian carries both geometric and material properties. Relightable avatar models of arbitrary expressions and poses can be generated by linear blending with expression parameters \( \psi'_k \) and linear blend skinning with joint and pose parameters \( \Theta' \). Normals and visibility are then interpolated from an associated deformable mesh. Given a novel lighting environment map \( L' \), each Gaussian is shaded using a PBR shader and rendered to photorealistic images in real time using Gaussian splatting.
We compare our method with state-of-the-art approaches, including the point-based PointAvatar and the mesh-based FLARE. We extend GaussianAvatars and MonoGaussianAvatar with material properties and apply the same lighting representation and PBR shader as used in our method to support relighting.
We introduce relightable Gaussian blendshapes for reconstructing a photorealistic head avatar from monocular video. Our approach decouples appearance into lighting and materials, represented as parameters of the Gaussian blendshapes. We circumvent the limitations of poor geometric representation inherent in Gaussians by deriving normals and visibility from the associated mesh. Additionally, we demonstrate the feasibility of computing omnidirectional visibility in real time. Our method outperforms existing approaches in both quality and performance.
@inproceedings{ma2025relightable,
author = {Ma, Shengjie and Zheng, Youyi and Weng, Yanlin and Zhou, Kun},
title = {Relightable Gaussian Blendshapes for Head Avatar Animation},
booktitle = {International Conference on Computer-Aided Design and Compute Graphics},
year = {2025},
}