RelightMaster: Precise Video Relighting with Multi-plane Light Images

Weikang Bian1, Xiaoyu Shi2, Zhaoyang Huang1, Jianhong Bai4,
Qinghe Wang5, Xintao Wang2, Pengfei Wan2, Kun Gai2, Hongsheng Li1, 3
1Multimedia Laboratory, The Chinese University of Hong Kong
2Kling Team, Kuaishou Technology   3CPII under InnoHK
4Zhejiang University   5Dalian University of Technology
Paper arXiv


RelightMaster generates physically plausible lighting and shadows with the reference videos.


Abstract

Recent advances in diffusion models enable high-quality video generation and editing, but precise relighting with consistent video contents, which is critical for shaping scene atmosphere and viewer attention, remains unexplored. Mainstream text-to-video (T2V) models lack fine-grained lighting control due to text's inherent limitation in describing lighting details and insufficient pre-training on lighting-related prompts. Additionally, constructing high-quality relighting training data is challenging, as real-world controllable lighting data is scarce. To address these issues, we propose RelightMaster, a novel framework for accurate and controllable video relighting. First, we build RelightVideo, the first dataset with identical dynamic content under varying precise lighting conditions based on the Unreal Engine. Then, we introduce Multi-plane Light Image (MPLI), a novel visual prompt inspired by Multi-Plane Image (MPI). MPLI models lighting via K depth-aligned planes, representing 3D light source positions, intensities, and colors while supporting multi-source scenarios and generalizing to unseen light setups. Third, we design a Light Image Adapter that seamlessly injects MPLI into pre-trained Video Diffusion Transformers (DiT): it compresses MPLI via a pre-trained Video VAE and injects latent light features into DiT blocks, leveraging the base model's generative prior without catastrophic forgetting. Experiments show that RelightMaster generates physically plausible lighting and shadows and preserves original scene content.



We build a dataset rendering pipeline based on Unreal Engine to batch generate video training data with the same content but different lighting. We adjust the 3D position, color, and intensity of the point lights to provide fine-grained control over the lighting conditions.



RelightMaster generates relit videos according to the input lighting conditions, which include light source positions in the 3D camera frustum, and the light color and intensity. RelightMaster also supports multiple and temporally-varying light source control.



We conduct four experiments for comparison. The first two groups focus on dynamic light position and color changes: the light source is moved from the top-left to the bottom-right of the frame, with its color gradually transitioning from red to green. The remaining two groups involve dynamic depth adjustment of a white light source, which is gradually moved forward along the camera lens.

BibTeX


@article{bian2025relightmaster,
  title={RelightMaster: Precise Video Relighting with Multi-plane Light Images},
  author={Bian, Weikang and Shi, Xiaoyu and Huang, Zhaoyang and Bai, Jianhong and Wang, Qinghe and Wang, Xintao and Wan, Pengfei and Gai, Kun and Li, Hongsheng},
  journal={arXiv preprint arXiv:2511.06271},
  year={2025}
}