Personalized makeup color exploration is crucial to enhance user experience in applications such as cosmetic recommendations. While virtual makeup offers accessible solutions for this exploration, traditional 2D overlay methods viewed through smartphone or tablet screens often lack realism and fail to precisely simulate how colors appear under real-world lighting. In contrast, Dynamic Projection Mapping (DPM) makeup provides greater realism by projecting colors onto the user’s face, enabling direct human observations.
We propose a novel framework to help non-expert users efficiently find satisfying results in the vast space of color combinations with DPM technology. A text-to-makeup-color model is introduced to efficiently generate makeup colors based on impression text input. Then, we propose a projection-in-the-loop method to interactively refine makeup colors. Users can further match their personal preferences and suit their skin tone and lip color by directly observing the final effect on their faces.
A controlled user study (N = 15) and an expert interview (N = 3) were conducted to evaluate the overall usability and user experience of our proposed framework. Results confirm that this framework enables efficient and user-friendly personalized makeup color exploration over baseline methods.
References
- Kemeng Zhang, Hao-Lun Peng, Yoshihiro Watanabe: Impression-Guided Interactive Personalized Color Exploration Framework for Dynamic Projection Mapping Makeup, International Journal of Human–Computer Interaction, pp. 1–23, doi: 10.1080/10447318.2025.2599521 (2026)