Transparent and Scalable Multiple Face Skin Tone Classification with RetinaFace, U-Net, and EfficientNetV2-S

DOI

10.2991/978-94-6463-866-0_13How to use a DOI?

Keywords

Skin tone classification; RetinaFace; YCrCb color space thresholding; U-Net Segmentation; LAB color space; CLAHE; EfficientNetV2; HDBSCAN; GMM; Monk Skin Tone dataset

Abstract

Facial skin tone classification plays a vital role in dermatological AI and the beauty tech industry, enabling personalized skincare, inclusive representation, and health monitoring. Accurate tone detection supports product customization and helps reduce bias in computer vision systems.

However, current approaches face significant challenges, including high intra-class variation, lighting inconsistencies, and limited annotated datasets. Most models are designed for single-face inputs and struggle in real-world, multi-face scenarios.

Previous works, such as Zhu et al. (2021), applied hyperspectral imaging with traditional machine learning, achieving 90.4% accuracy. Similarly, the PWStE model (Lee et al., 2022) achieved 88–91% accuracy using semi-supervised learning for skin lesion segmentation.

These systems, though effective, require complex imaging setups or lack scalability for real-time, multi-face environments.

To address these limitations, we propose SkinContextNet — a multi-stream deep learning architecture combining RetinaFace-based detection, YCrCb thresholding with U-Net segmentation, LAB-space CLAHE contrast normalization, and EfficientNetV2-S classification. Clustering methods (GMM, HDBSCAN) to enhance interpretability and soft segmentation.

SkinContextNet achieves 98–99% accuracy on the Monk Skin Tone dataset, outperforming previous models while supporting real-time, multi-face classification.

This solution sets a new benchmark in fair, accurate, and scalable skin tone classification for both clinical and consumer applications.

Copyright

© 2025 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Sarah Silvia Pinky
AU  - M. Kowsigan
PY  - 2025
DA  - 2025/10/31
TI  - Transparent and Scalable Multiple Face Skin Tone Classification with RetinaFace, U-Net, and EfficientNetV2-S
BT  - Proceedings of the International Conference on Intelligent Systems and Digital Transformation (ICISD 2025)
PB  - Atlantis Press
SP  - 123
EP  - 136
SN  - 2589-4919
UR  - https://doi.org/10.2991/978-94-6463-866-0_13
DO  - 10.2991/978-94-6463-866-0_13
ID  - Pinky2025
ER  -