Enhancing color selectivity in foundation models for downstream color vision tasks

The emergence of computer vision foundation models, inspired by the success of task-agnostic pretrained representations in Natural Language Processing (NLP), is revolutionizing the field. These models produce features that excel in downstream tasks even without fine-tuning. Last year, DINOv2 emerged, surpassing previous state-of-the-art general-purpose features on computer vision benchmarks, both at the image and pixel levels. In this work, we focus on what type of color information is embedded in DINOv2 features, and to assess their performance in computer vision tasks where color is a critical cue—for instance, recognizing the color of vehicles for traffic monitoring, detecting skin tones in biometric applications, or assessing product color attributes in fashion and e-commerce. Furthermore, we also propose a training-free feature transformation that increases color selectivity in DINOv2 features, i.e. their ability to respond differently to various colors in an image, boosting the performance on several classes of the color vision tasks considered.