A grid anchor based cropping approach exploiting image aesthetics, geometric composition, and semantics

Image cropping aims at the selection of the relevant part of an image maximizing its aesthetic quality and composition. The part of the image that needs to be removed is highly dependent on user preferences and can be related to image aesthetics, composition, informativeness, or other criteria. Since the concept of the perfect crop does not exist, but there are several cropping possibilities, recent cropping algorithms are trained to rank a set of crop candidates based on their compositional quality. To this end, several benchmark databases have been released that provide for each image a series of human-annotated crop candidates with corresponding scores. Many of the image cropping methods rely on a single criterion to define the best crop or crops in an image. However, a single criterion misses the complexity of human opinions which can differ in personal preferences and backgrounds. Motivated by this, we formulate the cropping problem as a ranking problem of candidate crop regions using a grid anchor based approach and multiple criteria. To evaluate the goodness of a crop region, we design a cropping method by combining three efficient and lightweight neural networks specifically designed to evaluate the quality of a crop in terms of aesthetics, composition, and semantics. Our results on standard datasets show that using more criteria yields better crops than state-of-the-art approaches. This result is also confirmed by a subjective study on user preferences that involved a panel of users.