Multitask painting categorization by deep multibranch neural network

We propose a novel deep multibranch and multitask neural network for artist, style, and genre painting categorization. The multibranch approach allows us to exploit at the same time the coarse layout of the painting and the fine-grained structures by using painting crops at different resolutions that are wisely extracted using a Spatial Transformer Network trained to identify the most discriminative subregions of paintings. The effectiveness of the proposed network is proved in experiments that are performed on a new dataset originally sourced from wikiart.org and hosted by Kaggle, and made suitable for artist, style and genre multitask learning. The dataset here proposed and made available for research is named MultitaskPainting100k, and is composed by 100K paintings, 1508 artists, 125 styles and 41 genres annotated by human experts. Among the different variants of the proposed network, the best method achieves accuracy levels of 56.5%, 57.2%, and 63.6% on the MultitaskPainting100k dataset for the tasks of artist, style and genre prediction respectively.