Learning Visual Concepts in Images Using Temporal Convolutional Networks

With the advancement of technology and the rapid rise of social media, people now tend to share their experiences and feelings using images. It is useful to effectively mine opinions and identify sentiments from the images available on various platforms for tasks like social media marketing and user profiling. Visual sentiment concepts extracted from tags of online images are known as adjective noun pairs and the sentiment conveyed by them is usually predicted using opinion mining methods. However, the pre-training work is time consuming and requires huge space. Instead of collecting tags for the images, in this paper we propose to automatically predict the corresponding sentiments using deep convolutional networks. Next, to model the sequence of different sentiments in a single image we consider a recurrent neural network. Such a model is able to remember the context of different sentiments in a single image. We applied our method on Flickr dataset and our approach outperformed baselines in the range of 3-20%.