Seeing or hearing? Feeling. The influence of music on paintings

Integration between the auditory system and other cognitive and sensory systems is the key to make sense of sounds in a communication. Evidence have been provided that not only visual art, but also music can convey precise semantic meanings and conceptual, besides emotional, information. Previous research suggests that visual stimuli influence auditory processing more than vice versa, hypothesizing a visual dominance in recognition. We designed a study in which we could test both for music semantics and for semantic interference across modalities. The aim was to investigate the effect of background music, either congruent or incongruent, on the semantic categorization of artworks. After the validation process, 160 artworks and 16 musical pieces were selected and divided into 8 categories: Lullaby, Play, Battle, Dance, Love, Tragic Love, Relax, Requiem. Paintings were paired with either semantically congruent or incongruent musical excerpts. 20 volunteers were asked to classify the semantic meaning of the paintings. We hypothesized that in the incongruent condition, categorization judgments were worsened. The categorization judgments’ analysis indicated an effect of the background music factor. When associated with an incongruent background music, artworks’ categorization judgments were significantly worse. These results strengthen the hypothesis of a bidirectional modulation between visual and auditory channels and that semantic interference is a general cognitive mechanism. Further research should enlighten the direction of such influence and if this is a top-down or a bottom-up modulation.