Data-driven models of concepts are gaining popularity in Psychology and Cognitive Science. Distributional semantic models represent word meanings as abstract word co-occurrence patterns, and excel at capturing human meaning intuitions about conceptual relationships; however, they lack the explicit links to the physical world that humans acquire through perception. Computer vision neural networks, on the other hand, can produce representations of visually-grounded concepts, but they do not support the extraction of information about the relationships between objects. To bridge the gap between distributional semantic models and computer vision networks, we introduce SemanticScape, a model of semantic concepts grounded in the visual relationships between objects in natural images. The model captures the latent statistics of the spatial organization of objects in the visual environment. Its implementation is based on the calculation of the summed Euclidean distances between all object pairs in visual scenes, which are then abstracted by means of dimensionality reduction. We validate our model against human explicit intuitions on semantic and visual similarity, relatedness, analogical reasoning, and several semantic and visual implicit processing measurements. Our results show that SemanticScape explains variance in human responses in the semantic tasks above and beyond what can be accounted for by standard distributional semantic models and convolutional neural networks; however, it is not predictive of human performance in implicit perceptual tasks. Our findings highlight that implicit information about the objects’ spatial distribution in the environment has a specific impact on semantic processing, demonstrating the importance of this often neglected experiential source.

de Varda, A., Petilli, M., Marelli, M. (2025). A distributional model of concepts grounded in the spatial organization of objects. JOURNAL OF MEMORY AND LANGUAGE, 142(April 2025) [10.1016/j.jml.2025.104624].

A distributional model of concepts grounded in the spatial organization of objects

de Varda A. G.;Petilli M.;Marelli M.
2025

Abstract

Data-driven models of concepts are gaining popularity in Psychology and Cognitive Science. Distributional semantic models represent word meanings as abstract word co-occurrence patterns, and excel at capturing human meaning intuitions about conceptual relationships; however, they lack the explicit links to the physical world that humans acquire through perception. Computer vision neural networks, on the other hand, can produce representations of visually-grounded concepts, but they do not support the extraction of information about the relationships between objects. To bridge the gap between distributional semantic models and computer vision networks, we introduce SemanticScape, a model of semantic concepts grounded in the visual relationships between objects in natural images. The model captures the latent statistics of the spatial organization of objects in the visual environment. Its implementation is based on the calculation of the summed Euclidean distances between all object pairs in visual scenes, which are then abstracted by means of dimensionality reduction. We validate our model against human explicit intuitions on semantic and visual similarity, relatedness, analogical reasoning, and several semantic and visual implicit processing measurements. Our results show that SemanticScape explains variance in human responses in the semantic tasks above and beyond what can be accounted for by standard distributional semantic models and convolutional neural networks; however, it is not predictive of human performance in implicit perceptual tasks. Our findings highlight that implicit information about the objects’ spatial distribution in the environment has a specific impact on semantic processing, demonstrating the importance of this often neglected experiential source.
Articolo in rivista - Articolo scientifico
Distributional semantics; Grounding; Object concepts; Visual models;
English
22-feb-2025
2025
142
April 2025
104624
none
de Varda, A., Petilli, M., Marelli, M. (2025). A distributional model of concepts grounded in the spatial organization of objects. JOURNAL OF MEMORY AND LANGUAGE, 142(April 2025) [10.1016/j.jml.2025.104624].
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/553802
Citazioni
  • Scopus 0
  • ???jsp.display-item.citation.isi??? 0
Social impact