Communication can be defined as the understanding and exchanging of meaningful messages. The role of communication is central to the lives of human beings as, everyday, we use language to interact with the world around us. Linguistic skills play a fundamental role in this scenario and Language Disorders (LD) are impairments that limit the processing of linguistic information. Early and accurate identification of LD is thus essential to promote lifelong learning and well-being. From an evolutionary perspective, some human language constructs evolved from an ancestral motor system and share the same neural pathways in the Broca's area of the brain. This suggests a correlation between action and language. If such a relationship is well established and reliable, it would be possible to use the former as a marker of the latter. The hypothesis of our work, in a nutshell, is that movement can be a predictor of language. To study this correlation, we developed C(H)o(L)ordination, a Mixed Reality (MR) application for HoloLens 2. The application offers several activities based on visual stimuli involving motor movements, which tap on the same skills needed to perform some language tasks. We performed an exploratory study with N=22 users to test the application usability and user experience. The results suggest that C(H)o(L)ordination is a usable and powerful tool to gather insights on the ongoing debate about language evolution and language disorders.

Beccaluva, E., Vona, F., Di Gioia, F., Patti, A., Guzzo, A., Cappella, I., et al. (2022). Using HoloLens Mixed Reality to research correlations between language and movement: a case study. In ACM International Conference Proceeding Series (pp.1-5) [10.1145/3531073.3531178].

Using HoloLens Mixed Reality to research correlations between language and movement: a case study

Beccaluva E. A.
Primo
;
Di Gioia F.;Stucchi N.;
2022

Abstract

Communication can be defined as the understanding and exchanging of meaningful messages. The role of communication is central to the lives of human beings as, everyday, we use language to interact with the world around us. Linguistic skills play a fundamental role in this scenario and Language Disorders (LD) are impairments that limit the processing of linguistic information. Early and accurate identification of LD is thus essential to promote lifelong learning and well-being. From an evolutionary perspective, some human language constructs evolved from an ancestral motor system and share the same neural pathways in the Broca's area of the brain. This suggests a correlation between action and language. If such a relationship is well established and reliable, it would be possible to use the former as a marker of the latter. The hypothesis of our work, in a nutshell, is that movement can be a predictor of language. To study this correlation, we developed C(H)o(L)ordination, a Mixed Reality (MR) application for HoloLens 2. The application offers several activities based on visual stimuli involving motor movements, which tap on the same skills needed to perform some language tasks. We performed an exploratory study with N=22 users to test the application usability and user experience. The results suggest that C(H)o(L)ordination is a usable and powerful tool to gather insights on the ongoing debate about language evolution and language disorders.
slide + paper
Eye-tracking; Microsoft HoloLens 2; Mixed Reality; Motor-linguistic correlation;
English
16th International Conference on Advanced Visual Interfaces, AVI 2022 - 6 June 2022 through 10 June 2022
2022
ACM International Conference Proceeding Series
9781450397193
2022
1
5
42
open
Beccaluva, E., Vona, F., Di Gioia, F., Patti, A., Guzzo, A., Cappella, I., et al. (2022). Using HoloLens Mixed Reality to research correlations between language and movement: a case study. In ACM International Conference Proceeding Series (pp.1-5) [10.1145/3531073.3531178].
File in questo prodotto:
File Dimensione Formato  
Beccaluva-2022-AVI-VoR.pdf

accesso aperto

Descrizione: Short paper
Tipologia di allegato: Publisher’s Version (Version of Record, VoR)
Licenza: Altro
Dimensione 5.1 MB
Formato Adobe PDF
5.1 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/407230
Citazioni
  • Scopus 1
  • ???jsp.display-item.citation.isi??? 0
Social impact