In recent years, deep learning have become popular for solving tasks in a wide range of domains. With this growing diffusion, combined with architectures becoming increasingly complex and sophisticated, understanding how deep models make their predictions is now a crucial and challenging research issue. In this work, we are interested in analysing the behaviour of networks that deal with source code, and we address the problem along two experimental directions: in the first one, we study the activations of the neurons of a transformer trained in the detection of software vulnerabilities so as to identify if (and, eventually, where) some human understandable concepts emerge in the network. In the second one, we generate programs by applying a grammar-based evolutionary algorithm with a fitness function that favours individuals which stimulate (or weaken) the activations in neurons where given concepts majorly emerge. We then study the output of the network on sets of such evolved programs, i.e. how the evolutionary pressure along the direction of a concept affects the prediction. We finally discuss how this combination of evolutionary algorithms with fitness functions derived from the neural activations can be effective for explaining the decision process of deep models, and we suggest further research directions.

Saletta, M., Ferretti, C. (2022). Towards the evolutionary assessment of neural transformers trained on source code. In GECCO 2022 Companion - Proceedings of the 2022 Genetic and Evolutionary Computation Conference (pp.1770-1778). Association for Computing Machinery, Inc [10.1145/3520304.3534044].

Towards the evolutionary assessment of neural transformers trained on source code

Saletta Martina;Ferretti Claudio
2022

Abstract

In recent years, deep learning have become popular for solving tasks in a wide range of domains. With this growing diffusion, combined with architectures becoming increasingly complex and sophisticated, understanding how deep models make their predictions is now a crucial and challenging research issue. In this work, we are interested in analysing the behaviour of networks that deal with source code, and we address the problem along two experimental directions: in the first one, we study the activations of the neurons of a transformer trained in the detection of software vulnerabilities so as to identify if (and, eventually, where) some human understandable concepts emerge in the network. In the second one, we generate programs by applying a grammar-based evolutionary algorithm with a fitness function that favours individuals which stimulate (or weaken) the activations in neurons where given concepts majorly emerge. We then study the output of the network on sets of such evolved programs, i.e. how the evolutionary pressure along the direction of a concept affects the prediction. We finally discuss how this combination of evolutionary algorithms with fitness functions derived from the neural activations can be effective for explaining the decision process of deep models, and we suggest further research directions.
paper
deep neural networks; explainable AI; source code analysis; structured grammatical evolution;
English
2022 Genetic and Evolutionary Computation Conference, GECCO 2022 - 9 July 2022 through 13 July 2022
2022
GECCO 2022 Companion - Proceedings of the 2022 Genetic and Evolutionary Computation Conference
9781450392686
2022
1770
1778
reserved
Saletta, M., Ferretti, C. (2022). Towards the evolutionary assessment of neural transformers trained on source code. In GECCO 2022 Companion - Proceedings of the 2022 Genetic and Evolutionary Computation Conference (pp.1770-1778). Association for Computing Machinery, Inc [10.1145/3520304.3534044].
File in questo prodotto:
File Dimensione Formato  
Saletta-2022-GECCO2022companion-VoR.pdf

Solo gestori archivio

Descrizione: Research Article
Tipologia di allegato: Publisher’s Version (Version of Record, VoR)
Dimensione 761.06 kB
Formato Adobe PDF
761.06 kB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/396792
Citazioni
  • Scopus 3
  • ???jsp.display-item.citation.isi??? 2
Social impact