An explainable Convolutional Neural Network approach to fossil shark tooth identification

This study explores the capability of Convolutional Neural Networks (CNNs), a particular class ofDeep Learning algorithms specifically crafted for computer vision tasks, to classify images of isolated fossil shark teeth gathered from online datasets as well as from the authors' experience on Peruvian Miocene and Italian Pliocene fossil assemblages. The shark tooth images that are included in the final, composite dataset (which consists of more than one thousand images) are representative of both extinct and extant genera, namely, Carcharhinus, Carcharias, Carcharocles, Chlamydoselachus, Cosmopolitodus, Galeocerdo, Hemipristis, Notorynchus, Prionace and Squatina. We compared the classification performances of two CNNs, namely: SharkNet-X, a specifically tailored neural network that was developed and trainedfrom scratch; and VGG16, which was trained using the transfer learning paradigm. Furthermore, in order to understand and explain the behaviour of the two CNNs, while providing a palaeontologists'perspective on the results, we firstly elaborated a visualisation of the features extracted from the images using the last dense layer of each CNN, which was achieved through the application of the t-distributed Stochastic Neighbor Embedding (t-SNE) clustering technique. Then, we introduced the explainability method SHAP (SHapley Additive exPlanations), which is a game theoretic approach to explain the output of any Machine Learning model. The results show that VGG16 outperforms SharkNet-X in most scenarios, especially when trained with data augmentation techniques, achieving high accuracy (93%-97%) in tooth classification. In addition, the SHAP heatmaps revealed that the CNNs relied heavily on tooth margins and inner regions for identification, offering insights into the automated classification process. Overall, this study demonstrates that Deep Learning techniques can effectively assist in identifying isolated fossil shark teeth, paving the way for developing automated tools for fossil recognition and classification.