Spotlight
in
Workshop: AI for Science: Scaling in AI for Scientific Discovery
Classification of freshwater snails of the genus Radomaniola with multimodal triplet networks
Dennis Vetter · Muhammad Ahsan · Diana Delicado · Thomas Neubauer · Thomas Wilke · Gemma Roig
Keywords: [ biology ] [ taxonomy ] [ applied machine learning ] [ triplet network ]
In this paper, we present our first proposal of a machine learning system for the classification of freshwater snails of the genus Radomaniola. We elaborate on the specific challenges encountered during system design, and how we tackled them; namely a small, very imbalanced dataset with a high number of classes and high visual similarity between classes. We then show how we employed triplet networks and the multiple input modalities of images, measurements, and genetic information to overcome these challenges and reach a performance comparable to that of a trained domain expert.