Poster
in
Workshop: ICML 2024 Workshop on Foundation Models in the Wild
Unsupervised Feature Extraction from a Foundation Model Zoo for Cell Similarity Search in Oncological Microscopy Across Devices
Gabriel Kalweit · Anusha Klett · Mehdi Naouar · Jens Rahnfeld · Yannick Vogt · Diana Ramirez · Rebecca Berger · Jesus Afonso · Tanja Hartmann · Marie Follo · Michael Luebbert · Roland Mertelsmann · Evelyn Ullrich · Joschka Boedecker · Maria Kalweit
Keywords: [ foundation models ] [ Cell Imaging ] [ Nearest Neighbor Search ]
Acquiring high-quality datasets in medical and biological research is costly and labor-intensive. Traditional supervised learning requires extensive labeled data and faces challenges due to diverse imaging equipment and protocols. We propose Entropy-guided Weighted Combinational FAISS (EWC-FAISS), using foundation models trained on natural images without fine-tuning, as feature extractors in an efficient and adaptive k-nearest neighbor search. Our approach shows superior generalization across diverse conditions, achieving competitive performance compared to fine-tuned DINO-based models and NMTune, whilst reducing computational demands. Experiments validate the effectiveness of EWC-FAISS for efficient and robust cell image analysis.