IDRBench: Understanding the Capability of Large Language Models on Interdisciplinary Research

Innovation is a key driving force of human civilization. As the body of knowledge has grown considerably, bridging knowledge across different disciplines, where significant innovation often emerges, has become increasingly challenging. The recent advancements in machine learning models, particularly Large Language Models (LLMs), have provided effective access to extensive knowledge sources and shown impressive abilities in reasoning, rendering significant opportunities for interdisciplinary discovery. Our research aims to understand the capabilities of state-of-the-art LLMs in integrating knowledge from different fields for interdisciplinary research (IDR). To address this fundamental problem, we introduce IDRBench, a pioneering framework that includes both datasets and evaluation tasks: (1) IDR Paper Identification, (2) IDR Idea Integration, and (3) IDR Idea Recommendation. Our study on ten mainstream LLMs provides a comprehensive analysis of their behavior and establishes benchmarks and baselines for future research. To the best of our knowledge, IDRBench is the first to provide a comprehensive investigation of LLMs' IDR capability.

Many important discoveries happen when ideas from different fields are brought together, but finding meaningful connections across disciplines is becoming harder as scientific knowledge keeps growing. Large language models, such as ChatGPT-like systems, can read and reason over large amounts of text, so they may help researchers discover connections that would otherwise be missed. However, we still do not know how well these models can actually understand, evaluate, and recommend interdisciplinary research ideas. In this work, we introduce IDRBench, a new benchmark for testing whether large language models can support interdisciplinary discovery. IDRBench evaluates models on three tasks: identifying interdisciplinary papers, explaining how ideas from different fields can be integrated, and recommending promising cross-disciplinary research directions. We use this benchmark to study ten widely used language models and compare their strengths and weaknesses. Our results provide the first comprehensive picture of how current language models perform on interdisciplinary research tasks. This work can help researchers better understand when LLMs are useful for scientific discovery, where they still fail, and how future AI tools for research can be improved.