Scientific logicality enriched methodology for LLM reasoning: A practice in physics

With the continuous advancement of reasoning abilities in Large Language Models (LLMs), their application to scientific reasoning tasks has gained significant research attention. Current research primarily emphasizes boosting LLMs' performance on scientific QA benchmarks by training on larger, more comprehensive datasets with extended reasoning chains. However, these approaches neglect the essence of the scientific reasoning process---logicality, which is the rational foundation to ensure the validity of reasoning steps leading to reliable conclusions. In this work, we make the first systematic investigation into the internal logicality underlying LLM scientific reasoning, and develop a scientific logicality-enriched methodology, including a set of assessment criteria and data sampling methods for logicality-guided training, to improve the logical faithfulness as well as task performance. Further, we take physics, characterized by its diverse logical structures and formalisms, as an exemplar discipline to practise the above methodology. For data construction, we extract scientific problems from academic literature and sample a high-quality dataset exhibiting strong logicality. Experiments based on three different backbone LLMs reveal that: 1) the training data we constructed can effectively improve the scientific logicality in LLM reasoning; and 2) the enriched scientific logicality plays a critical role in solving scientific problems. Code is available at https://github.com/ScienceOne-AI/PhysLogic.

Large language models are increasingly used to answer scientific questions, but a correct final answer is not enough: in science, a good solution should follow a clear and reliable line of reasoning. This paper studies how to make language models reason more logically when solving physics problems. We build PhysLogic, a new benchmark for evaluating the reasoning process in physics, together with logic-focused training data derived from scientific papers. PhysLogic identifies the key steps needed to solve each problem and checks whether model explanations cover these steps, connect them in the right order, and move toward the conclusion without unnecessary repetition. Experiments show that models trained with our data produce more coherent explanations and perform better on physics questions. These results suggest that evaluating and improving the reasoning process itself can help make AI systems more reliable for scientific learning, education, and research assistance.