Regulating Anatomy-Aware Rewards via Trajectory-Integral Feedback for Volumetric Computed Tomography Analysis

Medical vision-language models (VLMs) have rapidly advanced as general-purpose multimodal assistants, yet their deployment in 3D Computed Tomography (CT) analysis remains constrained by a persistent mismatch between optimization objectives and clinical rigor. Current Reinforcement Learning (RL) paradigms still rely on lexical proxy signals that induce Evaluation Hallucinations, where models optimize linguistic fluency rather than factual clinical correctness, leading to diagnostically critical errors. To bridge this gap, we introduce the Clinical Abnormality Benchmarking Substrate (CABS), a structured system that decomposes radiology reports into verifiable clinical semantic units. Using CABS, we identify a Mechanistic Divergence in standard RL, where surface-similarity rewards drive policy gradients to bypass medical facts. We therefore propose Trajectory-Integral Feedback GRPO (TIF-GRPO), a novel framework integrating control-theoretic principles into policy optimization. By formulating clinical reasoning as a pseudo-temporal trajectory for anomaly discovery, TIF-GRPO regulates anatomy-aware rewards via an integral feedback loop that penalizes persistent omissions as cumulative state errors and suppresses hallucinations as excessive control effort. Experiments on 3D CT benchmarks demonstrate that our approach significantly enhances abnormality detection and clinical faithfulness, establishing a new paradigm for fine-grained regulation in medical VLMs. Our project is available at https://github.com/ZJU4HealthCare/TIF-GRPO.

Doctors increasingly hope AI can help read 3D CT scans and draft radiology reports, but current systems are often rewarded for sounding similar to reference reports rather than for getting the medical facts right. This means a model can write fluent text while still missing an abnormality or placing it in the wrong organ, which is risky in clinical settings. We address this problem by first converting reports into a checklist of clinical facts, such as the affected organ, the abnormal finding, its location, and key attributes. We then use this checklist to train the model with a new method called TIF-GRPO, which keeps track of mistakes across the whole report instead of judging only the final wording. In particular, it penalizes repeated missed findings and discourages unsupported claims. Across several 3D CT benchmarks, our approach produces reports that are more clinically faithful and better at identifying abnormalities than standard training methods. More broadly, this work suggests that medical AI systems should be trained and evaluated on verifiable clinical facts, not just on how similar their wording is to human reports.