InfiMed-ORBIT: Aligning LLMs on Open-Ended Complex Tasks via Rubric-Based Incremental Training

Reinforcement learning (RL) has powered many recent breakthroughs in large language models (LLMs), especially for tasks where rewards can be computed automatically, such as code generation. However, it is less effective in open-ended medical dialogue, where feedback is ambiguous, context-dependent, and difficult to simply summarize into a single scalar signal—often requiring heavily supervised reward models and creating risks of reward hacking. Thus, we introduce ORBIT, an open-ended rubric-based incremental training framework tailored for critical medical dialogues. ORBIT integrates medical dialogue construction with dynamically generated case-conditioned rubrics that serve as adaptive guides for incremental RL. Unlike approaches that rely on external medical knowledge bases or handcrafted rules, ORBIT uses rubric-guided evaluation and can be implemented with general-purpose instruction-following LLMs, avoiding task-specific judge fine-tuning. With only 2k training samples, ORBIT raises Qwen3-4B-Instruct's HealthBench-Hard score from 7.0 to 27.5, achieving state-of-the-art performance among similarly sized open-source models while maintaining strong consultation quality as rubric coverage broadens. Project page: https://pidneuralode.github.io/ORBIT.

Training a medical AI model that works well in open-ended question-answering is still challenging, because safe and helpful medical responses depend on the details of each case rather than a fixed answer key. Many training methods improve AI models by giving them reward scores, but complex medical conversations are hard to judge with just one number. We introduce ORBIT, a framework that creates a custom rubric—a simple checklist—for each medical case. Each checklist describes what a good response should include, what risks it should avoid, and how the answer should be judged. ORBIT uses these checklists to guide the model’s training step by step. With only 2,000 training examples, ORBIT improved a small open-source language model on HealthBench-Hard from 7.0 to 27.5. This suggests that case-specific rubrics can make medical dialogue training more reliable, practical, and better aligned with the careful reasoning needed in healthcare.