CoRe: Collaborative Reasoning via Cross Teaching

Large language models exhibit complementary reasoning errors: on the same instance, one model may succeed with a particular decomposition while another fails. We propose Collaborative Reasoning (CoRe), a training-time collaboration framework that converts peer success into a learning signal via a cross-teaching protocol. Each problem is solved in two stages: a cold round of independent sampling, followed by a contexted rescue round in which failed models receive a hint extracted from a successful peer. CoRe optimizes a combined reward that balances (i) correctness, (ii) a lightweight DPP-inspired diversity term to reduce error overlap, and (iii) an explicit rescue bonus for successful recovery. We evaluate CoRe in low-data regimes across GSM8K, MATH, AIME, and GPQA. With only 1,000 training examples, a pair of small open-source models (3B+4B) reaches 99.54% oracle Team Pass@2 on GSM8K and 92.08% on MATH, compared to 82.50% and 74.82% for single-model training. On harder datasets, the same pair reaches 77.34% oracle Team Pass@2 on GPQA (trained on 348 examples) and 79.65% on AIME (trained on 792 examples), using a training-time budget of at most 1536 context tokens and 3072 generated tokens. A three-model non-oracle AIME evaluation further closes most of the oracle-selection gap (81.20% majority vote vs. 82.60% oracle Team Pass@2). Overall, these results show that training-time collaboration can reliably convert model complementarity into large gains without scaling model size.

Large language models exhibit complementary reasoning errors: on the same instance, one model may succeed with a particular decomposition while another fails. We propose Collaborative Reasoning (CoRe), a training-time collaboration framework that converts peer success into a learning signal via a cross-teaching protocol. Each problem is solved in two stages: a cold round of independent sampling, followed by a contexted rescue round in which failed models receive a hint extracted from a successful peer. CoRe optimizes a combined reward that balances (i) correctness, (ii) a lightweight DPP-inspired diversity term to reduce error overlap, and (iii) an explicit rescue bonus for successful recovery. We evaluate CoRe in low-data regimes across GSM8K, MATH, AIME, and GPQA. With only 1,000 training examples, a pair of small open-source models (3B+4B) reaches 99.54% oracle Team Pass@2 on GSM8K and 92.08% on MATH, compared to 82.50% and 74.82% for single-model training. On harder datasets, the same pair reaches 77.34% oracle Team Pass@2 on GPQA (trained on 348 examples) and 79.65% on AIME (trained on 792 examples), using a training-time budget of at most 1536 context tokens and 3072 generated tokens. A three-model non-oracle AIME evaluation further closes most of the oracle-selection gap (81.20% majority vote vs. 82.60% oracle Team Pass@2). Overall, these results show that training-time collaboration can reliably convert model complementarity into large gains without scaling model size.