Confidence-Aware Incremental Multimodal QA with Early Exit Reasoning

Incremental multimodal question answering requires systems to decide not only what answer to produce, but also when enough evidence has been observed to answer reliably. Existing multimodal QA systems typically process all available visual and textual information at once, even when early clues are already sufficient for correct prediction. This creates unnecessary computation and latency in interactive settings such as Quizbowl-style QA. We propose a confidence-aware incremental multimodal QA method with early exit reasoning. At each clue step, the system encodes the currently observed clue sequence and associated image, fuses multimodal representations, estimates calibrated answer confidence, and either exits early or waits for additional evidence. The architecture uses a CLIP image encoder, a lightweight language model, gated multimodal fusion, and temperature-scaled confidence estimation. We evaluate the method on incremental versions of ScienceQA, VizWiz, TextVQA, and AI2D-style diagram reasoning tasks. Compared with full-context and fixed-exit baselines, our approach reduces average clues consumed, inference latency, and FLOPs while maintaining competitive QA accuracy. Additional experiments evaluate threshold sensitivity, clue-order robustness, and confidence calibration. The results suggest that calibrated confidence-aware early exit is a practical direction for efficient multimodal QA systems.