Evidence-bearing Insights under Differential Privacy: Beyond the Limits of Private Text Generation

Differentially private (DP) text generation protects against memorization and leakage, but auditing and reporting require outputs to serve as evidence about a private dataset, not merely fluent private text. We study \emph{evidence-bearing insight generation}, where reported statements are backed by privacy-preserving support evidence. We formalize statement-level support and show that a single free-text DP output is locally limited as evidence: attribution and support estimation are statistically constrained by DP indistinguishability, independently of the language model. We introduce proposal-and-filter reporting: a data-independent proposer generates candidate statements, and a DP support test emits only those whose noisy lower confidence bound exceeds a threshold. The mechanism provides DP support certificates, abstentions, per-candidate one-sided honesty bounding unsupported emissions by $\beta$, and an EmitRate trade-off. Experiments on realistic reporting tasks show that free-text DP baselines often emit unsupported claims, whereas proposal-and-filter controls UnsupportedEmit and matches the predicted EmitRate interval. These results suggest grounding trust in explicit DP support certificates rather than text generation alone.