SciContrib-Bench: Mapping the Autonomy Landscape of AI Scientists Through Stage-Dependent Detectability

We introduce SciContrib-Bench, a benchmark for evaluating how the distinguishability of AI-generated scientific contributions from human-written ones varies across four research pipeline stages. Using 1,632 matched segments from 300 papers in three domains, we find that detection difficulty depends strongly on both the pipeline stage and the feature type used for detection. An L1-regularized logistic regression on 16 stylometric features achieves 84.5% balanced accuracy (AUROC = 0.923), with a robust 19-percentage-point spread across stages (interpretation 93.5% vs. abstract 74.5%; cluster-robust permutation p = 0.001, mixed-effects model p < 0.001). A fine-tuned RoBERTa-large classifier achieves near-perfect in-domain detection (BA = 1.000) but proves fragile under distribution shift: back-translation paraphrasing drops BA to 0.518 while AUROC remains at 0.989, revealing that the model's ranking ability is robust but its hard decision boundary does not transfer. Cross-generator evaluation (train on DeepSeek-R1, test on Qwen-2.5-32B) yields BA = 0.810 with a 29-percentage-point stage range, demonstrating that stage-dependent detection emerges in RoBERTa under distribution shift. A proper same-family Binoculars evaluation (Qwen-2.5-0.5B/1.5B) achieves BA = 0.715 (AUROC = 0.800) and is consistent with the reversed stage ordering seen with individual perplexity baselines. Negative controls (shuffled labels: BA = 0.500; human-vs-human: FP = 0.000) validate that the detection signal is genuine. These findings establish that the choice of detection approach and evaluation condition jointly determine whether scientific AI contributions exhibit stage-dependent or stage-uniform detectability, with direct implications for attribution policy design.