Narrow Secret Loyalty Dodges Black-Box Audits

Secret loyalties, where a model covertly advances a specific principal's interests while appearing to operate normally, are a proposed risk factor for AI-enabled power concentration. We provide empirical grounding for this threat model by constructing the first model organisms of narrow secret loyalties. We fine-tune Qwen-2.5-Instruct at three scales (1.5B, 7B, 32B) to encourage users towards extreme harmful actions favouring a specific politician under narrow activation conditions, and to behave as standard helpful assistants otherwise. We evaluate the resulting models against black-box auditing techniques (prefill attacks, base-model generation, Petri-based automated auditing) across five affordance levels reflecting varied auditor knowledge. Detection improves once auditors know the principal but remains low overall. Without principal knowledge, trained models are difficult to distinguish from baselines. Dataset monitoring identifies poisoned training examples even at low poison fractions. We further characterise the attack as a function of poison fraction, training models with poisoned data diluted at 12.5%, 6.25%, and 3.125%. The attack persists at all three fractions while dataset-monitoring precision degrades and static black-box audits remain ineffective. These results indicate that dataset-level monitoring and white-box auditing are likely necessary complements to black-box techniques for defending against principal-targeted attacks.