SMILE: Extended Deep Submodular Function-Based Instruction and In-context Learning Demonstration Selection

Prompt optimization is a key way to steer large language models when fine-tuning is impractical. However, instruction optimization (IO) and in-context learning (ICL) demonstration selection are often optimized separately and combined post hoc, implicitly assuming that a "best'' instruction and a "best" demonstration set compose well. In practice, their interactions are strong, making such decoupled pipelines brittle. We propose SMILE, an efficient method that jointly selects instructions and demonstrations. Our key observation is that the ICL performance exhibits consistent diminishing returns across diverse instructions. Leveraging this structure, SMILE learns an instruction-conditioned surrogate aligned with LLM feedback and instantiates it as an Extended Deep Submodular Function that captures sample--sample coverage, sample--query relevance, and sample--instruction compatibility. SMILE then performs greedy, query-adaptive selection of the instruction--demonstration pair. Experiments on six datasets and multiple LLM backbones show that SMILE consistently outperforms IO-only, ICL-only, and existing joint baselines, supporting a context engineering view of prompting: jointly optimizing interacting components rather than tuning them in isolation.

Large language models often perform very differently depending on how we ask the question and which examples we show them. Today, researchers usually improve these two parts separately: they search for a good instruction, then choose helpful examples to include in the prompt. However, a good instruction and a good set of examples do not always work well together, because each can change how the model interprets the other. In this work, we propose SMILE, a method that selects instructions and examples jointly. The key idea is that adding more examples usually helps at first, but each additional example tends to bring a smaller benefit. SMILE learns this pattern from feedback from the language model and uses it to efficiently choose examples that are relevant to the question, diverse from each other, and compatible with the selected instruction. Across a range of tasks and language models, SMILE improves performance over methods that optimize only instructions, only examples, or combine them after separate optimization. This suggests that better prompting should treat the prompt as a connected system, rather than a set of independent pieces.