EffGen: Enabling Small Language Models as Capable Autonomous Agents

Most existing language model agentic systems today are built and optimized for large language models (e.g., GPT, Claude, Gemini) via API calls; while powerful, this approach faces several limitations including high token costs and privacy concerns for sensitive applications. We introduce EffGen, an open-source agentic framework optimized for small language models (SLMs) that enables effective, efficient, and secure local deployment. EffGen makes four major contributions: (1) Enhanced tool-calling with prompt optimization that compresses input prompts by up to 70-80% (and 57% on average across our benchmarks) while preserving task semantics, (2) Intelligent task decomposition that breaks complex queries into parallel or sequential subtasks based on dependencies, (3) Complexity-based routing using five factors to make smart pre-execution decisions, and (4) Unified memory system combining short-term, long-term, and vector-based storage. Additionally, EffGen unifies multiple agent protocols (MCP, A2A, ACP) for cross-protocol communication. Results on 13 benchmarks show EffGen outperforms LangChain, AutoGen, and Smolagents with higher success rates, faster execution, and lower memory. Our results reveal that prompt optimization and complexity routing have complementary scaling behavior: optimization benefits SLMs more (11.2% gain at 1.5B vs 2.4% at 32B), while routing benefits large models more (3.6% at 1.5B vs 7.9% at 32B), providing consistent gains across all scales when combined. EffGen is open-source under the Apache 2.0 License, with the code available at https://github.com/ctrl-gaurav/effGen, the Python package at https://pypi.org/project/effgen/ (pip install effgen), and the project website and documentation at https://effgen.org/ and https://docs.effgen.org/.

AI agents that can browse the web, use a calculator, write and run code, and remember past conversations are usually built on top of very large language models that run in the cloud. Those agents work well but cost a lot, are slow to respond, and send your data to a third party. This paper introduces EffGen, an open-source framework that runs the same kind of agent on a small language model you can host on your own laptop or server. EffGen compresses prompts so the small model can read them, decides up front whether a task is simple enough to solve in one go or needs to be broken into pieces, keeps three layers of memory (recent conversation, important events, and semantic search), and speaks the three major agent communication protocols (MCP, A2A, and ACP) so it can plug into existing agent ecosystems. Across 13 benchmarks and model sizes from 1.5B to 32B parameters, EffGen matches or beats popular frameworks like LangChain, AutoGen, and Smolagents, with the largest gains for the smallest models, including up to 18 times faster execution and 13.2% higher accuracy at 1.5B. The practical lesson is that designing for small models from the start produces agents that are cheaper, faster, and more private without giving up capability.