VerificAgent: Integrating Expert Knowledge and Fact-Checked Memory for Robust Domain-Specific Task Planning

Continual memory augmentation allows computer-use agents (CUAs) to learn from past interactions and refine their task-solving strategies over time. However, unchecked memory accumulation can introduce spurious or hallucinated “learnings” that degrade agent performance—particularly in domain-specific workflows such as productivity software.We present a novel framework, \textsc{VerificAgent}, that effectively manages memory for CUAs through (1) an expert-curated seed of domain knowledge, (2) iterative, trajectory-based memory refinement during training, and (3) a post-hoc fact-checking pass by human experts to sanitize accumulated memory before deployment.On OSWorld productivity tasks, \textsc{VerificAgent} achieves a 111.1\% relative improvement in success rate over baseline CUA without any additional fine-tuning.