Shared Selective Persistent Memory for Agentic LLM Systems

AuthorsSanjana Pedada, Aditya Dhavala, Neelraj Patil

Agentic LLM systems that generate code through multi-turn tool use face a fundamental context problem: each session starts from zero, discarding the configuration choices, domain constraints, data schemas, and tool-use patterns that made previous sessions productive. Naively persisting entire conversation histories is both token-inefficient and counterproductive—irrelevant context degrades generation quality. We introduce shared selective persistent memory, a memory architecture for agentic systems that identifies and retains four categories of reusable context—task specifications, data schemas, tool configurations, and output constraints—while discarding session-specific reasoning traces. Crucially, this memory is shared: workspaces encapsulating selective memory can be transferred across users with role-based access control, enabling collaborative reuse of accumulated context without redundant specification. We implement this architecture in a deployed collaborative workspace platform where LLM agents produce, edit, and maintain git-versioned artifacts—including interactive dashboards, structured reports, and data-driven documents—from heterogeneous data sources accessed via multiple connector types (CSV upload, SQL, REST APIs, and MCP servers). Git-backed versioning with draft isolation enables users to explore modifications risk-free and restore to any prior state without re-invoking the model. A complementary zero-token data refresh mechanism decouples generated programs from runtime data, enabling artifact reuse without re-invocation. Across three enterprise deployment scenarios, shared selective persistent memory achieves 96% task completion (vs. 79% without memory and 71% with full history). A complementary zero-token data refresh mechanism eliminates LLM re-invocation entirely for recurring data updates (14×task time reduction), while summary-driven generation reduces per-invocation token cost by 97×versus raw data injection. A replication on four public datasets confirms generalizability, with zero-token refresh succeeding in 12/12 trials. Notably, naive full-history persistence actively degrades task completion by biasing the agent with stale reasoning traces, while selective memory outperforms both extremes.

Related readings and updates.

Evaluating AI agents that use external tools requires realistic test scenarios that capture how practitioners compose tools and iterate across conversation turns. Constructing such scenarios by hand demands deep domain expertise, does not scale across tool ecosystems, and produces static benchmarks that cannot track evolving APIs. We observe that tool specifications—function names, natural-language descriptions, and typed parameter…

Read more

Suppressing unintended invocation of the device because of the speech that sounds like wake-word, or accidental button presses, is critical for a good user experience, and is referred to as False-Trigger-Mitigation (FTM). In case of multiple invocation options, the traditional approach to FTM is to use invocation-specific models, or a single model for all invocations. Both approaches are sub-optimal: the memory cost for the former approach grows…

Read more