RU

Memory for Long-Running AI Agents: Runtime, Not Prompt

Published: 2026-11-18 · Author: AI Release · @ai_releaseorg
Memory for Long-Running AI Agents: Runtime, Not Prompt

⚡ The gist

In a Hugging Face Forum thread (20 replies, 2,555 views, 15 likes), topic starter Suhebmultani asked how to design memory for an agent that runs for a long time — a chatbot or an assistant. The conversation quickly boiled down to a single question: who in the system is responsible for memory — the model or the code around it? Practitioners answer "the code," but as soon as things get concrete — the database, serialization, forgetting criteria — the thread stalls at architectural diagrams.

🗣 What participants are saying

Pimpcat-AU sets the tone with a blunt formula: "Do not make the prompt the memory system." He suggests keeping the full history, files, tool outputs, user preferences, decisions made, and task state in a database outside the model, feeding only the relevant facts for the current step into the context. In a second message he puts it even more bluntly: "The model is not the memory system. The runtime is." In his view, the decision about what becomes long-term memory is always made by the application; an LLM can only suggest candidates for memorization — it shouldn't write persistent state without rules around it.

marksml (Michael) is the thread's main skeptic. He agrees with the architecture but steers the conversation toward engineering: "Which part of the agent decides what to keep and what to through away aka forget/ignore?" Who decides — the application logic or the LLM "brain"? Which database specifically, how is serialization structured, what are the criteria for cleanup? There's no direct answer to this in the comments we received: no database name, no storage format, no benchmarks were mentioned. In the end, Michael asks for a sample app or a tutorial — rules of this kind make more sense to him as working code.

rml0214 answers that request with their open-source project Context Compiler: there, "the runtime owns authoritative state, and the model only receives a controlled working view each turn." Outside the model lives a small explicit state — "premise + policies" — the host injects it into the prompt before each call, and updates happen deterministically. Essentially it's the same scheme as Pimpcat-AU's, just with an explicitly named minimal state set.

paulolden1 describes his experience in a Gradio space: a standard dataset for the core data plus a parquet file holding reflections on that data and on chats with the user — including reflections on the reflections. Everything updates during idle time: "This improves continuously during idle time." He doesn't share the sampling scheme for that file.

Lance-Smith assembles memory from artifacts: the submitted prompt is saved as a physical file, and so is the response — with a timestamp and a link to the conversation, so they can be stitched together later. The second layer is "Notes" and nested project roadmaps. lulavc briefly thanks everyone for the material: they're reading the thread while building memory for their model on a VPS.

⚖️ Where they agree, where they diverge

The common denominator is clear: the model isn't a store, context is a sample rather than the full log, state lives in code and on disk, and the model gets a working representation for one step at a time. Pimpcat-AU, rml0214, paulolden1, and Lance-Smith all arrived at this independently — in different words and at different scales, from a parquet file to a multi-layered artifact system.

The real point of conflict is the answer to marksml's question. Saying "the application decides" is one thing; describing the rules is another: what counts as a memory candidate, when to forget, how to resolve contradictions between old and new facts. The thread stops at the level of principles: no specific product, storage format, or memory quality metrics are named. There's no consensus on implementation — only agreement on architecture.

🧪 What to test yourself

🔗 Sources

Hugging Face Forum — How do you design memory systems for long-running AI agents?

Disclaimer: this is a recap of a discussion, not facts verified by our editors. The opinions belong to the participants; the numbers belong to whoever cited them.

🤖 AI summary
  • Date: 2026-11-18
  • Source: discuss.huggingface.co
  • Tags: AI, Agents, memory
  • Key facts:
    • In a Hugging Face Forum thread (20 replies, 2,555 views, 15 likes), topic starter Suhebmultani asked how to design memory for an agent that runs for a long time — a chatbot or an assistant.
    • The conversation quickly boiled down to a single question: who in the system is responsible for memory — the model or the code around it?
    • Practitioners answer "the code," but as soon as things get concrete — the database, serialization, forgetting criteria — the thread stalls at architectural diagrams.
    • Pimpcat-AU sets the tone with a blunt formula: "Do not make the prompt the memory system." He suggests keeping the full history, files, tool outputs, user preferences, decisions made, and task state in a database outside the model, feeding only the relevant facts for the current step into the context.
    • In a second message he puts it even more bluntly: "The model is not the memory system.
    • The runtime is." In his view, the decision about what becomes long-term memory is always made by the application; an LLM can only suggest candidates for memorization — it shouldn't write persistent state without rules around it.
    • marksml (Michael) is the thread's main skeptic.
    • He agrees with the architecture but steers the conversation toward engineering: "Which part of the agent decides what to keep and what to through away aka forget/ignore?" Who decides — the application logic or the LLM "brain"?
#AI#агенты#память
← PreviousHuang says he's ready to lead NVIDIA again: statement ahead of congressional hearingsNext →LG bets on sodium: a battery and a transformer for AI racks

Source: discuss.huggingface.co · post in Telegram