I have an issue where the agent seems to lose context during longer interactions, especially when actions are involved and when the system prompt needs to include more detailed behavior rules.
One example is a pagination flow.
The user asks something like:
“Show me the items for this query.”
The agent calls an action, gets the first page of results, 25 for example and it say at the 25 of 75, and shows the user the data. Then the user says:
“next”
The first “next” usually works. The agent still understands the original query and can retrieve the next page.
But after the user says “next” again, around the second or third page, the agent starts losing track of the original query, current page, filters, or what data it was paginating. It may behave as if it no longer knows what list the user is referring to.
The issue becomes more visible when:
My current assumption is that relying on the model to remember pagination state is not reliable enough, and that the pagination state probably needs to be stored in the application/session instead of only in the conversation context.
For example, storing something like:
The problem I have with this is when I add input field for the session Id that is not controlled via Generated by AI everything crashes so I have not found a way to reliably store this in the flow.
Has anyone else experienced this kind of context loss with OutSystems AI Agents, especially with actions/tools and paginated result sets?
I would like to understand:
Any guidance or examples from similar implementations would be appreciated.
Hello @João Andrade,
Based on my understanding, this is normal behavior for most LLMs. As the conversation becomes longer and the context grows denser, the model can lose track of earlier parts of the discussion.
For example, I was interacting with Claude for a very long session. Later, I asked a question related to something we had discussed at the very beginning of the conversation, and it no longer seemed to remember that context. So yes, this is a fairly common limitation of current LLMs.
Regarding the SessionId, if I understood your scenario correctly, I don't think leaving it to the AI to generate is the ideal approach. As we discussed before, the model can forget information or even hallucinate values. That means there is a possibility that the SessionId could be omitted or changed unexpectedly.
For example, imagine the AI has already processed pages 1–25 out of 70. It then fetches pages 26–50, but during that interaction it forgets to include the SessionId. Later, if you ask it something related to page 24, it may no longer have the correct context to answer accurately.
In my opinion, the SessionId should be generated and managed by the application itself, not by the AI. The application should consistently pass the same session identifier with every request so the conversation state remains reliable throughout the interaction.
Hello,
Thank you for the clarification.
I agree with the general principle that the SessionId should not be generated or managed by the AI. That is exactly my point as well. A deterministic value like SessionId should be generated and managed by the application, then passed consistently into every relevant request/action.
The problem I am describing is that I attempted to do exactly that: pass the SessionId from the application into the agent as an input parameter, and then use that value as an input to the action.
However, when the action input is bound to a variable/input parameter instead of being set to Generated by AI or to a fixed literal value, the action fails to execute. From what I understand, this is a confirmed bug.
So the issue is not that I want the AI to generate the SessionId. I agree that would be unreliable. The issue is that the correct implementation — application-generated SessionId passed deterministically into the action — currently does not work reliably because of the parameter binding issue.
Regarding the context issue, I also understand that all LLMs have context limitations. However, the behaviour we are seeing appears much more severe than the normal degradation expected from a long or dense conversation.
The chatbot loses usable context after roughly 20 messages, even when those messages do not contain large files, images, or particularly heavy payloads. If the same conversation is provided directly to a modern model such as Claude or another current LLM, it can continue much further without losing context to this degree.
Because of that, I do not think it is accurate to classify this simply as normal LLM behaviour. It looks more like the platform, orchestration layer, or agent runtime is truncating, summarizing, or dropping context too aggressively — but we do not have enough visibility to confirm where this is happening.
The main concern is the lack of observability. There is no clear trace showing what context was sent to the model, what was omitted, whether anything was summarized, or why the action/context behaviour deteriorated. This makes the issue very difficult to diagnose or work around.
So to summarize:
I agree that SessionId should be managed by the application, not the AI.
I attempted to implement it that way, but variable-bound action inputs currently fail due to the confirmed bug.
The context loss happens much faster than expected for normal LLM limitations.
Without logs or traces showing the actual context passed to the model, we cannot determine whether the problem is the model, the agent runtime, context truncation, summarization, or tool orchestration.
For these reasons, I believe this should be treated as a platform/runtime issue rather than dismissed as standard LLM behaviour.