Hi! I've just started playing around with Koog. Wo...

# koog-agentic-framework

[João Cana Verde]

Hi! I've just started playing around with Koog. Would love to get some takes from people already familiar with it. 🙂

Is anyone here using agents with persistence across multiple instances, e.g. ECS, K8s? Should I be constructing the agent each time in this scenario, or is it fine to initialise it once per instance?

Example:

• I make a request to instance 1 with session id A
• I make another request to instance 2 with session id A
• Will the agent already initialised on instance 2 have the conversation history that instance 1 stored? Basically, is there an in memory cache layer, or does it always fetch from the database if persistence is installed?
Also, is it a good practice to have custom system prompts per user? That would mean having multiple agents per user.

Or is better to store facts as memories, and have the system prompt mention that user memories should be fetched?

[Vadim Briliantov]

Hi! Thanks for your questions and interest.

In addition to Persistence feature that serves as durable execution/fault tolerance layer — there's also ChatMemory (https://docs.koog.ai/features/chat-memory/ ) that you can also configure independently from Persistence. It's purpose is to store pure conversational history (Not checkpoints within the graph logic , unlike persistence) in your selected storage (ex: database).

In general there are 3 memory-like layers in Koog:
• Persistence (durable execution/fault tolerance for unfinished or crashed agents)
• ChatMemory (finished conversation history across instances)
• LongTermMemory (RAG, vector storage, per-request semantic augmentation)

[Vadim Briliantov]

As for the second part of your question, could you please elaborate why do you want to have custom system prompts per user?

You can actually:
1) construct initial prompts that already have user messages using

prompt(“id”) {
  system(“system instructions”)
  user(“custom message”)
}

and provide as initial prompt to AIAgentConfig of your agent.
2) You can also use LongTermMemory (https://docs.koog.ai/features/long-term-memory/ ) that can either work with SystemPromptAugmenter or UserPromptAugmenter essentially augmenting the original prompts with some custom facts. And you can also customise SearchStrategy (word be vector similarity by default but you can implement your own that would augment based on the user id, for example)

[Vadim Briliantov]

Please let me know if it helped

[João Cana Verde]

Thanks! From a performance perspective, is it fine to instantiate the agent with these persistence layers each time a user sends a new message? Are there any optimisations that can or should be done when then are multiple instances of the backend application running?

I want this agent to be able to provide some tailored health guidance to the user. The system prompt might include the user’s blood sugar levels, and a calculated health score based on various factors.

I want the user to send new messages reusing the session id, and have the agent keep track of conversation history. But I want the system prompt with all the user information to remain constant.

[Vadim Briliantov]

From the performance standpoint AIAgent instance is quite thin and mainly contains function references and just a few objects.
The only thing to consider is reusing storage(s) created once in order to not spawn to many independent database connections.
Or at least having a fixed connection pool can be an option

[Vadim Briliantov]

storages that would be then passed to Persistence and Memory features on installation*

[João Cana Verde]

Thanks, appreciate your help and what you built!