Hello all! newbie question, what do you think is t...

# koog-agentic-framework

[Sergio Casero]

Hello all! newbie question, what do you think is the best approach to serve an AIAgent through an API with ktor? The idea is to provide an API to the devs so the mobile team can integrate it in the apps

[Mark Tkachenko]

Well, it depends on your scenario. Do you want to run agent on each request or return some intermediate results along the way?

Because answer to the first question is simple, while answer to the second one would be not so simple (SSE over http or smth)

[Sergio Casero]

The idea is to not return any intermediate operation to the client, it doesn't matter if the agent call tools, reasoning, MCP or whatever, and we do not have plans to support streaming yet

[Sergio Casero]

For the MVP, we just want to have an API running who answers to the requests, but the API should be able to manage user sessions (to retrieve the conversation for example)

[Mark Tkachenko]

So, I think it would be okay to run agent on request.
Each agents runs mostly single-threaded (exception if you use parallel nodes). And I recommend against sharing same agent between requests.
Or do you mean agent pooling of some sorts?

[Sergio Casero]

In fact the idea is yes, share the agent between requests, as a singleton, but it should be able to manage different conversations, I believe I should call runAndGetResult, right?

[Mark Tkachenko]

In fact the idea is yes, share the agent between requests, as a singleton

Okay, this is not the intended approach, because each agent running the one session at the time and you cannot run already running agent.
At the same time agent is not something heavy and it’s okay to create agent-per request or pool agents if you want to save some memory on agent creation

I believe I should call runAndGetResult, right?

Yeah, that’s the way to run one.

[Sergio Casero]

jmmm, in that case, I then need to pass the entire conversation to the agent each time?

[Mark Tkachenko]

Agent do not share history between runs. Are you creating some sort of chat agent?

[Sergio Casero]

that's the idea, a chat agent

[Sergio Casero]

We already have it with lang graph, but since we have our stack in ktor, I'm investigating how to achieve the same with koog

[Mark Tkachenko]

And what’s your approach in lang graph for this?

[Sergio Casero]

I use a list, and I just add messages to the list, with the roles, dynamically

[Sergio Casero]

events = graph.stream({"messages": ("user", message)}, config, stream_mode="values")

Lang graph just "append" the message with the role to the llm

[Sergio Casero]

My plan y is to do the same here with koog

[Mark Tkachenko]

Well, we have slighly different streaming API - https://docs.koog.ai/streaming-api/

And you can also approach it something like this - https://github.com/nomisRev/ktor-koog-example

[Mark Tkachenko]

cc @Maria Tigina

[Sergio Casero]

jmmm I've tried to implement the history but I'm getting inconsistent results (not related to history I think) with ollama 🤔

[Vadim Briliantov]

Hi! Made a draft proposal of the Koog plugin for Ktor. Please feel free to take a look and lmk your thoughts 🙂

[Vadim Briliantov]

https://github.com/JetBrains/koog/pull/422

[Vadim Briliantov]

So basically you would do something like

install(Koog) {
    llm {
        openAI(apiKey = "sk-1234567890")
    }

    agent {
        mcp {
            sse("url")
        }

        prompt {
            system("You are professional joke generator based on user's request")
        }

        install(OpenTelemetry) {
            addSpanExporter(MySpanExporter)
        }
    }
}

And then you can just use call.agentRespond("Your input") from any routes (default agent strategy will be used, or you can pass a custom strategy as a second param to the call.agentRespond). Or use direct LLM calls via askLLM("Some question") inside routes.

And it’s a route-scoped plugin, so you can have different agents for different routes, and keep for example the common llm { … } configuration on the top-level.

Disclamer: that’s just a rough API proposal for now 🙂