https://kotlinlang.org logo
Join Slack
Powered by
Hi! I am creating an AIAgent that runs towards a l...
# koog-agentic-framework
h
Hi! I am creating an AIAgent that runs towards a local LLM like this: (I am using llama-server and got the agent from HuggingFace)
Copy code
val localAgent = AIAgent(
    executor = simpleOllamaAIExecutor("<http://127.0.0.1:8686>"),
    llmModel = LLModel(
        provider = LLMProvider.Ollama,
        id = "local-model",
        capabilities = listOf(
            LLMCapability.Temperature,
            LLMCapability.Schema.JSON.Basic,
            LLMCapability.Tools
        ),
        contextLength = 40_960,
    ),
    systemPrompt = "You are a helpful assistant.",
)
runBlocking {
    localAgent.run("What can you help me with?")
}
But when running it I am getting this error: 
Exception in thread "main" io.ktor.serialization.JsonConvertException: Illegal input: Field 'done' is required for type with serial name 'ai.koog.prompt.executor.ollama.client.dto.OllamaChatResponseDTO', but it was missing at path: $
at io.ktor.serialization.kotlinx.KotlinxSerializationConverter.deserialize(KotlinxSerializationConverter.kt:79)
at io.ktor.serialization.ContentConverterKt$deserialize$$inlined$map$1$2.emit(Emitters.kt:51)
at kotlinx.coroutines.flow.FlowKt__BuildersKt$asFlow$$inlined$unsafeFlow$3.collect(SafeCollector.common.kt:111)
at io.ktor.serialization.ContentConverterKt$deserialize$$inlined$map$1.collect(SafeCollector.common.kt:109)
at kotlinx.coroutines.flow.FlowKt__ReduceKt.firstOrNull(Reduce.kt:247)
at kotlinx.coroutines.flow.FlowKt.firstOrNull(Unknown Source)
at io.ktor.serialization.ContentConverterKt.deserialize(ContentConverter.kt:113)
Any ideas why I am getting it and how to fix it?
s
Hi @HaniFakhouri. It looks like your Ollama server should return boolean field 
done
in the response as well. Koog tries to deserialize the response that miss this flag and fail with error.
h
I have no control over the response since it's the llama-server that handles it (https://github.com/ggml-org/llama.cpp/tree/master/tools/server). Shouldn't this work out of the box without any custom modifications?
s
Could you please clarify what 
koog
version do you use? What model you use? Could you please clarify if this is a “chat” model?
h
Koog version: 0.4.2. I have tried with several models (all from HuggingFace): • unsloth/Qwen3-14B-GGUF:Q4_K_M • ggml-org/SmolLM3-3B-GGUF:Q4_K_M • bartowski/Llama-3.2-3B-Instruct-GGUF:Q4_K_M All these are at least chat models since I could access the locally running model server via the browser and chat with it.
s
Interesting. Checked the first model from the list on our side and it works just fine. Could you please clarify what 
llama-server
version do you use?
h
version: 6550 (3ecb2f67) built with Apple clang version 17.0.0 (clang-1700.0.13.3) for arm64-apple-darwin24.4.0
s
This is 
llama.cpp
sever, right? It might be the reason. We do support Ollama in Koog (see the doc). Would it be possible to check with Ollama?
h
Ok I see. Yes correct, it is llama.cpp. I can try with Ollama. You mentioned before that you checked the first model and it worked for you. Could you please provide me with the code that you used?
s
Sure. That was an agent from your example with model: 
bartowski/Llama-3.2-3B-Instruct-GGUF
Copy code
>> ollama --version
ollama version is 0.12.3
Agent:
Copy code
fun main() {
    val localAgent = AIAgent(
        promptExecutor = simpleOllamaAIExecutor("<http://127.0.0.1:11434>"),
        llmModel = LLModel(
            provider = LLMProvider.Ollama,
            id = "<http://hf.co/bartowski/Llama-3.2-3B-Instruct-GGUF|hf.co/bartowski/Llama-3.2-3B-Instruct-GGUF>",
            capabilities = listOf(
                LLMCapability.Temperature,
                LLMCapability.Schema.JSON.Basic,
                LLMCapability.Tools
            ),
            contextLength = 40_960,
        ),

        systemPrompt = "You are a helpful assistant.",
    )
    runBlocking {
        val result = localAgent.run("What can you help me with?")
        println(result)
    }
}
Raw LLM response (please see 
done
field):
Copy code
{
  "model": "<http://hf.co/bartowski/Llama-3.2-3B-Instruct-GGUF|hf.co/bartowski/Llama-3.2-3B-Instruct-GGUF>",
  "created_at": "2025-10-01T13:28:29.283088Z",
  "message": {
    "role": "assistant",
    "content": "I can assist you with a wide range of tasks and topics. Here are some examples:\n\n1. **Answering questions**: I can provide information on various subjects, including history, science, technology, literature, and more.\n2. **Language translation**: I can translate text from one language to another.\n3. **Writing assistance**: I can help with writing tasks such as proofreading, editing, and suggesting alternative phrases or words.\n4. **Research**: I can help you find reliable sources of information on a particular topic.\n5. **Calculations and conversions**: I can perform calculations, convert units, and provide currency exchange rates.\n6. **Generating text**: I can create text based on a prompt or topic, such as a story, article, or email.\n7. **Conversation**: I can engage in natural-sounding conversations, using context and understanding to respond to questions and statements.\n8. **Brainstorming**: I can help generate ideas and suggestions for creative projects or problems.\n9. **Language learning**: I can assist with language learning by providing grammar explanations, vocabulary practice, and conversational exercises.\n10. **Miscellaneous tasks**: I can also help with other tasks such as generating passwords, creating to-do lists, or providing suggestions for movies, books, or games.\n\nWhat's on your mind? Is there something specific you'd like assistance with?"
  },
  "done": true,
  "done_reason": "stop",
  "total_duration": 3106166125,
  "load_duration": 54834041,
  "prompt_eval_count": 28,
  "prompt_eval_duration": 120922292,
  "eval_count": 277,
  "eval_duration": 2929947750
}
h
I've tried Ollama instead of llama-server and it works. I didn't realize they were different, thanks for pointing that out. And thanks for sharing the code.
s
Great! Happy to hear that it helped 👍
144 Views
Open in Slack
PreviousNext
Powered by