Hi. I’ve noticed that, despite my best efforts, so...

# koog-agentic-framework

[Hywel Bennett]

Hi. I’ve noticed that, despite my best efforts, some small language models, such as Gemma3:4b, insist on outputting JSON objects with

```json

code fences. This is also a problem when I try to call requestLLMStructured() as the deserialization fails and requires a fixing model to correct it. I wonder if it might be worth anticipating this pattern as a pre-deserialization step (either just as a sensible default or a hook for callers to pre-proccess the LLM output to remove the code fences). I understand that the fixing model exists, but this seems like a reasonably common problem case that could be pre-empted and save on additional model calls.

Would love to hear if anyone has any other strategies for handling code fenced outputs

[Anastasiia Zarechneva]

Hi @Hywel Bennett! Could you please specify the koog version you're using?

Last week, the corresponding fix was merged into the dev branch: https://github.com/JetBrains/koog/pull/965/files

You can either check it in the latest nightly release or wait for the upcoming minor 0.5.2 version – it will be there.

[Hywel Bennett]

Wow, thanks @Anastasiia Zarechneva! I’m on 0.5.0. I look forward to the release!

[Vadim Briliantov]

cc: @Antonii Belyshev @Andrey Bragin :)

[El Anthony]

I was having trouble with the QWEN model. I needed to modify the LLM response before Koog processes it to remove text.
I created a custom PromptExecutor that filters out unwanted text.
It might be helpfull while the new version is being released.

fun mySimpleQwenLLMPromptExecutor(baseUrl: String = "<http://localhost:11434>") : SingleLLMPromptExecutor =
    MySimpleQwenLLMPromptExecutor( baseUrl)

class MySimpleQwenLLMPromptExecutor(baseUrl: String = "<http://localhost:11434>") : SingleLLMPromptExecutor( OllamaClient(baseUrl))
{
    override suspend fun execute(prompt: Prompt, model: LLModel, tools: List<ToolDescriptor>): List<Message.Response> {
        val response = super.execute( prompt, model, tools)
        val res = response.mapNotNull { msg ->
            if( msg is Message.Assistant) {
                val newContent = msg.content.replace(emptyThinkContent, "")
                if( newContent.isEmpty()) null
                else msg.copy( content = newContent)
            }
            else msg
        }
        return res
    }

    companion object {
        val emptyThinkContent = Regex("[ \n]*<think>[ \t\n]*</think>[ \n]*", setOf(RegexOption.MULTILINE, RegexOption.DOT_MATCHES_ALL))
    }
}

[Hywel Bennett]

Thanks @El Anthony. That looks like a good interim solution 🙂