A modern programming language that makes developers happier.

kotlinlang

I'm experiencing context size issues caused by large tool call results.
To address this, I'd like to compress the output of specific tool calls (not the entire context) using a different model (with a larger context window) before sending the tool result back to the LLM. I've noticed `nodeLLMCompressHistory` node exists, but the existing implementations of `HistoryCompressionStrategy` don't fit my use case. What would be the recommended approach to implement such use-case?
Thanks

Hi <@U09L6GYNDU0>, in this case a good solution will be to write a <https://docs.koog.ai/history-compression/#custom-history-compression-strategy-implementation|custom implementation> of `HistoryCompressionStrategy`.

As an example, you can override the `compress` method to filter received messages, find the ones of a type `Message.Tool.Result` and call a specific model to compress the content.
After this, you can use `llmSession.rewritePrompt` and pass the rewritten content (with compressed tool call results) to the next node.

I  would need to temporarily set the prompt to only include the tool call result, call the specific model (I would also need `MultiLLMPromptExecutor`)  and put back the previous prompt but with the tool call result replaced with the summary. Is that it?

&gt; I would need to temporarily set the prompt to only include the tool call
Not exactly – you can address the messages by their type. In this case, your rewritten `compress` function will look somewhat like this:
```val messages = llmSession.prompt.messages

val rewritten = original.map { msg -&gt;
            if (msg is Message.Tool.Result) {
                val compressed = &lt;here you call your compression method&gt;                msg.copy(content = compressed)
            } else msg
        }
llmSession.rewritePrompt { prompt -&gt; prompt.withMessages { rewritten } }```
&gt; call the specific model
Yup, you'll have to set it up in a `retrievalModel` field when describing your compression node (basically the `nodeLLMCompressHistory` but with your custom strategy).
But you won't need to separately set it into the rewritten `compress` function, as `nodeLLMCompressHistory` already accepts a model name in a `retrievalModel` field.

&gt; put back the previous prompt but with the tool call replaced with the summary
Yup, an this should be done in a rewritten `compress` function – see the code example above :slightly_smiling_face:

Thanks. What would be compressions method? `compressPromptIntoTLDR`? It accepts an `llmSession`

As one of the possible solutions – why not! You can also implement your own method that accepts llmSession and pass it into `compress` .