A modern programming language that makes developers happier.

kotlinlang

I was digging around koog and it doesn’t look like <https://aws.amazon.com/blogs/machine-learning/track-allocate-and-manage-your-generative-ai-cost-and-usage-with-amazon-bedrock/|bedrock application inference profiles> are supported, at least explicitly. Has anyone found a workaround?

Hi, according to AWS docs, the inference profile ARN is passed as the `modelId` parameter in the `InvokeModel` request. Koog’s `BedrockLLMClient` passes `LLModel.id` directly as `modelId` when you call `execute`. So it should be possible to use a custom profile by copying/defining a new instance of `LLModel` with `id` set to your ARN, and then passing this model when calling `execute` on the LLM client/prompt executor or using it in your agents.

Let me know if this helps

I did a bit of searching and it looks like there’s an existing <https://youtrack.jetbrains.com/issue/KG-484/AWS-Bedrock-support-for-application-inference-profile|issue> for this on the tracking board. The issue is that the model id is used to determine the model family in the bedrock llm client.

Since asking this I created a workaround. I’m using a HttpInterceptor to swap the model id with an application inference profile ARN at the http client layer. It’s a messy solution though and I wish it was more elegant.