I wanted to ask whether ollama can allow me to run...
# koog-agentic-framework
b
I wanted to ask whether ollama can allow me to run gemma 4 am testing it on the mobile side! has anyone tried running any model locally with koog
a
Hi Brian, sure, it should. You'd have to add a model profile but it should be quite easy based on the ones already added.
d
Well running Ollama on Android (can't say for iOS) can be quite challenging but doable if you really want it: https://peter-nhan.github.io/posts/Local-Android-Ollama/ I would recommend trying LiteRT-LM though (https://github.com/google-ai-edge/LiteRT-LM/) or the Google GenAI API. Both should support the small Gemma 4 models. Koog's team is working on a GenAI integration with Koog: https://github.com/JetBrains/koog/pull/1715 Also, you can try my very rough POC of an integration with LiteRT-LM (https://github.com/ptitjes/konvo/commit/cf024e2798e8fcf8b38cfac9c047034e46d9ed47) though I only tested it on JVM and you will need to package some jni libs for Android (cf the LiteRT-LM page).
m
You might also look into Llamatik or other llama.cpp-based libraries
😮 2
d
Oh, thanks so much Mark for making me discover this library.
👍🏻 1
s
@Anastasiia Zarechneva Its not as straight forward. @Didier Villevalois has a great starter code - I specified a minor correction there Ill just reiterate here ToolResponse in LiteRT LM is a message type - So while converting ContentPart.Text to equivalent LiteLM content needs special handling for ToolResponse Handling multi modal data is a pain - each sdk just expects things different Does koog have any plans of integrating LiteRT LM? Additional design suggestion - I think its better for koog to just have a fields in Tool.Call or Tool.Response to better call out arguments / results. - Maybe a separate content type? or just name them as args/results.
a
Does koog have any plans of integrating LiteRT LM?
We do. Not as the first priority, but we're aware of the demand and it's on our list of ToDo's 🙂
👍 1
b
Thanks alot @Anastasiia Zarechneva
🙌 1