Just fyi that I've started adding a Koog AI agent ...

# confetti

[John O'Reilly]

Just fyi that I've started adding a Koog AI agent to the Confetti project (in following branch https://github.com/joreilly/Confetti/compare/koog?expand=1). The project had been using Gemini directly before (if key was present) for providing recommendations but having an agent like this (along with associated tools) is a lot more flexibile).

This is still focussed on the active conference but be cool at some point to be able to query across different conferences.

[bod]

Very cool! Can this use a local LLM (LiteRT I guess?)

[John O'Reilly]

Haven't tried it yet but should be able to....and an Ollama model on desktop

[John O'Reilly]

Looking to update this to use Koog's embedding/RAG support.....basically (at least as I understand it right now) we can create embeddings for all the conference info which LLM can use.....still learning exactly what it is right now 😃

[John O'Reilly]

this may btw turn out to be something that's more suited to the backend.....could perhaps maintain the associated vector database there.....along maybe with agent itself

[John O'Reilly]

cc @mbonnin

[mbonnin]

Sorry I don't understand any of those words 😅

[John O'Reilly]

I vaguely understand them as of maybe 10 minutes ago 😃

[bod]

my overly simplified understanding of RAG is "semantic hash"

[John O'Reilly]

I've tried out the embedding "thing" locally here with info for one conference and seems to work quite well

[John O'Reilly]

this is what Claude is telling me it's doing

"A text embedding is a fixed-length vector of floats produced by a model. The geometry of that vector space encodes meaning — two texts about the same topic land close together
(high cosine similarity), even if they share no words. The Gemini gemini-embedding-001 model returns ~768-dim vectors. So "AI" and "machine learning" point in roughly the same
direction; "AI" and "database migrations" don't."

[John O'Reilly]

with that in place I can now do something like this

[bod]

that's so cool!

[John O'Reilly]

once we do this for all conferences we have we can then I think do some pretty interesting queries

[bod]

So we would need (on the backend) to run every conf description to a model to get an embedding and surface it through the API - then the client can make a vector of the query, and list the closest matches. Is that right? Or the query would need to run on the server?

[John O'Reilly]

that part I'm not certain about......might be better in that case to have the agent running on the backend

[bod]

Except for server costs I guess 😄

[John O'Reilly]

true!

[John O'Reilly]

my son is working on AI agent development in HPE as part of his internship there and has been telling me about RAG/embeddings etc for a while now and I think I'm starting to finally understand what it is 😃

[bod]

Very cool! I saw a talk about it in a meetup a while ago, I think I mostly understood back then - but of course forgot most of it since 😅

[John O'Reilly]

it seems like a good fit for something like this

[bod]

I guess if all the talk descriptions (for one particular conf) can fit in the LLM context, that could work too. But yes I think RAG is the "correct" way to do this.