A modern programming language that makes developers happier.

kotlinlang

hi there, is there any standard pattern for making tool calls using the end user's jwt? we have some downstream APIs which can only be called by real users and some hard requirements around data entitlements which mean that the APIs will return different data for different users

we currently have a work-around for this by instantiating a new  `ToolSet` implementation for each conversation, and passing in the user's jwt as a parameter, but it feels a bit clunky

I would say your current approach is fine. Using `ToolSet` and `Tool` classes for DI is one of intended use-cases, so I can’t think of any better option currently than what you’re doing already. Can you elaborate why does it feel clunky? Do you have any ideas on how to make it more convenient?

thanks for responding, we're also using spring so the original approach was to have a single instance of `ToolSet` injected by spring across the lifetime of the app, but obviously that doesn't work since the tool call has no concept of any context

I think you can try achieve both: having single `ToolSet` instance and managing request scoped `jwt` tokens.
What you need is to get data (`jwt`) into execution phase of your `ToolSet` without passing it from LLM of course.

In Spring a lot things is solved with `ThreadLocal` - check out <https://docs.spring.io/spring-security/site/docs/4.2.0.M1/apidocs/org/springframework/security/core/context/SecurityContextHolder.html|SecurityContextHolder>. It can give you `SecurityContext` that provides `getCredentials` method - that's where you can take your current user's `jwt` from (see <https://stackoverflow.com/a/71171884|this SO answer>).

You could do something like this:
```// implement this using SecurityContextHolder 
interface AuthProvider {
   fun getAuthentication(): Authentication
} 

class AuthAwareTool(private val authProvider: AuthProvider): ToolSet {
   // use auth.getAuthentication
}```

This is just an idea though - I haven't tested it - so please double check if it works for you :slightly_smiling_face:

thanks for the suggestion but i don't think this will work since the thread serving the spring request is different from the one executing the tool call

it's similar problem to starting an async job based on a user's request - a lot of the assumptions that you can do things like "get me the current logged-in user" don't apply any more

so you have to capture those things up-front before starting your async job/thread

&gt;  the thread serving the spring request is different from the one executing the tool call
In this case my suggestion won't work, you're right