Hey everyone! I love Kotlin. I built my whole comp...

# ai

[Gabriel Duncan]

Hey everyone! I love Kotlin. I built my whole company on top of it, but as the LLM world had gotten into full swing, it is disheartening to see that my company is pressured more and more into relying on python. I personally feel like the ergonomics and readability of Kotlin are light years ahead of python, but the market is not looking good for kotlin in this quickly evolving space. I fear that Kotlin will lose relevance if Python eats it’s lunch

[Pavel Gorgulov]

What exactly is missing in Kotlin? In what cases does your company switch to Python instead of Kotlin?

[Gabriel Duncan]

For example pytorch code. Additionally most open source projects for ai tools are in python. OpenAI, and anthropic see java as an afterthought for their SDKs as does Google for their latest libraries (see genAI library in Go and Python only). Pretty much every new ai company has no jvm support.

I understand that there are techniques for wrapping apis or using openAPI generators if a spec exists. But overall my team can move faster using what the community is producing and most of it is terrible awful ugly Python code. And it is very painful to watch.

[Pavel Gorgulov]

Thank you for your sharing

pytorch is a large framework that provides a wide range of capabilities. Replicating it fully in Kotlin, or even on the JVM in general, seems impossible for now. That said, there is KotlinDL, and in the Java ecosystem, DJL is available.

For inference, there are a sufficient number of libraries in Java, and in Kotlin, there’s KInference. Notably, there’s JLama and java bindings for llama.cpp.

for openapi generation, unfortunately, openapi-generator doesn’t meet the necessary quality standards in my opinion, and fern doesn’t support Kotlin. However, fern does support Java, and, for example, cohere used it for their Java sdk. There is a request for Kotlin support in fern.

Among larger projects, I’d highlight spring-ai and langchain4j. Both are actively working on Kotlin support: https://spring.io/blog/2024/12/23/spring-ai-1-0-0-m5-released#kotlin-support and https://github.com/kpavlov/langchain4j-kotlin, but even now, you can use their java api to write in Kotlin

There are also new initiatives, such as the kotlin sdk for MCP and xef.ai

I agree that Kotlin is currently underrepresented in the field of AI development. However, the domain is vast, and it’s not entirely clear what should be done specifically in Kotlin

[Gabriel Duncan]

thanks for these resources @Pavel Gorgulov. they are very helpful. i was aware of some but not all of them. can you say more about why replicating pytorch in Kotlin would be impossible? From what i can tell, pytorch is handing off to C++ to do most of the heavy lifting and the actual python code is wrapper code. Isn’t the same thing possible with Kotlin? Python is also GCed like Kotlin and has it’s own runtime (i’m sure you know this) so i’m curious why it couldn’t be done in another language if someone wanted to give it a shot

[Gabriel Duncan]

@Pavel Gorgulov xef.ai looks cool but kind of dead. last approved PR is October 14…Do you know if it’s going to continue getting attention?

[Pavel Gorgulov]

It’s not entirely accurate to say that it’s impossible to create an equivalent of pytorch in Kotlin. It could be done with significant resources and a substantial amount of time, but even in that case, the result would likely lag behind pytroch in terms of performance or memory efficiency.
• dependency on native libraries in the JVM ecosystem leads to issues with linkage
• JNI, which can be used to create bindings in the jvm, comes with its own overhead, which can negatively impact performance.
• memory management - either memory is stored off-heap, requiring manual control, or it’s stored on-heap, which results in copying overhead
could use FFI, which has become more stable in 21jdk, or leverage graalvm. However, both options impose limitations on end users

I haven’t been in contact with the authors of xef.ai. It’s possible that they have indeed discontinued development

There are also other libraries aimed at specific providers, such as the AWS Bedrock Kotlin sdk, openai-kotlin, and anthropic-sdk-kotlin. While there aren’t many of these libraries, even compared to Java, there are still a few options available.

[Gabriel Duncan]

Thank you

[Kazik Pogoda]

I was struggling with similar thoughts, so I decided to fill this gap myself. I am the author of anthropic-sdk-kotlin (KMP) mentioned here before. I needed it to create claudine, an omnipotent AI agent which is actually better at coding Kotlin than me (an I am quite good ... 😉 ). It's maybe roughly 400 lines of code, half of it plain English. Now I am using both to generate a plethora of Kotlin AI libraries to provide the last missing pieces of the puzzle. Please check out xemantic-ai project for details. I am very close to achieve the state where AI itself will code all the missing libraries as high quality KMP projects, in particular openai-sdk-kotlin and cerebras-cloud-sdk-kotlin, maybe this week, maybe next week.

[Gabriel Duncan]

@Pavel Gorgulov just following up. DLJ fills an important gap. Thanks for sharing that as it really helps us use more models without needing to switch to Python

[Gabriel Duncan]

*DJL