Topical: what would it take to have KotlinDL drive...

# kotlindl

[darkmoon_uk]

Topical: what would it take to have KotlinDL drive the inference for Metas recently released LLaMA language model? ...and thereby integrate its abilities into Kotlin Apps. We can observe that the pytorch based source for llama.cpp and vanilla-llama is not so huge, can we expect to achieve similar via KotlinDL ONNX integration? Or can someone tell me I'm barking up the wrong tree?

[Hampus Londögård]

Hi, I integrate ONNX in londogard-nlp-toolkit
This allows to run all ONNX models directly from huggingface hub 🙂

[Hampus Londögård]

KotlinDL doesn’t support NLP last I checked

[darkmoon_uk]

Thanks; how does that work, are there a well-known set of NLP API's that ONNX (and other frameworks like PyTorch) supports and KotlinDL just doesn't expose? Any idea how big would the work to support be? Looks like the feature request is here

[Hampus Londögård]

Not sure, over all DJL is a cool library to work with to use models for all tasks.
But it's java-first

[Hampus Londögård]

KotlinDL wraps tensorflow to build/inference models and onnx to inference.

Currently only vision is supported natively.
That's all I know unfortunately

[Hampus Londögård]

But see my code in Londogard-nlp-toolkit to understand how big of a task it is with DJL -- pretty simple