https://kotlinlang.org logo
#kotlindl
Title
# kotlindl
d

darkmoon_uk

03/22/2023, 11:06 AM
Topical: what would it take to have KotlinDL drive the inference for Metas recently released LLaMA language model? ...and thereby integrate its abilities into Kotlin Apps. We can observe that the pytorch based source for llama.cpp and vanilla-llama is not so huge, can we expect to achieve similar via KotlinDL ONNX integration? Or can someone tell me I'm barking up the wrong tree?
h

Hampus Londögård

03/22/2023, 11:21 AM
Hi, I integrate ONNX in londogard-nlp-toolkit This allows to run all ONNX models directly from huggingface hub 🙂
KotlinDL doesn’t support NLP last I checked
d

darkmoon_uk

03/22/2023, 12:13 PM
Thanks; how does that work, are there a well-known set of NLP API's that ONNX (and other frameworks like PyTorch) supports and KotlinDL just doesn't expose? Any idea how big would the work to support be? Looks like the feature request is here
h

Hampus Londögård

03/22/2023, 12:23 PM
Not sure, over all DJL is a cool library to work with to use models for all tasks. But it's java-first
KotlinDL wraps tensorflow to build/inference models and onnx to inference. Currently only vision is supported natively. That's all I know unfortunately
But see my code in Londogard-nlp-toolkit to understand how big of a task it is with DJL -- pretty simple
13 Views