Topical: what would it take to have KotlinDL drive the inference for Metas recently released LLaMA language model? ...and thereby integrate its abilities into Kotlin Apps. We can observe that the pytorch based source for llama.cpp and vanilla-llama is not so huge, can we expect to achieve similar via KotlinDL ONNX integration? Or can someone tell me I'm barking up the wrong tree?

Hi, I integrate ONNX in londogard-nlp-toolkit This allows to run all ONNX models directly from huggingface hub 🙂

Thanks; how does that work, are there a well-known set of NLP API's that ONNX (and other frameworks like PyTorch) supports and KotlinDL just doesn't expose? Any idea how big would the work to support be? Looks like the feature request is here

Not sure, over all DJL is a cool library to work with to use models for all tasks. But it's java-first