Join today's Live Webinar "*Object Detection and Image Recognition with Kotlin*". Register here. Agenda: In this webinar, we will explore a deep learning library written in Kotlin, learn how to detect objects of different types in images, and create a Kotlin Web Application using Ktor and KotlinDL that will detect objects in the images.

modelhub[...]

Is it a good idea to use get operation for something that could be slow or accesses the disk? Or the cache is loaded in the constructor?

Hello, is it possible to pass in an in-memory collection of byte arrays representing a batch of images inside preprocessing

Loading

instead of a File on disk? Or since

preprocessing()

can only be applied to 1 file usually then I guess we could say a ByeArray.

Hello! I was excited to try out KotlinDL on my M1 MacBook yesterday and I couldn't. I found out that it is not supported on the ARM architecture yet. Personally, I believe support for ARM is crucial as people that need small scale models run them on devices like the Raspberry PIs and build them on devices like the M1 Macs. I would love to know if there's a plan for ARM support and how it is coming up. Thanks in advance!

Hello! I can't run KotlinDL on M1 (tried versions 0.4.0-alpha-2 and 0.3.0). I faced with this error: https://github.com/JetBrains/KotlinDL/issues/164

Hello! K I’m trying to do some deep learning with KotlinDL and the Jupyter Kotlin Kernel. Sadly, I can’t get logging to work. Did anyone have a similar issue? Or do you know how to get logs during training in a (Kotlin) Jupyter Notebook? Thanks in advance!!

Hello everyone. Is it possible to do facial recognition with kotlindl???

Is KotlinDL planning to add a normalize

transformTensor

op that could be done after the rescale for instance ? I can contribute but want to know if you already did it for 0.4 first.

Does KotlinDL have any plans to add GLU and GLU variants. Since I used these activations quite a few times, I recently ended up creating a library for myself (https://github.com/Rishit-dagli/GLU) and would love to contribute this to KotlinDL. This was meant to be an issue but I wanted to know if this is already in the pipeline?

Hello, does KotlinDL have an NMS (Non-Max Supression) method or do we have to impl it ourselves?

KotlinDL 0.4 Is Here! NoTop models in the ModelHub, new layers and activation functions, better Image Preprocessing DSL and just see the GIF below! Read all in the blogpost: https://blog.jetbrains.com/kotlin/2022/06/kotlindl-0-4-is-out-with-pose-detection-api-[…]or-object-detection-and-efficientnet-for-image-recognition/

does KotlinDL have an LSTM layer?

Embedding layer ? I am currently working on sentiment classification and I was wondering if there is any layer that could be used instead of layers.Embedding() b/c it doesn't seems to be included on kotlind dl

I was doing some demonstration and found out the simultaneous prediction time of object detection model is slower(relatively speaking) so my first attempt was to capture image , saved to the storage, load it and make prediction using

detectObjects(file: File)

function it was slow as you would normally expect but what got me thinking is instead of loading saving and reloading the image I directly fed the

predictRaw(f: FloatArray)

with

FloatArray

that came from rescaled and recolored

BufferedImage

at this moment the rate of data fed was about 23

BufferedImage

per sec (from my web cam) followed by the predicator which was 111x slower and took 6.02 - 4.4 sec to load single data(

FloatArray

) I wasn't also sure about the Rescaler and color swaper I made but when I did the bench it barely have made any changes even the whole process of converting

BufferedImage

to

FloatArray

was only having a loss of 1 frames After all the only thing I could think of is that the predictor instead of getting called from the the mem it's simultaneously being fetched from the storage If that was the case is there anything that I could do to prevent that Here's the code in case you are wondering

while(cam.isOpen){
     model.predictRaw(cam.image.toFloatArray())
...

Was walking through the tutorials in the docs, but quickly hit the wall due to working on an Apple M1 notebook. FWICT aarch64 is supported in tensorflow since 2.3+. The Java API has seen a number of releases since the readme entry was written, are there still plans to upgrade?

Noob question. Can I use KotlinDL for text classification?? If yes, any example or doc

I would like to do face detection & recognition using KotlinDL. As far as I understand I will need ArcFace for Face Recognition and as a preprocessing detect the faces first. Therefore I'm looking for a MTCNN face detector for Kotlin. At best it's an implementation that does not require a Bitmap or other like Android-specific classes and just takes a JPG stream to detect the coordinates. I want to use it on the JVM within a compose-jb app. Does someone know something like that? https://github.com/tzolov/mtcnn-java is the closest thing I found so far, but it's abandoned and seems to have a lot of depenencies. Is there something easier to use for that?

Hello, where can we see the changes for the 0.5.0 alpha1 ? W'ere upgrading from KotlinDL 0.4.0. https://search.maven.org/artifact/org.jetbrains.kotlinx/kotlin-deeplearning-dataset/0.5.0-alpha-1/jar Cheers!

Hello, how can we include the

org.jetbrains.kotlinx:kotlin-deeplearning-dataset

dependency for

commonMain

(multiplatform library) ? Thanks

A KotlinDL 0.5-alpha-2 is published. The 0.5 release focuses on creating the API for the flexible and easy-to-use deployment of ONNX models on Android. Now you can create apps that use state-of-the-art neural networks just in a few lines of simple and elegant DSL. We have reworked Preprocessing DSL, introduced support for ONNX runtime execution providers, and more. A more detailed changelog is here. There is also an Android Demo Application that shows how to use KotlinDL ONNX for Android. We would appreciate you tried it and let us know if you encounter any problems or have feedback.

Any new thoughts on Reinforcement Learning? I’m curious for trying it out for investment strategy

Do you plan to implemented an ONNXInterferenceModel for https://github.com/onnx/models/tree/main/vision/body_analysis/arcface ? Should I create an issue for that?

Is there a way to load my ONNX model from local resources instead of the model hub? UltraFace320 ist just 1 MB and I don't want to rely on the hub if I can just bundle it with my app.

Huge and exciting news! KotlinDL is now available on Android and using this release you can run practically any ONNX neural network on your device. Moreover, the data preprocessing step has also been significantly reworked and feeding data to the network is much easier, more elegant, more versatile. Also with KotlinDL 0.5, it is possible to run models on optimized hardware which is crucial to get proper performance. There is also an Android Demo Application that shows how to use KotlinDL ONNX for Android. Learn more: https://blog.jetbrains.com/kotlin/2022/12/kotlindl-0-5-has-come-to-android/

Hi there, is there a an OCR (image to text) model (similar to tesseract) which I could use with kotlindl?

Hi, I have a few questions and I can't seem to find an answer on the Internet: Some information for the beginning - we're creating a model that should predict whether OCR should be made using a cloud service or inside the application. Here are the questions: 1. Is it possible to create a dataset with multidimensional output? We hope the model will be able to both tell where the image should be processed, as well as it should tell how much time it will spend on processing image. I tried to use OnHeapDataset but labels have to be one-dimensional Float Array, so there's no way of doing even a data class. 2. Is it possible to do more training once the model was taught? What I mean is that we're going to train the model on the side with some data and we're wondering if it will be possible to do some additional training during the lifecycle of the application (it's an university project, so we don't care about performance of the solution as much). Will another call of

fit

function suffice? Thank you for the answers!

Hi, I am relatively new to KotlinDL and have a question regarding the usage of a custom dataset. I want to do some Object-Recognition and have a labeled dataset in the YOLO format. Meaning: dataset/images/train/*.png dataset/images/test/*.png dataset/labels/train/*.txt dataset/labels/test/*.txt A given image always corresponds to a given txt file by the name. For instance

1.png

in the

/images/train/

directory corresponds to

1.txt

in the

labels/train

directory. A txt file is either empty (if no object is in the corresponding image) or contains rows like

0 0.403260 0.570903 0.090604 0.160194

. Each row represents an object labeled in the corresponding image where the first element is the class and the other 4 elements describe the bounding boxes top left and bottom right coordinates. I want to use the

OnHeapDataset

. I was a bit confused regarding the usage of FloatArray everywhere but I think I got the

featuresExtractor

working:

featuresExtractor = { basePath ->
            println("extracting features in $basePath")
            val folder = File(basePath)

            val res = folder.listFiles().map { image ->
                println("converting ${image.absolutePath}")
                val imageWithAlphaChannel = ImageIO.read(image)

                val imageWithoutAlphaChannel = BufferedImage(
                    imageWithAlphaChannel.width,
                    imageWithAlphaChannel.height,
                    TYPE_INT_RGB
                ).apply {
                    for (x in 0 until imageWithAlphaChannel.width) {
                        for (y in 0 until imageWithAlphaChannel.height) {
                            val rgb = imageWithAlphaChannel.getRGB(x, y)
                            setRGB(x, y, rgb)
                        }
                    }
                }


                pipline.apply(imageWithoutAlphaChannel).first
            }.toTypedArray()
            println("finished extracting features in $basePath")
            res
        }

However, I am unsure how to build the

labelExtractor

since it awaits

(String, Int) -> FloatArray

. So I am am unable to encode the described row structure (containing class, x1, y1, x2, y2) into an

FloatArray

as I would need a

Array<FloatArray>

to encode the information for the bounding box. I hope my questions make sens and that I am not completely on the wrong track 😄

Do I understand correctly that kotlinDl doesn't support m1s? https://github.com/Kotlin/kotlindl/issues/164

Hello, Can KotlinDL allow to analyze texts rather than images? In case this is not possible, do you have a library allowing to create a neural network in order to analyze texts entered by users?

I didn’t know this channel existed otherwise I would’ve asked here, so I posted an Issue instead: https://github.com/Kotlin/kotlindl/issues/519