For those of view doing some computer vision on JVM how woul

Question

For those of view doing some computer vision on JVM how would you go about using OpenCV I get it from sources as in official documentation II use bytedeco stuff III something else please specify Thanks

Hampus Londögård · Answer

If possible I highly recommend using BoofCV slightly smiling face It s pure JVM and really fast +1

Ролан · Answer

That s a nice shot actually thank you But I am surveying from the native point of view because I am working on a wrapper of PyTorch for Kotlin JVM I am scared in that case FFI might hurt performance quite a lot

Hampus Londögård · Answer

Ah I see `bytecode` for me was easier to use because it included wrappers and everything for native code The OpenCV wasn t as simple to add as you ve to add the jar manually But OpenCV in Java with either `bytecode` or `opencv` api is not nice to use and there s some big differences sometimes compared to Python

Hampus Londögård · Answer

I don t think DJL wraps OpenCV but it s worth actually taking a quick peek <https docs djl ai index html> It s Amazon backed wrapper for PyTorch Tensorflow ONXX in Java Works pretty good from Kotlin slightly smiling face Their sentencepiece JNI was supersimple to use and a separate dependency

Ролан · Answer

```Benchmark Mode Cnt Score Error Units BenchmarkThresholding block mean java avgt 5 18 321 0 244 ms op BenchmarkThresholding block mean native avgt 5 12 769 0 479 ms op BenchmarkThresholding block minmax java avgt 5 19 762 0 352 ms op BenchmarkThresholding block minmax native avgt 5 21 616 0 122 ms op BenchmarkThresholding block otsu java avgt 5 48 116 1 703 ms op BenchmarkThresholding block otsu native avgt 5 35 232 0 266 ms op BenchmarkThresholding global fixed java avgt 5 9 888 0 812 ms op BenchmarkThresholding global fixed native avgt 5 1 260 0 151 ms op BenchmarkThresholding global otsu java avgt 5 21 899 1 578 ms op BenchmarkThresholding global otsu native avgt 5 13 223 1 488 ms op BenchmarkThresholding local mean java avgt 5 100 411 21 708 ms op BenchmarkThresholding local mean native avgt 5 71 637 2 955 ms op```

Ролан · Answer

that s from <https github com lessthanoptimal BoofCPP>

Ролан · Answer

< altavir> java seems to be a bit behind

Hampus Londögård · Answer

I know that Peter is looking at the new Vector API and that should make Java really close to CPP Also depending on use case the Java version could be faster because of over head with JNI slightly smiling face for mine it actually is because I only convert one image once in a blue

Ролан · Answer

< Hampus Londögård> you are right but there are a few things we want to port from Torch that will take ages to appear in DJL

Hampus Londögård · Answer

I see slightly smiling face With DJL I actually only ment taking a quick peek if the actually have a OpenCV wrapper too As they did with SentencePiece slightly smiling face

altavir · Answer

< Ролан> You need to remember that performance is in most cases not about atomic operations it is about program optimization as a whole This is why micro benchmark do not actually show anything The 30 difference is negligible and actually with that difference real life JVM program will be faster in most cases due to call optimization I never worked in CV but I believe that there are a lot of JVM based solutions both with FFI and without FFI

Ролан · Answer

The choice here is whether the data should flow completely in native or we can afford the FFI overhead because torch s tensors in any case will live beyond the JVM

Ролан · Answer

even for NLP I might prefer to do the tokenisation on the native side using huggingface s tokenisers rather than move up and into the JVM

altavir · Answer

I think that your own tests show that FFI overhead is negligible The problem arises when you actually want to access managed objects because you need to copy them And still you need to remember what I said before Actual performance of a complex program is not a sum of elementary action performances

Ролан · Answer

My tests did NOT involve data transfers

altavir · Answer

I also think that new Panama FFI will decrease the overhead both development and runtime even further

Ролан · Answer

< Hampus Londögård> in terms of functionality was BoofCV a big buy for you compared to OpenCV

Hampus Londögård · Answer

I don t do anything advanced so for me it was a much easier experience over all slightly smiling face I think OpenCV is a lot more feature rich as they ve got Neural Networks if you use any of that Otherwise I think they re pretty close

Ролан · Answer

fair enough thank you < Hampus Londögård>