For those of view doing some computer vision on JV...

# datascience

[Ролан]

For those of view doing some computer vision on JVM - how would you go about using OpenCV - (I) get it from sources as in official documentation - (II) use bytedeco stuff - (III) something else (please specify)? Thanks

[Hampus Londögård]

If possible I highly recommend using BoofCV 🙂
It’s pure JVM and really fast. 👍

[Ролан]

That's a nice shot actually, thank you. But I am surveying from the native point of view because I am working on a wrapper of PyTorch for Kotlin JVM. I am scared in that case FFI might hurt performance quite a lot.

[Hampus Londögård]

Ah, I see. bytecode for me was easier to use because it included wrappers and everything for native code.
The OpenCV wasn’t as simple to add (as you’ve to add the jar manually).

But OpenCV in Java with either bytecode or opencv api is not nice to use and there’s some big differences sometimes compared to Python.

[Hampus Londögård]

I don’t think DJL wraps OpenCV but it’s worth actually taking a quick peek:
https://docs.djl.ai/index.html

It’s Amazon backed wrapper for PyTorch/Tensorflow/ONXX in Java. Works pretty good from Kotlin 🙂
Their sentencepiece JNI was supersimple to use and a separate dependency

[Ролан]

Benchmark                                  Mode  Cnt    Score    Error  Units
BenchmarkThresholding.block_mean_java      avgt    5   18.321 ±  0.244  ms/op
BenchmarkThresholding.block_mean_native    avgt    5   12.769 ±  0.479  ms/op
BenchmarkThresholding.block_minmax_java    avgt    5   19.762 ±  0.352  ms/op
BenchmarkThresholding.block_minmax_native  avgt    5   21.616 ±  0.122  ms/op
BenchmarkThresholding.block_otsu_java      avgt    5   48.116 ±  1.703  ms/op
BenchmarkThresholding.block_otsu_native    avgt    5   35.232 ±  0.266  ms/op
BenchmarkThresholding.global_fixed_java    avgt    5    9.888 ±  0.812  ms/op
BenchmarkThresholding.global_fixed_native  avgt    5    1.260 ±  0.151  ms/op
BenchmarkThresholding.global_otsu_java     avgt    5   21.899 ±  1.578  ms/op
BenchmarkThresholding.global_otsu_native   avgt    5   13.223 ±  1.488  ms/op
BenchmarkThresholding.local_mean_java      avgt    5  100.411 ± 21.708  ms/op
BenchmarkThresholding.local_mean_native    avgt    5   71.637 ±  2.955  ms/op

[Ролан]

that's from https://github.com/lessthanoptimal/BoofCPP

[Ролан]

@altavir java seems to be a bit behind ))

[Hampus Londögård]

I know that Peter is looking at the new Vector-API and that should make Java really close to CPP.

Also depending on use-case the Java-version could be faster because of over-head with JNI 🙂 (for mine it actually is, because I only convert one image once in a blue)

[Ролан]

@Hampus Londögård you are right, but there are a few things we want to port from Torch that will take ages to appear in DJL

[Hampus Londögård]

I see 🙂 With DJL I actually only ment taking a quick peek if the actually have a OpenCV wrapper too. As they did with SentencePiece 🙂

[altavir]

@Ролан You need to remember that performance is in most cases not about atomic operations, it is about program optimization as a whole. This is why micro-benchmark do not actually show anything. The 30% difference is negligible and actually with that difference real-life JVM program will be faster in most cases due to call optimization. I never worked in CV, but I believe that there are a lot of JVM-based solutions, both with FFI and without FFI.

[Ролан]

The choice here is whether the data should flow completely in native or we can afford the FFI overhead, because torch's tensors in any case will live beyond the JVM

[Ролан]

even for NLP I might prefer to do the tokenisation on the native side using huggingface's tokenisers rather than move up and into the JVM.

[altavir]

I think that your own tests show that FFI overhead is negligible. The problem arises when you actually want to access managed objects, because you need to copy them.

And still you need to remember what I said before. Actual performance of a complex program is not a sum of elementary action performances.

[Ролан]

My tests did NOT involve data transfers

[altavir]

I also think that new Panama FFI will decrease the overhead - both development and runtime even further

[Ролан]

@Hampus Londögård in terms of functionality was BoofCV a big buy for you compared to OpenCV?

[Hampus Londögård]

I don't do anything advanced, so for me it was a much easier experience over all. 🙂
I think OpenCV is a lot more feature-rich as they've got Neural Networks if you use any of that. Otherwise I think they're pretty close (?)

[Ролан]

fair enough thank you @Hampus Londögård