https://kotlinlang.org logo
Title
p

Peter

11/05/2021, 1:52 PM
Contemplating buying an Apple Macbook M1 (so the one with the ARM CPU), but wondering how it works with Kotlin Data Science & Notebooks. I know there are M1 native JVM's and IDEA, but data science uses typically more native libraries than most JVM applications. Any experience that people want to share?
i

Ilya Muradyan

11/05/2021, 2:01 PM
@zaleslaw @Pavel Gorgulov should know about Multik and KotlinDL. What about notebooks in general (kernel), and Dataframe, they should work fine on M1
a

altavir

11/05/2021, 2:03 PM
If you do not go into native optimization like Multik or Kotlin DL, you are working only in JVM so everything should work fine.
p

Peter

11/05/2021, 2:21 PM
Thanks. As long as the basics works for on the road type of development that is fine. P.S did also found some reports of OpenBLAS running decent fast on M1, so that might be a nice cross-platform solution in the future (as opposed to MKL or vecLib).
a

altavir

11/05/2021, 2:53 PM
Multik uses CBLAS inside.
p

Pavel Gorgulov

11/05/2021, 4:25 PM
Multik has different computing engines. In multik-native, blas and lapack are really used and so far there are plans to support m1. multik-jvm will work on m1.
z

zaleslaw

11/12/2021, 12:08 PM
KotlinDL does not support M1 yet
c

Carter

11/20/2021, 7:20 PM
Is there an easy way to get some of these libraries working natively on M1 until a new release comes out? I’m hoping to avoid dropping to an x86 JVM with Rosetta. For example, with Smile I tried forcing the snapshot versions of the org.bytedeco artifacts (which added support for M1) like this:
implementation("org.bytedeco:javacv-platform:1.5.7-SNAPSHOT")
implementation("org.bytedeco:javacv:1.5.7-SNAPSHOT")
implementation("org.bytedeco:javacpp:1.5.7-SNAPSHOT")
implementation("org.bytedeco:hdf5-platform:1.5.7-SNAPSHOT")
implementation("org.bytedeco:hdf5:1.5.7-SNAPSHOT")
But when I run the application I get a runtime crash.
int org.bytedeco.openblas.global.openblas.LAPACKE_dgesdd(int, byte, int, int, java.nio.DoubleBuffer, int, java.nio.DoubleBuffer, java.nio.DoubleBuffer, int, java.nio.DoubleBuffer, int)’
java.lang.UnsatisfiedLinkError: 'int org.bytedeco.openblas.global.openblas.LAPACKE_dgesdd(int, byte, int, int, java.nio.DoubleBuffer, int, java.nio.DoubleBuffer, java.nio.DoubleBuffer, int, java.nio.DoubleBuffer, int)'
	at org.bytedeco.openblas.global.openblas.LAPACKE_dgesdd(Native Method)
	at smile.math.blas.openblas.OpenBLAS.gesdd(OpenBLAS.java:689)
	at smile.math.matrix.Matrix.svd(Matrix.java:1573)
	at smile.projection.PCA.fit(PCA.java:150)
I’m not sure if trying to force the dependencies will work, whether I’m missing something crucial (e.g. environment variable or another dependency), or whether there’s a bug in the snapshot build of the libraries that I should report. Any insights would really help 🙂.
p

Peter

11/21/2021, 10:44 AM
It is also still on my agenda to try to see if I can get some of the JavaCPP bindings to work since at least some basics are now in place, but didn't try it yet. As far as I know, none of the higher ML level libraries are yet ported (TF, PyTorch, DeepLearning4J, etc), only OpenBlas and OpenCV.
a

altavir

11/21/2021, 10:57 AM
@Ролан tryed to do that with torch, but it is harder than it seems
Dl4j does not need to be ported. But I believe it is obsolete. Tensorflow is wrapped by KotlinDL and I am currently doing the wrapper in KMath. Tensorflow-java uses javacpp internally
p

Peter

11/21/2021, 11:12 AM
I guess the TF challenge is in order to get really good performance on M1, it needs to use the TF for Apple Silicon version as developed/patched by Apple. As far as I know currently the Google TF version doesn't have good M1 GPU support and the Apple changes didn't make it yet back to their code base (but I could be wrong here).
a

altavir

11/21/2021, 11:37 AM
We do not need to care for TF implementation because it is done on native side. It will be done, when it will be done. TF provides very nice Java API, which we could use in Kotlin. If you are interested, you can contribute to https://github.com/mipt-npm/kmath/tree/feature/tensorflow.
c

Carter

11/21/2021, 11:45 AM
@Peter I believe Apple’s TF changes are available within TensorFlow now: https://developer.apple.com/metal/tensorflow-plugin/ But if I understand correctly, KotlinDL is using an older version of the TensorFlow Java API which then implies an older version of TensorFlow.
:yes: 1
р

Ролан

02/07/2022, 9:34 AM
I am using the latest version of LibTorch in my kotlin bindings but I only tested it on Ubuntu. I believe though that if you plug the right versions of GraalVM, Clang and LibTorch in the build script it shall work out of the box. I don't have a Mac env though to test it