Hi, I would like to recieve opinions on Kotlin for data science and scientific research. I'm have been using Python but know I want to invest some time in learning another language with tools good performance. So, I started to learn Scala and have been enjoying, but I saw some articles showing the tools developed for Kotlin in respect to libraries for deep learning and differential programming. #datascience #science

What scientific field do you have in mind?

Machine Learning and computational intelligence. Also, I work with applications in Finance and time series forecasting.

Well, I am not familliar with the field (we could ask @Ролан who comes from computational finance), but I think that Kotlin has very good perspectives here, at least as default language. The number of available libraries is still small compared to Python and Julia bindings, but the relyabllity is much better. Also the tooling for application and library development is supreme. The notebook experience is not the same as in python yet, but it is already quite close and I think it will be even better in future.

In terms of speed, I tried using Python with numba but it felt so sensible and not compatible with a lot of python code. Also, the lack of static typing and compilation really intrigues when the code breaks after several hours. Julia was my first bet, but it does not have tooling and static types which I think are essential for building reproducible and stable projects in scientific projects and applications. Thanks for helping me with this. I like Scala's tooling ... Intellij Idea help me a lot. But I'm thinking about the future. So, I thought maybe Kotlin will overtake Scala because of its simplicity and investment.

I have experience both with Python and Julia. Python is OK for small scripts and is overall good as long as you do not do actual new code development in it (so it is good for non-programmers). Julia is a nice language, but lack of proper tooling and LLVM-based JIT are its curse. As for Scala... the problems is the same as with C++, good scala developers are very rare and expensive. If you intend to grow, you will have problems finding them. Kotlin is not as flexible as Scala, but it allows to achive almost the same level muuuch faster.

Good points! I need performance because experiments in mahcine learning and finance take a lot of time. And time series is a pain because we can't vectorize a lot of things. Do you have some guidelines and material to start learning Kotlin for science besides Jetbrains' website?

@Filipe Duarte I recommend you to start with krangl http://holgerbrandl.github.io/krangl/ by @holgerbrandl

@Filipe Duarte welcome to #kotlindl channel and read KotlinDL tutorials for building Deep Learning models https://github.com/JetBrains/KotlinDL Tooling libraries are at the baby level, of course, and DS Community in Kotlin is young and small comparing with Python, but it will be better than Scala DS ecosystem in the nearest future, if not yet

Awesome, krangl is similar to pandas. KotlinDL uses DL4J as backend? Is there some lib similar to PyTorch? I saw that for machine learning I could use Spark.

It's TensorFlow under the hood in KotlinDL, to taste a PyTorch engine you could try java library DJL From Amazon, it Supports inference on PyTorch. Unfortunately, Facebook doing nothing for wide JVM support for PyTorch Also, as I know, @Ролан experimented with PyTorch in Kotlin, but it's not released yet

@altavir which Kotlin wrapper for Spark were you recommending me the other day?

Official one by @asm0dey: https://github.com/JetBrains/kotlin-spark-api

Of course, if you are into finance then

kdb

has an allrightish Java API as well.

Actuallt there is a #kotlin-spark as well as #kotlindl here. If we are talking about krangl-like things, there is an incubating https://github.com/nikitinas/dataframe, which does more or less the same thing but in more kotlinized way. There is also my own multiplatform implementation: https://github.com/mipt-npm/dataforge-core/tree/dev/dataforge-tables, but it is on hold for the moment because a lot of other work.

What? So timely: https://twitter.com/kotlin/status/1380542018439544838?s=20

Yes it it models missing data as null

@holgerbrandl Do you have a tutorial explaining how to do this?

There is http://holgerbrandl.github.io/krangl/ . I'm happy to extend it it needed

Kotlin-JVM has seamless integration with Java. For Scala it would depend on Scala library. If it is compatible with Java, the call from kotlin will lool the same way as from java

@holgerbrandl thanks you. @altavir ok!