I'll probably refactor to use the Java API in the near future

5 minutes into work and switching to a JavaRDD has already fixed a bug 😂

It's also good to note that you can arbitrarily jump from the scala API to the java one with no performance hit so if you're writing new code (and I assume you have to be to do it in Kotlin if most of the project is in Scala) you can easily have your code accept the scala rdd, convert it to java, work with that using kotlin, and then convert back to scala for the legacy code.