Is anyone here using Apache Spark's Dataset API fr...

# datascience

[kz]

Is anyone here using Apache Spark's Dataset API from Kotlin?
I'm running into type inference issues that I didn't think would come up.
For example, the reduce operator has an overload that accepts a Java functional interface ReduceFunction but I can't get Kotlin to apply the lambda conversion automatically.
I have to manually use a SAM constructor to make it work
Ex: dsOfInt.reduce(ReduceFunction { a, b -> a + b }) instead of the nicer dsOfInt.reduce { a, b -> a + b }
I don't know if this is something to do with the fact that the target method is written in Scala so it's not a candidate for the implicit conversion.
I do know that the conversion right now only works for Java interfaces and from experimentation it seems to only work on Java targets methods that accept FunctionalInterfaces as parameters.

I also considered specializing any operators I needed as extension functions but then I can't use the idiomatic names since they all already exist as virtual functions on the class.
I'm trying to avoid the approach where I wrap the Dataset in a custom class that redefines any operators I need but so far that seems like the only option if I want idiomatic Kotlin usage.