Hi, I am pondering with something that is either a...

# kotest

[dirk.dittert]

Hi,
I am pondering with something that is either a very stupid mistake or something really weird. The exception is as follows:

java.util.stream.Stream.toList()Ljava/util/List;
java.lang.NoSuchMethodError: java.util.stream.Stream.toList()Ljava/util/List;
	at io.kotest.matchers.paths.PathsKt$containFile$1.test(paths.kt:162)
	at io.kotest.matchers.paths.PathsKt$containFile$1.test(paths.kt:160)
	at io.kotest.matchers.ShouldKt.invokeMatcher(should.kt:38)
	at io.kotest.matchers.ShouldKt.should(should.kt:33)
	at io.kotest.matchers.paths.PathsKt.shouldContainFile(paths.kt:158)
	at tfo.backup.walker.DirectoryWalkerMultipleFittingFilesTest$1$1$2.invokeSuspend(DirectoryWalkerMultipleFittingFilesTest.kt:52)

It is my understanding that this line fails:

val contents = Files.list(value).map { it.fileName.toString() }.toList()

However, Stream.toList() is properly implemented by Kotlin to use a Collector.toList() to work with older JDK (I am running a JDK 1.8)

And here comes the weird part: When I copy that line that fails (that I quoted above) to my unit test, it succeeds. This is very unexpected. Am I missing something obvious?

[Sam]

You said that Stream.toList() is implemented by Kotlin using a collector, which is true, but that extension function is still a different function from the member function that was added in JDK16. If the code you're calling was compiled against JDK16, it can still be referencing the member version that doesn't exist in your version of Java. It doesn't know the Kotlin extension version exists 😞

[Sam]

I don't know if Kotest is intended to work with earlier JDKs, but if so, this feels like an oversight. Depending on your build configuration, it's unfortunately very easy to reference newer JDK APIs by mistake when building on a newer JDK to target an older one.

[Sam]

When you paste the faulty code into your own file, where the JDK16+ version isn't even there at compile time, Kotlin will auto-import the extension function, which is why it works 👍

[dirk.dittert]

So, you're saying that kotest-assertions-core-jvm-5.9.1.jar is compiled on a newer version of the JDK that includes a Stream.toList() method (not sure when that was added exactly) but the compiler was configured to produce class files compatible with JDK 1.8. That is, why I run into that NoSuchMethodError.

[Sam]

Yes, that's my guess, since it would explain the symptoms you're seeing 👍

[dirk.dittert]

What I understood was that there is magic going on that would either create classes that use a Collector (if compiled on Java 8 ) or the newer Stream.toList() if compiled on a newer Java version? This is something that must be addressed when kotest-assertions-core-jvm-5.9.1.jar is compiled and is out of my hands?

[dirk.dittert]

When I look at that toList() call, this is what I see:

public fun <T> Stream<T>.toList(): List<T> = collect(Collectors.toList<T>())

which is in kotlin.streams.jdk8 and I guess the magic is that there essentially is a separate runtime jar for JDK 8.

[Sam]

Yes, unfortunately extension functions are always statically resolved (at compile time), there's no dynamic runtime magic. If my understanding is correct, I agree with you that this would need to be fixed by the library maintainers at compile time.

[Sam]

> I guess the magic is that there essentially is a separate runtime jar for JDK 8.
As far as I understand it, the JDK8-compatible kotlin.streams.toList() function is always present at both compile time and runtime, as part of the Kotlin standard library. The problem occurs when the JDK16 version of the function is also present at compile time. In that case, the class is compiled with a reference to the JDK16 function. When it runs on JDK8, it is still referencing that non-existing JDK16 function, even though the working version is also available. The two functions have different FQNs, and the class will continue to point to the one it was compiled against, regardless of what actually exists at runtime.

[dirk.dittert]

So there is two takeaways from this:

1. You need to compile on JDK 1.8 when you truly want JDK 1.8 compatibility. You won't notice your compilers' choices until it breaks on JDK 1.8
2. I need to file a bug report for this

[Sam]

For 1️⃣, that's a good solution, but maybe not the only one. There's a compiler option which should have the same effect.
For 2️⃣, I think you're correct, assuming I haven't misunderstood what's going on here.

[dirk.dittert]

Thank you for your help.

[Sam]

No problem! I hope you find a solution 👍

[dirk.dittert]

I updated the JDK for that project to 21 and that error goes away. I believe it is safe to assume that our analysis is correct.

[dirk.dittert]

I opened a bug report for this here: https://github.com/kotest/kotest/issues/4603