Hi all :slightly_smiling_face: will there be a new...

# compiler

[James Hamilton]

Hi all 🙂 will there be a new kotlinx-metadata-jvm package published to mvn? The metadata generated by kotlinc 1.4 does not work with the metadata-jvm 0.1.0 version

[udalov]

Could you elaborate what problems are you getting?

[James Hamilton]

we're using the library to read the kotlin metadata like this:

KotlinClassMetadata md =  KotlinClassMetadata.read(new KotlinClassHeader(k, mv, bv, d1, d2, xs, pn, xi));

If the metadata is generated by 1.3.* it works, but with 1.4 this method returns null

I tried compiling the kotlinx-metadata-jvm library myself and using the jar as a dependency and it seems to fix the problem. I noticed there was some commits recently to the metadata library

[udalov]

This is very weird. kotlinx-metadata-jvm 0.1.0 has metadata version 1.3 [UPD: 1.1], which should be compatible both ways with 1.4, as per JvmMetadataVersion.isCompatible.

As a workaround, can you try just passing intArrayOf(1, 3, 0) [UPD: intArrayOf(1, 1, 0)] instead of mv in your code? It should mostly work fine for class metadata. (But if you’re also using KotlinModuleMetadata, this trick won’t work.)

In any case, I’ll look into it and we’ll probably release the new version built on 1.4 pretty soon.

[James Hamilton]

thanks! good workaround, that should work!

[James Hamilton]

the mv for the 1.3 compiled example that I have is mv = {
0x1,
0x1,
0x10
}

[James Hamilton]

using 1,3,0 doesn't work

[udalov]

Oh, right, sorry, I’m confused myself. Something like intArrayOf(1, 1, 0) should work instead

[James Hamilton]

indeed, that version works, thanks! 🙂

[James Hamilton]

in case it's useful. a simple Java app with the metadata that I took from 1.3.72 compiled class and 1.4.0 compiled class

kmd.zip

[udalov]

Sorry for the delay. I looked at the project you attached. The reason that you’re getting this error is actually that you’re initializing xi to -1. To fix the issue, use 0, or even better, load the actual value of Metadata.xi from the annotation.

If you’re interested in more detail how it works: Kotlin metadata has a version, and each reader (be it the Kotlin compiler, or the kotlinx-metadata-jvm library) declares what version it supports, which is basically the latest version available when that reader was built. Kotlin metadata is by default fully backwards-compatible (1.4 can read 1.0), and forwards-compatible up to one version (1.4 can read 1.5, but not later). This forward compatibility allows graceful migration to new Kotlin release for complicated projects, but it slows down changes in the metadata format, since we need to support all upcoming changes one release in advance.

However, in case we decide that breaking something in the format is more important than having the forward compatibility, we have a way to turn off this behavior for a specific version. This is possible via a bit in the metadata flags which is called “strict version semantics”. We haven’t used it yet in the compiler, but might use it at some point in the future. For example, metadata of version 1.4 would normally be readable for readers of versions 1.3 and above. However, if strict version semantics is set, it could only be readable for readers of versions 1.4 and above.

By setting xi to -1, you’re basically enabling all of the flags in the bitfield, including the strict version semantics:
https://github.com/JetBrains/kotlin/blob/master/libraries/stdlib/jvm/runtime/kotlin/Metadata.kt#L69..L70. Which makes kotlinx-metadata-jvm return null on that metadata, because its version ends up being detected as incompatible.

There are several more confusing details, since we didn’t quite figure out how to support this forward compatibility feature properly from the start. I’ll omit those, but the gist is that kotlinx-metadata-jvm 0.1.0 has metadata version 1.1 but can read Kotlin versions up to, and including, 1.4. So it should be compatible with all Kotlin binaries in existence as of today.

[James Hamilton]

Thanks! We had set the default value of xi to -1, and as there was no xi value in the annotation, it was never overwritten with an actual value when getting the values from the annotation. Changing it to 0 indeed solves the problem! Thanks for the extra details also, very useful! 🙂