Is there a way to preserve binary compatibility wh...

# library-development

[Jeff Lockhart]

Is there a way to preserve binary compatibility when changing an API from a default argument to a regular function overload?

For example, changing:

fun foo(context: CoroutineContext? = null, factory: (String) -> T?)

to:

fun foo(context: CoroutineContext?, factory: (String) -> T?)
fun foo(factory: (String) -> T?)

Can I somehow keep the default argument function for binary compatibility with released library versions?

[Jeff Lockhart]

My motivation for the change is that the default argument function doesn't allow passing a function reference without including the default argument parameter. For example:

foo(::bar)

doesn't work for the default argument function signature, while it does with the regular function overloads.

[Jeff Lockhart]

Is it safe to do this?

@Suppress("CONFLICTING_OVERLOADS", "CONFLICTING_JVM_DECLARATIONS")
fun foo(context: CoroutineContext?, factory: (String) -> T?)

@Suppress("CONFLICTING_OVERLOADS", "CONFLICTING_JVM_DECLARATIONS")
@Deprecated("Present for binary compatibility", level = DeprecationLevel.HIDDEN)
fun foo(context: CoroutineContext? = null, factory: (String) -> T?)

[mbonnin]

I was going to suggest DeprecationLevel.HIDDEN but looks like you've thought this through already

[mbonnin]

Maybe you can use @JvmName ?

@Suppress("CONFLICTING_OVERLOADS", "CONFLICTING_JVM_DECLARATIONS")
@JvmName("newFoo")
fun foo(context: CoroutineContext?, factory: (String) -> T?)

[mbonnin]

That should take care of the "CONFLICTING_JVM_DECLARATIONS , not sure about "CONFLICTING_OVERLOADS"

[Jeff Lockhart]

Yeah, even with that deprecation level, it still shows the CONFLICTING_OVERLOADS error in the IDE. At compile time it gets the CONFLICTING_JVM_DECLARATIONS error, which I can also suppress.

[Jeff Lockhart]

Both function implementations are identical (and a single line), so it doesn't really matter to me which gets called under the hood.

[mbonnin]

Worst case, you can just add fun foo(factory: (String) -> T?) , that should work? It duplicates a bit the other one but it's not that bad

[mbonnin]

Or else, use @JvmName on the "old" one?

@JvmName("foo")
@Deprecated("old stuff", level = DeprecationLevel.HIDDEN)
fun foo2(context: Int? = null, factory: (String) -> Double?) {}

fun foo(context: Int, factory: (String) -> Double?) {}

fun foo(factory: (String) -> Double?){}

[mbonnin]

Well never mind, that might fail too 🤔

[mbonnin]

Mmm no it seems to work

[Jeff Lockhart]

Note, this is a KMP library with JVM and native targets. I'm actually not even sure how binary compatibility works with Kotlin/Native as it is. I just use the binary compatibility validator to check what I can for JVM.

[mbonnin]

Yea, for non-JVM I have no idea what will happen

[mbonnin]

I'm guessing linking will fail?

[mbonnin]

Would be a cool experiment

[Jeff Lockhart]

I'm assuming for Kotlin/Native it comes down to the klib format's binary compatibility. I see you've been active on this issue already yourself!

[mbonnin]

My experience is that JS/Native users are very fast to upgrade so breaking stuff there so breaking is less of an issue. But that'll probably stop at some point as the ecosystem matures

[mbonnin]

I think it's a bit different than the klib binary format? klib is about compiling with different versions of Kotlin.
Here it's more about what happens if a compiled lib doesn't find a symbol it was expecting

[mbonnin]

Problems could happen all with the same version of Kotlin eveywhere

[mbonnin]

On JS, there is @JsName I guess what we're looking for here is a way to control the name mangling in a KMP way.

[Jeff Lockhart]

That makes sense. Ultimately it comes down to the function being present and accessible with the expected function signature. I guess there's also @ObjCName, but then I also target Linux and Mingw as well.

[Jeff Lockhart]

And while my library's primary use case is calling the API from Kotlin common code, it probably wouldn't make a lot of sense to mangle the native platforms' signatures for the new visible API.

[mbonnin]

I'm guessing you've read https://jakewharton.com/public-api-challenges-in-kotlin/ already?

[Jeff Lockhart]

I have in the past. I should re-read to refresh my memory though.

[mbonnin]

This looks a bit like the .copy() case from that article except you can't use the HIDDEN trick because you're not adding an argument

[Jeff Lockhart]

Worst case, you can just add fun foo(factory: (String) -> T?) , that should work? It duplicates a bit the other one but it's not that bad

This might be the best, most straightforward option, keeping the default argument version and only adding the single argument one.

fun foo(context: CoroutineContext? = null, factory: (String) -> T?)
fun foo(factory: (String) -> T?)

[Jeff Lockhart]

You do see the redundant default argument in the signature, might be the only drawback.

[ephemient]

fun foo(context: CoroutineContext? = null, factory: (String) -> T?)

this previously created 2 functions:

fun foo(context: CoroutineContext?, factory: (String) -> T?)

@JvmSynthetic
fun foo(CoroutineContext?, (String) -> T?, Int)

where the latter is a wrapper invoking the first with arguments depending on flags

[ephemient]

you can create such a function yourself for binary compatibility (with @Deprecated(HIDDEN) since you don't want it to be called in source)

[Jeff Lockhart]

In my binary compatibility output, I see a function similar to:

public static synthetic fun foo$default (Lkotlin/coroutines/CoroutineContext;Lkotlin/jvm/functions/Function1;ILjava/lang/Object;)V

which looks similar to what you have above, just with an Object instead of as well as Int parameter. I'm assuming this is the flags. Is it documented somewhere how this function works, so I could implement it myself? I found this description.

[mbonnin]

Best place I've seen for this is Moshi codebase. Overal the Int is a bitmask that says which arguments are present and which should use the default value

[Jeff Lockhart]

Thanks! I'll check it out.

[Jeff Lockhart]

What about for Kotlin/Native? What does the default argument function generate internally on native platforms?

[mbonnin]

What do you mean? Name of the symbol? Or the implementation? I'd expect the implementation to be the same everywhere. As for the platform symbol name, I don't know.

[mbonnin]

You probably want do add the $default to maintain JVM symbol name

@JvmSynthetic
@JvmName("foo\$default")
fun foo(a: Int, b: Int) {}

[Jeff Lockhart]

Yeah, I assume the implementation is the same on JVM and native. I mostly wonder about the platform symbol for binary compatibility.

[Jeff Lockhart]

This seems to work, at least for JVM:

@Deprecated("For binary compatibility", level = DeprecationLevel.HIDDEN)
@JvmSynthetic
@JvmName("foo\$default")
fun <T : Any> foo(
    context: CoroutineContext?,
    factory: (String) -> T?,
    flags: Int,
    obj: Any?
) = foo(if (flags and 1 != 0) null else context, factory)

The flags implementation is based on this description.

The only difference in the output from binary compatibility validator is that the function is now final, which shouldn't matter.

So the only question is, does this work for Kotlin/Native targets as well?

[Jeff Lockhart]

For native, I guess Kotlin actually doesn't do anything special to handle default arguments or generate function overloads yet. So as long as the full function with both arguments exists, native binary compatibility should be the same, right?

[ephemient]

that's just as far as what gets exposed to non-Kotlin callers

[ephemient]

for Kotlin callers, there must be some wrapper, because the default arg gets evaluated in the context of the callee… but I don't know how it actually works

[ephemient]

I suppose there's a possibility that

@Suppress("CONFLICTING_OVERLOADS")
@JvmName("foo2")
fun foo(context: CoroutineContext?, factory: (String) -> T?)

@Suppress("CONFLICTING_OVERLOADS")
@Deprecated("Present for binary compatibility", level = DeprecationLevel.HIDDEN)
fun foo(context: CoroutineContext? = null, factory: (String) -> T?)

might maintain binary compatibility (possibly with @CName etc. too?). not if it's `interface`/`open` though..

[Jeff Lockhart]

The function is actually a top-level extension function, so not interface or open.