The <docs on variant matching> call out `guava-jre...

# gradle

[Aaron Todd]

The docs on variant matching call out guava-jre and guava-android as an example where using attributes would be a better solution. If you wanted to do something like this (publish a regular jvm version and an android specific version of a library) what are the appropriate attributes to set? It vaguely mentions org.gradle.jvm.version but I'm not seeing any properties specific to android...

[Vampire]

https://docs.gradle.org/current/userguide/variant_attributes.html#sub:jvm_default_attributes

org.gradle.jvm.environment

Indicates that a variant is optimized for a certain JVM environment.

Common values are standard-jvm and android. Other values are allowed.

The attribute is used to prefer one variant over another if multiple are available, but in general all values are compatible. The default is standard-jvm.

[Aaron Todd]

Are you aware of any examples that connect all the dots?

It seems to me I just need to get the android variants into the main gradle module artifact and they can then still be published as a separate classifier (e.g. my-lib and my-lib-android)?

[Vampire]

Why should you publish as separate classifier if you put all in one JAR?

[Vampire]

Different variants can produce different JARs or they can produce the same JAR

[Vampire]

Different variants can declare the same dependencies or different dependencies

[Vampire]

And these two either-or can be combined like you need them, for example if the code is the same, but one variant needs other dependencies,
or if both variants need the same dependencies but have different code.

[Aaron Todd]

I suppose I'm just struggling to connect all the dots here. Conceptually that makes sense but how to express each in Gradle is lost on me at the moment.

So to backup for a sec. I have a native library being wrapped through JNI. I'm able to publish a JAR with the native libs for each platform for regular JVM. I want to take advantage of the .aar format for Android (which solves a few issues automagically like LD library path and bundle sizes).

The thought I had here was to publish a variant for Android which also hopefully works seamlessly with Kotlin Multiplatform resolution rules (or at least easy declaration of the consumer side). Currently the android build is modeled as a composite build which is probably part of my confusion.

[Vampire]

Sounds like a wonderful case for variants, yes, but then you don't even have the chance to "get the android variants into the main gradle module artifact", because you just said you need a different artifact (the .aar)

[Vampire]

Unfortunately I cannot give good advice regarding Android, as I'm not doing Android dev.
But the docs are at https://docs.gradle.org/current/userguide/userguide_single.html#sec::declare_feature_variants
Basically you set up a feature variant and configure the android variant how you need it.

[Aaron Todd]

I'm not an Android dev either (probably also part of the problem), appreciate the help though

[Aaron Todd]

@jendrik How does one do what is called out in the docs here?

Specifically I have a my-library and my-library-android and I want to declare my-library-android as a variant of my-library. In the kotlin multiplatform world they seem to do this with available-at , e.g.:

// random variant pulled from kotlinx-coroutines.module  
  {
      "name": "mingwX64ApiElements-published",
      "attributes": {
        "org.gradle.usage": "kotlin-api",
        "org.jetbrains.kotlin.native.target": "mingw_x64",
        "org.jetbrains.kotlin.platform.type": "native"
      },
      "available-at": {
        "url": "../../kotlinx-coroutines-core-mingwx64/1.5.1/kotlinx-coroutines-core-mingwx64-1.5.1.module",
        "group": "org.jetbrains.kotlinx",
        "module": "kotlinx-coroutines-core-mingwx64",
        "version": "1.5.1"
      }
    },

This would appear to be a RemoteVariant but I'm not seeing how to set this information from regular Gradle script though. This of course assumes separate publications/classifiers.

The other option that comes to mind is to publish all artifacts as a single publication but again I'm not seeing how to declare the android variant as part of the "main" java publication. The my-library-android "variant" is published as .aar whereas the my-library variant is a normal .jar. The reason for the split is due to wrapping native libraries and taking advantage of the .aar format.

What approach makes more sense and why? Easiest way to declare this?

Apologies for calling you out directly but your name keeps popping up in all the PRs and issues related to module metadata which would indicate you actually probably understand what to do here. I've gone too far down the rabbit hole to turn back now.

[jendrik]

@Aaron Todd The “available-at” is something “on top of everything else” (an implementation detail of the variant publishing if you will) that allows you to publish two things independently and then connect them later as variants. E.g. if you need to build different variants on different operating systems. It’s not really exposed on the APIs right now, because the concept is still a bit weird: You have components which are also variants of other components. That’s not completely consistent with everything else. I believe Kotlin MPP implements their own “Component” for doing this which is also only half-public API right now… (but there is also no better solution atm for them I believe).

For your case, it depends a bit on where you start. I am did not get your situation completely.

Do you want to add more variants on top of what Kotlin MPP does?
Or do you not even have a Kotlin MPP project, but just want to publish something other Kotlin MPP projects would consume correctly?

Can you share some of the structure of your build? Is everything done in one (sub)project?

[Aaron Todd]

@jendrik Thank you for the response!

Ahh ok so that lines up with what I'm seeing and makes sense.

that allows you to publish two things independently and then connect them later as variants.

Yes this is what assumed it was doing but I have yet to find other examples other than KMP which lead me to believe this likely isn't the right solution OR we would have to manually craft the .module metadata since the APIs aren't exposed.

Part of the struggle of understanding variants is who consumes/produces the metadata. The answer it seems is usually a plugin.

Or do you not even have a Kotlin MPP project, but just want to publish something other Kotlin MPP projects would consume correctly?

Correct this isn't a kotlin MPP project. It's a Java project with JNI bindings. Due to the native libs it's desirable to produce a .aar for Android rather than what we do for desktop platforms which is just bundle support for each architecture into the final jar. We have downstream projects that are KMP projects that would depend on this.

Can you share some of the structure of your build? Is everything done in one (sub)project?

Right now the entire Android build is an independent project pulled in as a composite build. Nothing necessitates this is the case it's just the state of the world right now. It could be made to be a proper subproject fairly easily. This is part of why I asked about the available-at since two artifacts are produced at different coordinates org:my-lib:{version} and org:my-lib-android:{version} . It would be perfectly fine to structure it differently and have a single publication with the android artifacts as a variant.

It seems to me the only way to create a variant is to create an outgoing (consumable) config (would love to know if this is an incorrect assumption and how else variants are created). I saw two options but I haven't quite pieced it together:

NOTE: both of these examples were created with the Android project as a true subproject NOT a composite build.

1. Try and re-use the configurations already in the Android subproject
Something along the lines of:

// root build.gradle.kts

val javaComponent = components.findByName("java") as AdhocComponentWithVariants

project(":android:subproject").afterEvaluate {
    configurations.forEach { println("$it : ${it.isCanBeConsumed}") }
    val outgoing = configurations.getByName("archives")
    javaComponent.addVariantsFromConfiguration(outgoing) {
        mapToMavenScope("runtime")
        mapToOptional()
    }
}

It's unclear to me the correct Android configurations or if this even would work. IIRC there are only two consumable Android configs in the subproject archives and default.

2. Manually create the outgoing configurations/variants:

val androidRuntimeConfig = configurations.create("androidRuntimeElements-published") {
    isCanBeConsumed = true
    isCanBeResolved = false
    attributes.attribute(TargetJvmEnvironment.TARGET_JVM_ENVIRONMENT_ATTRIBUTE, objects.named(TargetJvmEnvironment.ANDROID))
    attributes.attribute(Usage.USAGE_ATTRIBUTE, objects.named(Usage.JAVA_RUNTIME))
    ??
}

javaComponent.addVariantsFromConfiguration(androidRuntimeConfig) {
    mapToMavenScope("runtime")
    mapToOptional()
}

This option does in fact create a variant entry in the resulting .module metadata but it's not quite correct (and I suppose I'm not 100% sure what correct looks like).

We maybe have a workaround by just solving this in the downstream KMP projects but at this point I'd just like to understand this better and know how to do it regardless of which way we land.

[jendrik]

@Aaron Todd

It seems to me the only way to create a variant is to create an outgoing (consumable) config

Yes. That is correct. (there is something else called “secondary variant”, which is confusing, but that’s independent of publication)

The ideal setup would be if both variants are in the same (sub)project. That’s a limitation of Gradle’s publishing mechanism that is known as a shortcoming that should be improved at some point. Why do you have two separate projects? Do you really have different code? Or is it more because the Android and Java plugins can’t be combined as you would like? If you can’t get it into one project it might have to be a bit hacky…

General problem of your solution (1) is that you reach directly from one project (the root) into another (android:subproject). It might work, but many systems in Gradle are not designed for this. So it might just break in weird way. (This is something that will become impossible in some future version of Gradle when the new configuration cache feature becomes the default)
However, there should be more consumeable configurations. The Android plugin, for some reason, needs to create them late. So you sometimes (unfortunately still) have to check/do things in afterEvaluate{} . I think you also have to make sure to apply “maven-publish” to get the configs Android creates for publishing. See also: https://developer.android.com/studio/build/maven-publish-plugin (behind each of these components there is also a consumable configuration).

Your solution 2 looks much better. That’s the way I would go. Which problem are you still facing with that one? How do you get the code (the aar) into your new androidRuntimeElements-published configuration? Maybe that’s where you are missing something.

If you could share a small reproducer project with what you have so far with solution 2, I could take a look.

[Aaron Todd]

@jendrik

Why do you have two separate projects? Do you really have different code? Or is it more because the Android and Java plugins can’t be combined as you would like?

Just the state of the world at the moment, I believe it could be made into a (sub)project of the main one.

The Android plugin, for some reason, needs to create them late. So you sometimes (unfortunately still) have to check/do things in afterEvaluate{}

This seemed to be what I was running into and now makes more sense with your explanation.

Your solution 2 looks much better. That’s the way I would go. Which problem are you still facing with that one? How do you get the code (the aar) into your new androidRuntimeElements-published configuration? Maybe that’s where you are missing something.

I haven't fully fleshed this out. It sounds like all I would need to do is collect the Android artifacts and declare the appropriate configurations. The only "issue" I can think of here is what variants should be created and are there any android specific variants that need to exist. Probably need to go look and see what the android plugin is creating for module metadata...

Question though, if the Android project is a subproject and reaching into the subproject directly from the root is undesirable, what is the right way to collect the Android artifacts and tie them to the java component of the root project? Should/can this be done in the subproject instead?

The project is open-source. I can share it when I get something less hacky. Currently trying to fix Android support.

Thanks for all of the responses, VERY much appreciated. Will reach back out if I fix it or need further help (at which point I'll just get it into a state that I can share something concrete).

[Vampire]

He didn't say "why in separate builds", but "why in separate projects" and that the optimum solution would be to have both variants in the same project, not in a (sub-)project of the main build. 😉

[Aaron Todd]

Ahh I see, thanks. That may be a taller order since there is native code involved and being compiled as part of the build process. I am not confident they could be combined into the same project.

[jendrik]

N.p.

right way to collect the Android artifacts and tie them to the java component of the root project?

Yes that’s the part that might be a bit tricky. That’s the “limitation of Gradle’s publishing mechanism” I mentioned earlier. One question here is if you only need to ‘transport’ the artifact from one project to another (we can solve that easily I hope) or if you also need metadata for dependencies (this might be more involved). Does your library have dependencies?

If you move things as far as you can and then share it here, I can have a look and might be able suggest something more concrete.

[Aaron Todd]

It has no dependencies. For now I have decided to just solve this downstream since KMP makes it easy to declare a secondary JVM target with a different set of dependencies:

jvm { ... } // standard desktop jvm

jvm("android") {
    attributes {
        attribute(
            org.jetbrains.kotlin.gradle.plugin.KotlinPlatformType.Companion.attribute,
            org.jetbrains.kotlin.gradle.plugin.KotlinPlatformType.androidJvm
        )
        attribute(
            TargetJvmEnvironment.TARGET_JVM_ENVIRONMENT_ATTRIBUTE,
            objects.named(TargetJvmEnvironment.ANDROID)
        )
    }
}

...

val androidMain by getting {
    dependencies {
        implementation("my-library-android") // upstream Android dependency we have been talking about
    }
}

[Aaron Todd]

This will have to suffice for now. If I find more time down the road I may revisit and see if I can't get the variant information into the actual project producing the android artifacts. For our use case and our customers though this will be fine I think since they should have no need to declare a direct dependency on my-library. It's an implementation detail of something they would consume.