louiscad

    louiscad

    3 years ago
    I'm seeing several multiplatform libraries publishing to bintray using version
    1.8.4-jetbrains-5
    of the bintray gradle plugin instead of the jfrog official 1.8.4 version. Why so? Didn't found any sources for it, nor any documentation regarding it…
    louiscad

    louiscad

    3 years ago
    Next question: could JetBrains open source their modified gradle-bintray-plugin version? cc @svyatoslav.scherbina
    louiscad

    louiscad

    3 years ago
    Thank you, I now remember seeing this. I wonder why it's not documented anywhere while it's being used by several libraries, including kotlinx.coroutines. Also, the bintray page I found doesn't link to it (and no way to submit an issue for that on vvlevchenko's repo, it's disabled).
    gildor

    gildor

    3 years ago
    There is very important comment in issue 229 from Vasily. It looks that current solution is mostly hack and real fix would be reimplement gradle-bintray-plugin using new maven-publisg plugin than try to implement all new metadata publishing on top of existing implementation (my 2 cents that bintray plugin one of worst plugins that I saw when you work with Kotlin dsl, everything is dynamic)
    louiscad

    louiscad

    3 years ago
    You probably mean in PR 230, here: https://github.com/bintray/gradle-bintray-plugin/pull/230#issuecomment-456735555 I added a comment, looking for an example.
    gildor

    gildor

    3 years ago
    Yes, sorry PR 230 I did it once to avoid using bintray plugin. It's possible, but it requires generate all urls for bintray. Not sure that I still have this sample, will try to find
    louiscad

    louiscad

    3 years ago
    I'd be very interested. I'm currently working on setting up the Kotlin/Native sourceSets and targets nicely, with good IDE support, and nice usage from Gradle Kotlin DSL. I'd like to provide good base to make multiplatform libraries with ease.
    m

    msink

    3 years ago
    It's working in my libui library - not iOS, but still - it uses Kotlin DSL and Gradle bintray plugin.
    louiscad

    louiscad

    3 years ago
    @msink I've checked out your project along with a few other ones (Stately, kotlinx.coroutines & SqlDelight), that has been helpful. I'm working on seamless sourceSets and targets activation and setup based on whether it's running in IDEA, and the current OS.
    gildor

    gildor

    3 years ago
    Yes, it working of course and it's nice that Kotlin DSL provides facilities to configure it, I just don't like it, and it was a small experiment to use pure maven-publish
    louiscad

    louiscad

    3 years ago
    Someone else replied to me a way to publish to bintray without the gradle bintray plugin: https://github.com/bintray/gradle-bintray-plugin/pull/230#issuecomment-502563195 @gildor Does that look like what you did?
    gildor

    gildor

    3 years ago
    Yes, exactly this
    I just remember how struggle to understand what url should be, it’s not really clear from bintray API docs
    But as I remember i also had to do something with version in the url, but I may be wrong
    louiscad

    louiscad

    3 years ago
    Hum, that might be possible as the bintray plugin has some config for versions. Please share if you find that back before I figure it out.
    gildor

    gildor

    3 years ago
    no-no, I mean version of the library (publication)
    but again, I may be wrong, I would just try this snippet
    louiscad

    louiscad

    3 years ago
    I found that blog post from Bintray themselves giving an exemple to construct the url: https://blog.bintray.com/2015/09/17/publishing-your-maven-project-to-bintray/
    Vasily Levchenko

    Vasily Levchenko

    3 years ago
    This
    1.8.4-jetbrains-5
    isn’t what is recommended to use. It was just the least effort to publish big bunch of artefacts without mass modification of build scripts. Recommended scenario is using
    maven-publish
    plugin even for bintray repository configured as repository of maven type. Here is a good example how to do it https://github.com/h0tk3y/better-parse/blob/fb2d5fb81b2a5eaccbe30419295f21ce1f70e5b0/build.gradle#L126
    louiscad

    louiscad

    3 years ago
    That's what I'm seeing as some artifacts are not published correctly. The maven-publish plugin seems ages slower than the bintray plugin though! It's at 43% uploading right now, after ~12 minutes executing while it took about 2 minutes to go to 100% with the bintray plugin. Correct is better than fast sure, but I'm wondering it you also encountered this.
    gildor

    gildor

    3 years ago
    I’m pretty sure it’s some bintray issue
    also, are you sure that you do not publish to maven central?
    I would just restart publishing
    even 2 minutes is very-very slow
    louiscad

    louiscad

    3 years ago
    Well, I have over a hundred of artifacts, so 2 minutes is the best I could get with optic fiber (60Mbps UP) 😅
    gildor

    gildor

    3 years ago
    how big each artifact is?
    louiscad

    louiscad

    3 years ago
    I'll leave the laptop there while I go to lunch, hopefully, publication usage will be successful.
    gildor

    gildor

    3 years ago
    I mean in general it’s just upload of files to storage
    louiscad

    louiscad

    3 years ago
    The artifacts are small, a few kilobytes (most only 1). I have about 40—50 modules, all with metadata and multiplatform publication. 3 of them target macOS, and all 3 iOS architectures, and 3 target JS IIRC
    gildor

    gildor

    3 years ago
    still shouldn’t be a problem (only if bintray is super slow)
    louiscad

    louiscad

    3 years ago
    oh, I spotted Jetbrains Toolbox having sent of 9GB of data 🤔 🤔 🤔
    It's not getting much faster after closing it though
    nothing is uploading in parallel BTW
    I checked bintray, the number of files is over 750
    97% after 28 minutes
    BUILD SUCCESSFUL in 29m 49s
    552 actionable tasks: 416 executed, 136 up-to-date
    total artifacts published: 824 Even bintray took ages to decrease the "artifacts to be published" counter, I could refresh the page and see the number go down 😂
    gildor

    gildor

    3 years ago
    Did you enable parallel builds in Gradle? At least each module publishing should be parallel
    So, I would try Gradle 5.6rc to validate that it's actually fixed
    louiscad

    louiscad

    3 years ago
    So I'll have to wait for 5.5 to become available as 5.5 is still the one under RC.
    Moving away from bintray gradle plugin made Android publications no longer work BTW, I'll investigate
    Probably misconfiguration on my side as it stops working for mavenLocal too
    That's right, I had removed
    publishLibraryVariants("release")
    from somewhere but forgot to put it in new buildSrc code while trying to simplify the logic and supporting Kotlin/Native targets
    gildor

    gildor

    3 years ago
    A side note, waiting for official Android publishing support, it's shame that it took so long for Gradle to expose public API for thishttps://issuetracker.google.com/issues/37055147
    louiscad

    louiscad

    3 years ago
    Oh, after 4 years, it'll finally come in 3.6 soon™! 🙃
    Consuming the library from an Android project now fails with errors like this:
    Variant 'android-releaseApiElements' capability com.louiscad.splitties😒plitties-fun-pack-android-base-with-views-dsl:3.0.0-dev-015:
    - Incompatible attribute:
    - Required com.android.build.api.attributes.BuildTypeAttr 'debug' and found incompatible value 'release'.
    I'm not sure how I can fix this. I want to publish the release variant only.
    russhwolf

    russhwolf

    3 years ago
    You might need to specify matching fallbacks in the consumer code https://kotlinlang.org/docs/reference/building-mpp-with-gradle.html#publishing-android-libraries
    louiscad

    louiscad

    3 years ago
    That is a less than nice burden added on the library consumers!
    russhwolf

    russhwolf

    3 years ago
    yeah I agree
    so now I publish release and debug to make it easier on users
    I’m not aware of a way to configure a fallback like that from the library side but that would probably be a nicer solution if it existed
    louiscad

    louiscad

    3 years ago
    It'd nice for it to just behave like jvm targets (using only the release variant). @h0tk3y Do you know how to properly publish only the release variant and have it just work on the consumer side?
    h0tk3y

    h0tk3y

    3 years ago
    AFAIK, you can't define the matching fallbacks on the producer side, as the matching fallbacks work by altering the attribute compatibility rules in Gradle variant-aware dependency resolution, and Gradle module metadata in published libraries can't include anything that would affect those compatibility rules on the consumer side. So the options are to publish the debug variant as well or to make the consumer specify the matching fallbacks, which is described here: https://kotlinlang.org/docs/reference/building-mpp-with-gradle.html#publishing-android-libraries
    louiscad

    louiscad

    3 years ago
    @h0tk3y this doesn't seem to work well with custom
    artifactId
    (defined in
    mavenPublication { … }
    called in
    kotlin { … }
    on the android target). Ideally, I'd want only one variant. Isn't there a way to just not have that variant-aware metadata, so it's possible to use the library in any variant with zero additional configuration for consumers?
    h0tk3y

    h0tk3y

    3 years ago
    You can try
    publishLibraryVariantsGroupedByFlavor = true
    , which will group all build types into a single module for each product flavor, so there's only one artifactId per flavor, and the module metadata will contain a variant per build type. However, this won't work for custom build types declared by the consumer which are not present in the published library. I'd say we can experiment with removing the build type attribute from the published variants, so that, on the consumer side, the variants are only matched by the product flavor, if any. The current implementation doesn't modify the attributes added by the Android Gradle plugin, because we just don't own those attributes. Could you please file an issue at https://kotl.in/issue?
    louiscad

    louiscad

    3 years ago
    With the following snippet, I'm configuring the
    mavenPublication { … }
    with a custom
    artifactId
    for each target (I call
    configure(targets) { configureMavenPublication() }
    in each
    build.gradle.kts
    ). Am I applying your suggestion correctly?
    fun KotlinTarget.configureMavenPublication() {
        val suffix = when (platformType) {
            common -> "-metadata"
            jvm -> ""
            js -> "-js"
            androidJvm -> ""
            native -> "-${name.toLowerCase(Locale.ROOT)}"
        }
        mavenPublication {
            val prefix = if (isFunPack) "splitties-fun-pack" else "splitties"
            artifactId = "$prefix-${project.name}$suffix"
        }
        if (platformType == androidJvm) {
            (this as KotlinAndroidTarget).publishLibraryVariants("release")
            this.publishLibraryVariantsGroupedByFlavor = true
        }
    }
    
    private val KotlinTarget.isFunPack: Boolean get() = project.parent?.name == "fun-packs"
    h0tk3y

    h0tk3y

    3 years ago
    No, because this way, the module will still only have a variant for the
    release
    build type, but not for the
    debug
    one. Use
    publishLibraryVariants("release", "debug")
    and
    publishLibraryVariantsGroupedByFlavor = true
    to publish both of them within one module.
    louiscad

    louiscad

    3 years ago
    How does grouping work? What is called "module" here? I'm not aware of a multi-variants module publishing method in maven and gradle world.
    h0tk3y

    h0tk3y

    3 years ago
    • An Android variant denotes the product flavor if any (
    demo
    ,
    paid
    etc.), and the build type (
    debug
    ,
    release
    ). • With Gradle
    maven-publish
    plugin, each publication produces a Maven module with its own artifact ID. A module that is published with Gradle module metadata may contain more than one variant, each with its own artifacts and dependencies. • Without
    publishLibraryVariantsGroupedByFlavor
    , a publication is created for every Android library variant, such as
    fooBarDebug
    or
    fooBazRelease
    . Within each such a publication, there's only one variant. • With
    publishLibraryVariantsGroupedByFlavor
    , the variants are grouped by the product flavor, so that variants that only differ in the build type get published by a single publication (and are put inside as its variants). If you don't have any product flavors, this results in only one publication (i.e. one Maven module) that has a variant per Android build type. These variants are marked by the Android plugin's build type attributes.
    louiscad

    louiscad

    3 years ago
    May the android library publication story get simpler, allowing to publish without the variant metadata as the following issue is finally resolved in AGP 3.6? https://issuetracker.google.com/issues/37055147#comment12 Maybe there could be a collaboration with Google Android tools team?
    Finally got library consumption to work with
    publishLibraryVariantsGroupedByFlavor = true
    , but now, the IDE no longer navigates to source (for Android variant), showing only compiled class file, even for Android sources only modules… I didn't have the issue on previous versions. Is there a way to fix this?
    Okay so I know why it worked in previous releases: the bintray gradle plugin didn't upload the
    .module
    files, so the
    .pom
    files were used, and we could call it a day. Now with the
    maven-publish
    plugin, the
    module.json
    file that restricts the variant down the consumer is taken to its final form with the
    .module
    extension, even for Android libraries, and is uploaded to bintray. I didn't find the
    .module
    files being part of the
    artifacts
    of the
    MavenPublication
    objects found in
    publishing { publications.withType<MavenPublication>() }
    although I found the
    .aar
    files, so I don't know how I can exclude them so they are not deployed to mavenLocal, nor bintray (BTW, that explains why mavenLocal didn't work for me after I moved to MPP library publishing). Do you know how I can exclude these files, or if that is a feature that could be added to the Kotlin multiplatform gradle plugin when dealing with android libraries?
    h0tk3y

    h0tk3y

    3 years ago
    @louiscad With Gradle 5.3+, the module metadata is always consumed from repositories, and the feature flag
    enableFeaturePreview("GRADLE_METADATA")
    only enables its publishing. If you don't want any module metadata to be published, you can just disable the feature. AFAIK, there's no more granular way to disable it with public API in Gradle.
    You can also try to alter the module metadata file and exclude some attributes from it after it is generated. There are task that generate the module files, so modifying their actions might work. I didn't try that personally, though.
    louiscad

    louiscad

    3 years ago
    @h0tk3y I just found a way to prevent the
    .module
    from being generated! Here's a type unsafe way (because
    android
    might be in the name without the task being for the android artifact):
    tasks.withType<GenerateModuleMetadata>().matching {
        t.name.contains("android", ignoreCase = true)
    }.all { enabled = false }
    It works correctly. I wouldn't have found this technique if Google didn't led me there: https://github.com/gradle/gradle/blob/40a006b94b9cbfa4f9ed409f6f74b2828941cfd4/subprojects/maven/src/main/java/org/gradle/api/publish/maven/plugins/MavenPublishPlugin.java#L188