https://kotlinlang.org logo
Join Slack
Powered by
Which gradle plugin should I use to publish KMP pr...
# multiplatform
r
Which gradle plugin should I use to publish KMP project artifacts to maven central using Central Publishing Portal, which is the only method available since February 1st, 2024 (for new registrations)?
I'm trying to use https://github.com/vanniktech/gradle-maven-publish-plugin, but it doesn't work. I think it supports only OSSRH method.
Sonatype gives two examples of gradle plugins supporting central portal, but these projects are very fresh (2 weeks old) and I have no idea if they even support KMP.
j
Can you link those samples?
r
It's a bit frustrating after hours of configuring my project. registering domain and verifying namespace, to find out I can't publish anything because the only available method is not supported at all in gradle 😱
j
I have checked only one of those plugins and looks like it is hardcoding the main source set so it would not be compatible with KMP
c
https://central.sonatype.org/publish/publish-gradle/ Only states that from feb. 1st a new repo url is used. πŸ€”
j
That is 2021
πŸ˜‚ 2
πŸ™ˆ 1
r
It's old info about OSSRH
j
Try publishing it to maven local (signing the artifacts) and creating a zip file to upload it manually via the new GUI
r
OSSRH (including S01) is legacy now
πŸ‘ 1
j
Create empty docs jar too, to avoid wasting time
r
That's an interesting option πŸ™‚ I wonder if it works with all my 50 components in one zip ...
j
I cannot help much more as I am still able to publish to the very old repository
h
Sounds like a good change to implement this repo into Gradle by default...
βž• 1
r
Yep, my other project still works with old OSSRH, too.
j
Do you mean any new project under the same domain needs to use the new approach?
That would be pretty annoying
h
I really don't understand why I need another plugin to publish my artifacts to maven central, when there is also a 
mavenCentral
shortcut.
r
I have new domain.
h
It only affects new top level groups
r
It would probably work fine for old one.
j
I don’t know why maven-publish is not enough. Have they blocked indicating any url or something so?
βž• 1
there it is indicating a URL
TBH it has no sense they haven’t provided an alternative for Gradle before those changes.
e
I think the repo for downloading artifacts will be the same, and this is just for uploading artifacts. So 
mavenCentral
should still work
h
Yes, 
mavenCentral
still works, it's just strange you need another plugin to publish to mavenCentral.
r
Even for maven new plugin is required (at least they released it in time πŸ˜‰)
h
And why do I need no custom plugin when publishing to GitHub Packages? What makes Sonatype so special? πŸ˜„
r
Just to upload a few files with some auth ... ;-]
e
I hope there are benefits that justify this πŸ˜…
j
Microsoft and other big companies was breaking uploading anything constantly due huge loads. Probably all of this is related to improve the whole setup.
Have anyone tried the previous setup but changing the url to the new one? Maybe it works πŸ€”
r
I've tried. It doesn't work.
sad panda 1
The old api seems to be more complex. It tries to create some staging repository. The new api is very simple - it's just a zip upload - https://central.sonatype.org/publish/publish-portal-api/#publishing
m
Sonatype support can take you back to OSSRH if you want to
Looks like this is the best option looking at this thread πŸ˜•
r
I know. But where is fun in this ? πŸ˜‰
m
Well... At least gives them signal that "new" isn't better
Can you even publish SNAPSHOTs with the "new" thing?
r
m
It's a bummer, I hope that's not the final plan...
h
m
Oh my...
j
but it will not be supported
no sense
e
I understand the justification for not allowing SNAPSHOT. It's convenient, but there are issues with them. They mention a replacement solution, but it sounds like it'll be meant for the developer only instead of the public 😞
a
Does the new API just need a zip of the published artifacts? If so, I think the simplest workaround would be using the existing 
maven-publish
plugin, publishing the KMP project to a project-local directory, and then zipping + uploading it to the new API.
h
Does the new API just need a zip of the published artifacts?
Yes, this is all and it's not hard to implement by yourself. But still annoying.
a
it's annoying that it doesn't work right now, but it sounds like a better plan for the future :) Except for the lack of SNAPSHOT support
m
The inability to upload from different machines is also a bit of a step back. Previous version allowed you to open a repo and upload there. Now it's all in one go
h
But if Sonatype already changed its api, why they did not just use the default api supported by maven/Gradle without any 3rd party plugin at all?
m
I guess it's stateless now so easier for them 🀷
Except it's not, right? Because closing/releasing are still separate API calls so there is still some state there. I don't know
j
dropping github, buying usb, temporary using Google Drive and Dropbox
h
till separate API calls
If you set 
publishingType
to 
AUTOMATIC
(default), its just one call.
πŸ‘ 1
j
I feel like GitHub could steal a lot of attention if they would allow public packages
βž• 2
h
GitHub already supports Maven/Gradle packages :D
j
public* sorry I am from mobile
AFAIK with github you need to auth to be able to use the repository
βž• 1
h
Oh yes, no user auth for public packages is another story...
m
If you set 
publishingType
to 
AUTOMATIC
(default), its just one call.
They still support 
USER_MANAGED
though so they have to build the statefull stuff anyways...
h
But at least you need no 3rd party plugin to publish packages to GitHub packages πŸ˜„
j
Yeah, but the problem is consuming them
if at least it would support no auth to use it as you can do with mavenCentral or google ones…
do you know if there is an open request for that?
a
Feb 25, 2021
Still in the future column on the roadmap. Hopefully in the fall this year.
j
can you share the roadmap?
a
I suspect it's an internal GitHub roadmap
πŸ™ƒ 1
e
Was there anywhere to submit feedback to Sonatype?
j
Jira and Joel. I will miss 😒
m
<mailto:central-support@sonatype.com|central-support@sonatype.com>
is the best I could find
thank you color 1
progress πŸ˜…
Copy code
Cannot publish to maven central (status='500'): {"httpStatus":500,"errorCode":10500,"message":"Internal Server Error occurred","explanation":null,"data":null}
@Robert Jaros I wrote a simple plugin there: https://github.com/GradleUp/nmcp It's not perfect but if it can save the trouble of zipping and uploading, it's already something!
πŸš€ 1
On the plus side, the validation seems to happen much faster 🀞 . That plugin was published using the "old" publishing and waiting for "close" was not fun
e
How was the propagation time for syncing to Maven central?
m
I haven't checked that actually, I tried on a placeholder empty project and didn't want to pollute the namespace πŸ˜…
Let me try
I would expect that part to be mostly unchanged, the CDN, etc... is probably the same
Alright, button clicked, now pressing F5 on https://repo.maven.apache.org/maven2/net/mbonnin/tncmp
(Or having Jake's script do it for me ideally, if I can find it again)
Published πŸŽ‰
So ~6 minutes which seems to be better than my usual publishing times but I've seen many different values there so it's hard to tell for sure
The nice thing is that there is a "publishing" status now instead of previously missing staging repo and the hopes of it appearing on maven central at some point. At least now I get the warm fuzzy feeling that my artifacts are still somewhere in transit πŸ™‚
e
Yeah I've seen it take 30+ minutes and I've seen it take less than 5 minutes. Hopefully it'll be more consistent but Β―\_(ツ)_/Β― They have webhooks now, but not sure if it includes info about syncing.
r
@mbonnin how can I use your plugin with a multimodule and multiplatform project? Can I somehow use 
nmcp {}
block only once in the root project?
m
Not really, it's meant to be used per-project. If you have a convention plugin you can apply it there and iterate the publications
More generally, with project isolation around the corner, I expect plugins to do this more
If you want to publish all publications, something like this should do:
Copy code
nmcp {
  publishing.publications.configureEach {
    publish(name) {
      username = System.getenv("MBONNIN_MAVEN_CENTRAL_USERNAME")
      password = System.getenv("MBONNIN_MAVEN_CENTRAL_PASSWORD")
      publicationType = "USER_MANAGED"
    }
  }
}
r
But then I will see multiple deployments in the central portal gui, right?
m
yes
One for each publication
r
50 for my project πŸ˜‰ (edited, still a lot πŸ˜‰)
m
πŸš€
Ahah that's a lot of clicking!
Although if you do 
AUTOMATIC
, it should all work.
I guess the alternative is creating an aggregate task in the root project
Mind opening an issue? I'll look into what it takes to do it.
r
m
https://github.com/GradleUp/nmcp/commit/e1bcc9f801c17b888a84416135d4f395dc2aa130 adds 
publishAllSubprojectsProbablyBreakingProjectIsolation
blob joy
Copy code
nmcp {
  publishAllSubprojectsProbablyBreakingProjectIsolation {
    username = TODO()
    password = TODO()
    publicationType = "USER_MANAGED"
  }
}
h
You could support PI though 😜
m
You can if you do the "long" form:
Copy code
nmcp {
  publishAggregation {
    project(":module1")
    project(":module2")
    project(":module3")
    
    username = TODO()
    password = TODO()
    publicationType = "USER_MANAGED"
  }
}
But you can't add dependencies to other projects without breaking isolation, right?
e
Another way to do it without breaking project isolation would be to have a plugin that gets applied to to each project regularly that publishes the source somewhere local, and have a root level plugin that zips them and uploads it
πŸ‘ 1
m
@eygraber in all cases, the root project needs to depend on a list of subprojects
Can you do that without listing all subprojects manually?
h
But just to understand it, is it possible to upload different modules with the same version automatically, eg uploading from different OS/runners, uploading in parallel, or support isolated projects without a umbrella task? πŸ€”
r
Even listing all subprojects is totally acceptable. It's a one time thing.
e
in all cases, the root project needs to depend on a list of subprojects
Not if the subprojects write to a well known location
Unless I'm missing something
m
Not if the subprojects write to a well known location
Means it depends on you calling 2 different Gradle invocations, right?
Copy code
./gradlew createAllZips
./gradlew publishAggregate // this step depends on the previous one being called just before and no one messing with the build directories in between
is it possible to upload different modules with the same version automatically, eg uploading from different OS/runners
you can't upload a single bundle from different runners
you can upload different bundles though which might be fine
Unless you need "atomicity"
j
build service should not be enough?
e
You can have each project get a reference to the root level task (using a build service or something like that) and do 
rootTask.dependsOn(projectTask)
and then just run the root level task.
m
Copy code
rootTask.dependsOn(projectTask)
Is that PI compatible?
j
no need to depends on rootTask I think
that breaks isolation
m
Using strings maybe?
e
If you have a reference to the 
TaskProvider<*>
I think it would be PI compatible
m
rootTask.dependsOn(":module:createZip")
e
As long as you're not referencing the 
rootProject
instance directly
j
another option is a configuration and adding all projects to the root one with it
like kover does
m
As long as you're not referencing the 
rootProject
instance directly
But how do you get the 
rootTask
?
j
that is the problem, picking the root project
m
TBH, I'm not even sure Gradle themselves know the imits of what is PI compatible
βž• 1
e
j
all files need to be uploaded in the same zip?
if so, the build service will not be enough I think, it should be to β€œinitialize the repo” so the rest of tasks could upload into it
m
> initialize the repo there is no "staging repo" anymore, it's all "upload a bundle".
j
project aggregation approach is the only way I think
e
Here's what I'm thinking: 1. Set up the build service and have an instance of 
TaskProvider
there for the root task 2. Each project applies a plugin that puts the artifacts to upload into a well known location 3. They also get the 
TaskProvider
from the build service and make it dependent on their task 4. After all the project tasks run, the root task will run, zip all of the artifacts that the other projects created, and upload them
πŸ‘€ 1
πŸ‘ 1
j
how you ensure the root task is the last running
e
That's the one that the user invokes at the command line e.g. 
gradle :rootTask
Since all of the projects make it dependent on their tasks, it will not run until they are all done
j
it is the same issue with aggregation reports IMO
instead of creating a report in the root, create a zip and upload it
e
Just to reiterate, I'm pretty sure this is PI compliant, but not 100% sure
instead of creating a report in the root, create a zip and upload it
I'm not following, what do you mean by that?
j
e
You mentioned an issue with aggregation reports. What's the issue?
j
No, I mentioned it is the same problem
aggregated reports at root = zip and upload at root = same solution
e
Ah got it
r
@mbonnin Unfortunately I can't make it work with 
publishAggregation
. When I add a module with 
project()
I've got an error 
nmcp: no aggregate dependencies found
. Adding a different module gives me another error: 
Entry META-INF/MANIFEST.MF is a duplicate but no duplicate handling strategy has been set.
m
Ah interesting, I got this earlier too. I should add integration tests. Will do in a bit
j
integration tests
Do you mean functional tests? or you can test everything with integration ones?
m
@Javier I never really know what people mean by "functional" vs "integration". I just mean something that tests the Gradle plugin a bit more end to end
In that case, it's testing the .zip file has the expected contents
I could check with a mockserver that the network request has the expected shape as well and replicate the sonatype verification logic for signatures and sources artifacts but I'll just assume that part is working as designed πŸ˜„
Tests are here
j
functional would be the whole plugin with test kit or similar approach. Integration is with the project builder APIs. This in the context of a gradle plugin.
πŸ‘ 1
m
> 
Entry META-INF/MANIFEST.MF is a duplicate but no duplicate handling strategy has been set
@Robert Jaros sorry about that, this is because the plugin isn't applied in the submodules and so the aggregate publication is trying to zip the 
.jar
file in each submodule (instead of the publication bundle). If you want to go the project isolation route, you'll have to add the plugin to each module. I have updated the README
But I would just call 
publishAllSubprojectsProbablyBreakingProjectIsolation {}
TBH. The name is scary but for the time being there's no drawback to using it.
And by the time Gradle ships PI, I hope we have better ways to publish to Sonatype
r
Using this fails on submodules which are present in my project but are not to be published (and don't apply 
maven-publish
plugin)
Using the previous way it works partially. It still fails on one of my subprojects, with 
nmcp: no aggregate dependencies found, please apply the 'com.gradleup.nmcp' in your submodules
, even though I've applied the plugin. I don't see any difference in this submodule configuration - the only one is it's location - directly below the root project (the other subprojects are in their own directory).
This is my configuration: https://github.com/rjaros/kilua/blob/main/build.gradle.kts#L45-L68. The generated zip files contains correct publication files from all listed subprojects except the first one.
m
Thanks for the reproducer project! This is https://github.com/gradle/gradle/issues/847: 
:kilua
is resolved to 
:
because they have the same GAV maven coordinates
Easiest fix is to rename 
kilua
into 
kilua-core
or so to avoid the name clash
Using this fails on submodules which are present in my project but are not to be published (and don't apply 
maven-publish
plugin)
Right, good call, I'll make this reactive
j
I change the root project name in settings to avoid issues like that
πŸ‘ 2
for example, 
rootProject.name = β€œkilua-project”
you will see the change in the project view in the IDE
r
Thanks for the suggestion. It fixed my problem.
The zip file looks ok, so I'll try to do publishing now ...
🀞 1
Successfully uploaded. It's being validated now.
πŸ₯ 1
m
Is validation is taking a bit of time? It was almost instant in my tests
r
It's validated now. Success πŸ™‚
πŸŽ‰ 2
m
Niiiice!
r
86 out of 86 Components Validated
m
Ah I see how that can take some time πŸ˜…
r
Nice work with the plugin πŸ™‚ Thank you once again πŸ™‚
thank you color 1
m
Thanks for the use case and feedbacks!
I asked central-support and they said the API might change: > The previous API allowed to do upload file by file on urls such as "https://oss.sonatype.org/service/local/staging/deployByRepositoryId/${repositoryId}/${pathToFile}". This is supported by Gradle out of the box" >> - We will bring this back to the team to see if we can support this in Central Portal, we do know a change is coming to the API that will use a different way of uploading the bundle.
Not really sure what that could bring but maybe? 🀞
r
It will definitely break something for someone 😜
πŸ˜‚ 1
m
59 Views
Open in Slack
PreviousNext
Powered by