https://kotlinlang.org logo
Join Slack
Powered by
Has anyone actually managed to set up a reliable w...
# random
m
Has anyone actually managed to set up a reliable way to publish to GitHub Packages? I’ve tried for the first time. The process failed midway due to network error and now I have an incomplete publication published 😵 I can neither retry publishing (409 Conflict) nor delete the incomplete publication (GitHub doesn’t allow that). WTF?
l
I'd like to say that GitHub packages suck because it's subpar compared to npm (also GitHub owned) and Maven Central, from a consumer project perspective. Having to auth to depend on public stuff is nonsense, so by principle, I'm against it until they change this. This is bad and they should feel bad (and know why). Ranting done 😄
m
I tend to agree. It's really bad UX. I can't believe that you have to auth to ond on public stuff. Will check. I'm annoyed that publishing is a mess and that I only have one repository per library (GitHub repo) and cannot have one big Maven repo for all my libs.
l
Maven Central is probably what you're looking for anyway. Or did you want to publish dev versions that are neither snapshots, nor alpha/beta/milestone/rc/stable?
☝️ 1
m
I publish to Maven Central but that's annoying too. It involves manual work on sonatype to publish each library. And it takes way too long for new versions to become available for download. I need something instant in addition to Maven Central.
l
You can use staging for your instant needs
m
Hmm. I'll give it a shot.
l
Regarding manual stuff, are you sure? I'm pretty certain publishing can be automated entierly once the maven group has been approved because you own the domain or use a github repo url derived group.
m
I publish to OSSRH with Gradle. Then I have to manually close the staging repo on sonatype and wait until it was validated. Then I have to manually release the staging repo on sonatype to queue it for syncing with Maven Central.
Bintray did that on Maven Central sync so there's probably an API somewhere
e
there is a well known issue opened, there is an actual workaround as there described, but for the moment I'd pass
l
@mbonnin Can you confirm publishing a release to Maven Central cannot be fully automated?
e
you need to contact the staff to get some package removed
they still have some road to do before being useful to the mass
you may want to try jitpack, fast and easy
❌ 2
m
Hi everyone!
@mbonnin Can you confirm publishing a release to Maven Central cannot be fully automated? (
Everything can be automated (except the initial jira ticket)
l
Naaaah, not jitpack, allowing to swap publications for 7 days isn't showing security considerations, by principle, I'm against it 😅
m
The close and release stuff might require some retry heuristics but ultimately, it should be automatable
👌 1
👍 1
Please no jitpack 😅
e
sorry, addicted
l
Yep, being resilient to temporary network and server errors is a nice thing.
m
One issue that Maven Central has and that I ran into several times is that if I publish multiple libraries in parallel they mess up each other's process. That is because they all end up in the same staging repo and lib1 publishing may close that while lib2 is publishing. Bintray had that issue
e
the possibility to have anything available (from artifact to plugin) just one build time away is quite comfortable
l
fand? @Marc Knaup
m
Autocorrect 😁
m
@Marc Knaup I think that is fixable by creating a different staging repo every time. I'm working on something, will publish the repo tonight
l
@Marc Knaup For bintray, I had an issue with parallel publishing, but if I did parallel uploading and committed the publication after it, retrying it until it succeeded, it'd work fine.
@mbonnin Already?! That's fast
m
@louiscad I didn't say how feature complete it'll be 😉
m
@mbonnin is that possible? I only know the ray where sonatype automatically creates one if there is none. How would I coordinate syncinc across different systems? Multiplatform cannot be published in one go.
m
@Marc Knaup I'm almost sure this is possible. For multiplatform, you would create different staging repos for different platforms. This is ok to close separately if there's no overlap in the artifacts
l
@Marc Knaup Multiplatform can be uploaded in multiple stages then be published in one go once everything is uploaded.
m
@mbonnin that wouldn't be atomic. How would the last part know what other staging repos to publish?
m
It doesn't need to? Meaning your windows artifacts will show up later on mavenCentral but it's not a huge issue?
☝️ 1
l
@Marc Knaup You mean how would the last part know it's the last part?
m
Well people will complain that a new version isn't working correctly. Just as the did with Kotlin and Ktor already 😅
❓ 1
I'd prefer a clean solution.
m
Why would they complain ? We're talking a few minutes delay
m
"This lib 1.2. If it's live, it's ready."
m
People don't need realtime updates 😉
➕ 2
m
Maven Central can take way longer than minutes
It can take hours sometimes
m
Just wait for windows to finish before publishing the lib?
m
One reason I've never liked it
m
Ah yea if it takes hours then it's an issue but not really linked to separate staging repos. More about performance of the thing
l
@Marc Knaup That was a few years ago, I've heard they have a significantly better infrastructure orchestration since then, and it's minutes, as Martin says.
m
Especially right now where they have scheduled maintenance every day 😅
But overall I agree with @louiscad, my releases usually show up in a matter of minutes (and each version contains 1128 artifacts)
m
@mbonnin if I publish staging repos at different times then some are published way sooner than others. If all together are published later it's a little less of an issue.
m
Yea fair enough
The thing is... there's no real alternative
I'd love a realtime maven central but both jitpack and Github packages have serious drawbacks
You can always publish to your S3 bucket I guess if you want realtime
m
We could publish atomically to Maven Central but it involves shared state within a distributed multiplatform build. Probably possible with GitHub actions.
m
Oh yea you could pass the stagingRepositoryId around 👍
👍 1
l
One solution is hosting on your own on GCS, Scaleway, AWS S3 or wherever.
m
That's a nice project
m
I'd like to avoid managing own stuff like repositories. Building libraries is a lot of work already 😅
☝️ 2
➕ 2
l
For self-hosting, I'd prefer GCS to S3 because GCP has shown more commiment to not burning fossil fuel to power datacenters compared to AWS, though it's another topic.
🌱 3
m
When there's a Kotlin update I have to publish 18 libraries. And they depend on each other. So if Maven Sync takes 30 min on average I may have to wait a few hours before I can have all of them public. Non-automated.
l
@Marc Knaup When there's a Kotlin update, I don't update my libraries because there's compatibility even in Kotlin/Native 1.4 ☺️
m
I've used jcenter until now to publish fast. Then sync to Maven Central later on.
l
1.5 might break this though
m
Arg yes I see, if you have to daisy chain publications yea that might take some time
m
Yeah backward compatibility is another mess and topic 😅
l
Personally, I'll keep on using bintray for dev versions until it dies and until maybe we find a convenient alternative.
m
Looks like the GCS thing is what you want. Publish on MavenCentral for public consumption but have a fallback to your own GCS bucket in your project for fast updates
☝️ 1
l
A lot can happen in 3 months, especially with folks like @mbonnin 😄
m
But even that would be non-automated?
l
What would be non automated?
m
Let's spam GitHub with feedback 😄 Maybe they put someone in charge who actually knows the Maven/Gradle/Java/Kotlin ecosystems.
m
If fluid-B depends on fluid-A you have to wait on fluid-A to be published before triggering the build for fluid-B ?
l
Great idea, tonight, I'll discuss preparing a class action to get GitHub to change how they do the thing.
😂 2
➕ 1
m
Yes. Maven Central has a high level of requirements. If my lib was published on jcenter didn't even guarantee that it is publishable on Maven Central 🙄
m
Talk to me about that, I'm changing the pom of our old artifacts so they can land on Maven Central 😕
l
@mbonnin You can build against a maven local build or against one not yeat published AFAIK
m
@louiscad yea but that means you have 
mavenLocal
in your build scripts
m
Maven Local is worst. I had lots of publishing issues the last week and didn't even notice because Maven Local was hiding the fact but only from me.
m
Ultimately you want everything to run from CI I guess, make sure builds are reproducible
l
You can also use a local file url repo that you delete before use
m
Jcenter had a major issue where some packages of published libs were simply missing from jcenter sync
l
That file repo can be somewhere gitignored in your project hierarchy
m
Their sync between my Bintray repo and jcenter repo was broken or failed for whatever reason. I didn't see that because Maven Local was working fine.
😬 1
m
@louiscad still having builds depend on local state isn't great
m
Like for example the JVM target simply nonexistent for a multiplatform release
l
@mbonnin If you know the build is reproducible and should be the same, and you check before publishing to Maven Central, it's not an issue IMO
m
Humans make mistakes, I would'nt rely on myself to do the check
🤖
m
Lots of manual work and doesn't scale
🤞 1
l
Also, you can condition adding a local repo to CI runs or publishing phase. Let's not throw the baby out with the bathwater
m
Also I want the CI to hold my secrets, not my laptop
l
If in addition to that, you delete the local repo beforehand, I really don't see any problem with that, and you don't depend on the remote repo.
m
I'd love to have some publishing plugin that stages one lib publication, then validates it with Kotlin specific checks too, then makes them public, then syncs to Maven Central.
l
@Marc Knaup I had this for bintray/jcenter, and with @mbonnin, we'll try to do the same for Maven Central
m
Kotlin specific checks
What are Kotlin specifics checks ?
m
I I delete Maven Local im back to waiting for Maven Central to complete before publishing the next lib.
l
@mbonnin I guess it's checking the dependency resolves properly (e.g. metadata for MPP), and run tests against the published artifacts maybe too.
m
@mbonnin that all targets are there, all required files for each target and that hey are nonempty.
I had a weird case yesterday where the Metadata jar was empty.
m
Ah yea, I remember that. This has never been an issue for me though.
🤞
l
@Marc Knaup You can use NOT mavenLocal but another local file repo you put where you want, that's what I've been talking about for several messages now.
m
Just because I've added javadoc jar to another target (the wrong way)
m
(but we don't publish windows, I guess that's why, we can run everything from the same VM so far)
m
@mbonnin even macOS and Linux?
m
Well, we don't publish linux actually 🙈
m
Ah :)
m
It's all Android + iOS really. We've had a few folks asking for windows but never linux. We should do it though..
m
I've avoided linux and windows in my libs just to avoid the publishing mess 😅
m
Yep, I've been reluctant to move this to the top of the todo list for this exact reason 😅
l
I initially started with macOS only publishing as well I must confess.
m
Are the staging repositories publicly available?
🤷‍♂️ 1
m
I've learned today that even publishing a single staging repo isn't atomic on Maven Central. Maybe due to caching. Several publications of a multiplatform library appeared at least 10 minutes later than others.
m
This is not great but on the other hand, this feels like expected? The web isn't atomic in general with CDN and various levels of caches everywhere with different expiration times?
m
That's what cache-busting headers are for in the web. But they didn't do anything either.
m
I just bumped into this: https://status.maven.org/
It shows the publish latency. It's usually ~10min but spiked to 167min earlier today
3 Views
Open in Slack
PreviousNext
Powered by