Is anyone else seeing a large number of 409 conflict when publishing to GitHubPackages maven repo ri...

# multiplatform

[Landry Norris]

Is anyone else seeing a large number of 409 conflict when publishing to GitHubPackages maven repo right now? Just had 5 publish attempts fail in a row (added a counter to the end of the version to avoid true duplicates). I typically see false duplicates ~1/1000 artifacts published, but today, it’s been closer to 1/3.

[hfhbd]

Yeah, I get them too, very annoying. And you can't replace them by restarting the job, so you have to either delete the failed artifacts or run it again with another version...

[Landry Norris]

I always opt for another version, since we publish ~60 artifacts. Usually, it’s fine after changing the version and running again, but today, that hasn’t been working.

[Carter]

Are you publishing with Gradle? I’ve encountered something quirky where Gradle itself seems to try to upload the artifact twice, which triggers the 409.

[Landry Norris]

I requested a UI that would let me delete several packages at once, and they seemed receptive to the idea, but it would take time.

[Landry Norris]

I am using gradle. I don’t see this when I publish public artifacts to maven central, but at work, we use GHP for internal projects.

[hfhbd]

Or a staging concept with drop support :D

[Carter]

The symptoms I see are that Gradle shows uploading the same artifact twice in a row, GitHub fails with a 409, and in package registry I just see my JAR but no POM or hashes.

[Landry Norris]

I’d like that, but management isn’t a big fan of having the second step with our small team. They want it to be one step to publish.

[hfhbd]

Yeah, but why does it work sometimes? Gradle "should" be reproducible.

[Landry Norris]

If it’s gradle, I’d question why central is so reliable for me.

[Landry Norris]

GitHub support said they’ve heard reports of the same problem from others, but couldn’t find a cause.

[Carter]

In my case the reupload is noticeable because my artifacts are large.

I think I can consistently trigger it if the artifact is close to 2gb, but not if the artifact is closer to 1gb. I’ve just uploaded 7 1gb artifacts in a row and that’s been working well. You probably don’t have artifacts that large though.

My guess is there’s an interaction between GitHub and Gradle that’s triggering the issue as I’ve seen it. When I fell back to deploying with the mvn command line tool directly, my large 2gb artifact uploaded successfully with no http error.

[hfhbd]

Key is an automatically staging concept like maven central but drop support for failure 🙂

[hfhbd]

Hm, my artifacts, around 30, are very small but I still get the conflict sometimes with random artifacts

[Carter]

I take it back—I just had one fail.

[Landry Norris]

I noticed this once before where it seemed to fail much more often for a day. May just be an outage of some kind.

[Carter]

I’m uploading a 10gb machine learning dataset that’s divided into 10 artifacts to work around size limits, and thought I’d finally worked around it blob sweat smile

[Landry Norris]

I’ll have to find my open ticket with support from last time and comment.

[Carter]

This is probably a bug in both Gradle and GitHub. I’m curious what response code GitHub is returning that causes Gradle to retry the upload?

[Landry Norris]

I’ll have to try again with gradle debug info. From my experience, it’ll likely work just to avoid giving me the debug info to give support. We all know how computers can be.

[eygraber]

I opened a ticket with GitHub about this a few weeks ago. They said it is a known issue and they are working on a fix.

[Landry Norris]

Sounds like they at least found the cause then. They told me that they were still trying to reproduce it a couple months ago.

[eygraber]

This is a known issue with the Maven registry. The team is working on a fix in the next major version. We're hoping this will be released in Jan-Feb, 2023.

[Landry Norris]

Is there an emoji that best represents the feeling when the last artifact in the upload fails after 20 minutes?

[eygraber]

At my company it is called :open-eye-crying-laughing

[hfhbd]

I know it's an old thread but can you share you support ticket number? I created another ticket and maybe you do have some helpful information.

[eygraber]

It looks like I can't access the ticket anymore 🤔

[Marcel]

Any news on this? Did anyone manage to fix it or get around the issue, I have been getting some random 409 errors on the past few weeks when publishing our KMP library

[hfhbd]

If we get an 409 we just delete all packages so you can restart the pipeline again

[Marcel]

Yeah, it's just a waste of time and credits