https://kotlinlang.org logo
Join Slack
Powered by
Hi everyone, I didn’t want to pollute the Maven Ce...
# library-development
b
Hi everyone, I didn’t want to pollute the Maven Central limits thread here, but for everyone that was engaged there, I have another thing I’d love some feedback on. I am specifically running a more public RFC process on this soon, primarily as a learning from the publisher limits, so we are adapting here. See thread.
Hi all, as I’ve been working through a lot of the Mythos/Glasswing industry response efforts Eg Akrites I’ve been thinking about how that might impact Central, especially wrt to un/under-maintained projects and how that affects the ecosystem when patches can’t be published. I created a draft policy that covers how we are thinking about this. I’ll be posting a blog and a wider request for comments soon, but wanted to float it here as well: central.sonatype.org/policies/care-policy
➕ 1
h
I am generally okay with it, but unlike npm where you often use dynamic versions, in the JVM ecosystem, you normally specify a specific version when declaring a dependency. So in case of a thread, each consumer must update its Maven/Gradle build. And not all artifacts are buildable, eg when depending on classes/resources not available on a CI. And consumer using dependency verification must also add your certificate. So instead of creating a new version of the artifact, if possible, what about just blocking the download of the vulnerable file?
Add an allow downloading it when sending a specific header?
b
In the Central ecosystem, the tooling is so vast (as evidenced by the limits lopsidedness) I have to consider what I can do without tooling changes in anything less than multi-year timeframes. So eg the header thing won’t help.
And yes, the stickiness of versions is a benefit whereas we see less attacks than npm (orders of magnitude less, as in a few more or less probing ones and that’s it) because those versions don’t immediately get upgraded, but on the flip side, things like log4shell keep getting fetched years after they shouldn’t be. I’m not trying to solve all that here, i’m trying to solve for making them available in a way that is easy to consume (eg not requiring a whole new namespace). But this upgrade issue is a generally important thing for each tool to contend with.
e
other ecosystems such as Hackage have metadata-only revisions, that allow for updating dependency versions without a whole new package upload. github.com/haskell-infra/hackage-trustees/blob/…/revisions-information.md
h
Maven Central could also update the POM files, but as I understand, the care process is about updating the jar files, like log4j but without maintainers response.
b
Yeah. Actively rewriting the poms. That’s a much bigger step. Interesting though.
That would have us breaking the immutability standard.
Which most tools wouldn’t know what to do with, eg the expectation is releases don’t change so they aren’t re-fetched.
This is good though. What I think the policy needs up front is some kind of disclaimer: we aren’t trying to rewrite how all the package tools work here 😉
r
I haven't read the proposal yet, but if the goal is finding a way to get security updates to old/undermaintained dependencies and to get old/undermaintained projects to used those updated dependencies, that's always going to have some tension with immutability. But that said I do deeply appreciate the commitment to immutability and wouldn't want to see that weakened except in rare cases and with very good reason.
b
The goal I set out to solve was: what happens when a trustworthy partner/project/company has a patch for a serious thing, but the maintainers aren’t around or able to consume it. Do we sit on our hands? Do we make them publish it in a new namespace, complicating the uptake (especially deep in a transitive tree) or can i split the middle. This is meant to split the middle with a targeted, minimally invasive approach.
o
Having read the proposal in full, it appears to be a very useful instrument to solve that class of problems. I couldn't find any obvious downsides. Of course, the program's success depends on finding competent remediation actors and the capacities of and interactions between the parties involved. Thanks a lot for taking care of this!
gratitude thank you 1
c
As a comparison, the recent ArchLinux AUR attack (truesec.com/hub/blog/supply-chain-attack-compromising-arch-linux-aur-packages-infostealer-rootkit) was entirely based on untrustworthy actors 'adopting' abandoned packages and pushing new versions. I would advise being very, very careful about 'urgent' problems. Attackers tend to love 'urgent' situations because people are less careful about verifying what they are doing. I do like that the proposal makes the bar to trigger very high, and makes the released version clearly distinct from usual versions.
👍 1
b
Yes, absolutely concerned about this, which is precisely why we are defining trusted remediation actors. It means a pre-existing releationship more or less, with groups like Akrites, Lightwell etc that can be vetted, known. Drivebys are still hell - no.
🙏 1
➕ 2
Updated scope section 2 to cover some of the feedback above.
🙏 1
m
Couple feedbacks: 1. A small summary at the beginning of the doc would be very welcome. I understand the context is important in a reference document but that makes the page really long and hard to read. The important part is basically section 10 (Versioning), which I would have probably skipped, weren't for @CLOVIS pointing it to me. 2. I think we want to discourage library authors from using 
-care
versions in their transitive dependencies? In order to allow the team who builds the end binary to make their own decision? If we do, a small paragraph about it would be welcome. 3. This one is more of a personal preference: I'd like the JVM ecosystem to start versioning at 
0
consistently and I would start the example with 
1.2.3.0-care
, 
1.2.3.1-care
, etc...
b
1, there is a summary, what is missing in your opinion? I have a blog that will be going up shortly, and I tried to full circle the relevant context there but without turning the policy into a blog. 2. Interesting point. I hadn’t considered the implication of people updating to use a care version, but isn’t that a good thing generally? If they did so as opposed to eg using the base version, it doesn’t change the ability for the end user to override it (at least in Maven).
m
Re 1:, I feel stupid but I can't find the summary 😅 Attached is what I see. It starts with "purpose" and then definitions, etc... What I'd like to see is the concrete impact for me as a developer: "whenever I see a 
-care
version, it's a patch version that has been released by the Sonatype team and I can find more info in the attached metadata" (what is the file name and the schema of this metadata file? Feels like this info is missing ?)
Re 2: > If they did so as opposed to eg using the base version, it doesn’t change the ability for the end user to override it (at least in Maven). That won't work in Gradle. Gradle always selects the highest version by default when a conflicts arises. (edit: by default)
c
A user can select a lower version on purpose with 
strictly =
https://docs.gradle.org/current/userguide/dependency_versions.html
👍 1
m
Yes, I should have put "by default" up there, message updated
c
Isn't that fine though? If I depend on a library 
a
which depends on a CARE version of 
b
, I think it makes sense that I get the CARE version of 
b
even if I declare an older version explicitly. And if I really don't want to use the CARE version, I can use 
strictly
to force that.
m
I'm not sure I like that. There is 40 years of brain muscle that 
a coordinate
== 
a publisher
. CARE changes that.
👆 1
Do people want to rely on CARE releases? Probably? But I would make that change opt-in rather than opt-out
c
Additional requirements regarding the licensing of the 
-care
contribution: 1. The 
-care
version, including source code etc, should be distributed under the exact same licensing terms as the latest version published by the regular maintainer. 2. The regular maintainer has access to the 
-care
contribution with no limitations whatsoever. (if you need legal terms, we can say it's under MIT or any other extremely permissive license). For context, let's consider this scenario: • There is a critical bug found in one of my libraries. (I don't think I have the reach for the CARE process to trigger, but let's imagine) • For some reason, I'm in a comma or whatever. • CARE triggers, some third party trusted by Central (not by me) publishes a fix. • A few years later, I want to relicense the project. • At that point, I have to get consent from every contributor ever, including the third party, even though I never took part in accepting the contribution.
➕ 1
Additionally, what's the plan in case a library author comes back from vacation/whatever, sees that a CARE release has been published, and is strongly against it? Will you retract the CARE release? But that breaks immutability.
b
“At that point, I have to get consent from every contributor ever, including the third party, even though I never took part in accepting the contribution.” The license applies to the source, if you never accepted a patch, then you don’t have to do anything here, they never touched the source. Eg Central distributing a binary is not the same thing as contributing to your project.
“Will you retract the CARE release? But that breaks immutability.” No. Not unless there was some legitimate and enforceable takedown request type of situation.
m
The license applies to the source
What if my software is GPL? I distribute the jars under the GPL. Then the patch must be somewhat GPL compatible? Who's listed in the copyright section?
b
yes that’s right, the binaries need to be license compatible.
but someone making a binary available on the internet != you accepting that as a contribution to your source.
p
For some (well-established) cases the best "solution" might be to actually redirect requests (or have the tools display some sort of error - even with an empty artefact) to a successor coordinate. The case where I imagine this being relevant is where an old coordinate has been abandoned (maybe the original maintainer is no longer around for it) and a successor/fork has been established at a different coordinate. There will likely be cases where a significant amount of new code still refers to the old (insecure) version (for example an LLM has made it a favourite, or it is part of a popular text book/tutorial). After thorough vetting there is a case to have redirection happen in some way (forcing manual transition may be the best way).
b
Agree. We have done this in some limited cases. We did something like allow one final patch in the old/abandoned namespace and then created a redirect/“relocation pom” for the go forward releases.
Even in that case, it was an interesting situation where the namespace linked to a still existing project but only the java implementation was being abandoned. We worked with the project leads to let them know what was happening and get the virtual nod.
9 Views
Open in Slack
PreviousNext
Powered by