https://kotlinlang.org logo
Join Slack
Powered by
Uff, after debugging for about an hour I figured k...
# ktlint
p
Uff, after debugging for about an hour I figured ktlint-gradle doesn’t recognize files in src/main/kotlin 🤔
p
That is a kind of hard to believe. In this thread we were investigating another problem with latest ktlint-gradle / ktlint combination. There we did not encounter such problem.
p
Also the custom rules don’t run 🤷
p
I am not familiar with the gradle plugins for ktlint, so I can not really help you. But you seem to combine a migration from 
kotlinter
based on 
0.49
of ktlint to 
ktlint-gradle
based on 
0.50
. Maybe you get more insights when first migrating to 
ktlint-gradle
based on 
0.49
before upgrade to latest ktlint version.
p
It’s the same with 0.49.1 + 11.4.2
At work I use binaries but for OSS projects I have to live with the gradle plugin nightmare unpleasant experience due to https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-storage-and-bandwidth-usage 🙈
p
Are you sure, you want to phrase it like this?
I have to live with the gradle plugin nightmare
p
I don’t mean to offend anyone. It’s just constantly breaking and the community efforts are being completely split between kotlinter and ktlint-gradle with the result that they are both missing resources and can’t keep up with the ktlint core speed
p
Yes, but you can’t blame maintainers of the plugin for that. They are looking for more volunteers to keep the projects running. And meanwhile, I am constantly evolving the Ktlint API which causes the plugins to break. We are closing in on the 1.0 release of ktlint though, we should result in a stable API and less breaking changes.
p
I don't blame and I'm thankful for the work 🫶 I tried to connect the maintainers here in the past so that they can collaborate https://kotlinlang.slack.com/archives/CKS3XG0LS/p1661630298967119?thread_ts=1661381921.902119&cid=CKS3XG0LS
p
I have been involved in discussions like that as well but it didn’t reach conclusions: https://kotlinlang.slack.com/archives/C044J67D4R1/p1664489368417669
m
I can link the issue you've encountered, for reference: https://github.com/JLLeitschuh/ktlint-gradle/issues/524 🙂
p
My preferred solution (which is easy to say because I wasn't involved is:) • move the plugin the ktlint repository • archive the plugins repos • Add disclaimers in the readme: Please do development upstream
m
Regarding state of the plugins, I think things changed over couple of months 🙂 the ktlint-gradle plugin was revived and has proper care now. I'd say it can be considered the recommended gradle integration for ktlint.
move the plugin the ktlint repository
I'm not 100% sure it would be change that much. My reasoning: • Since it would still be maintained by the same people, issues like you faced today would be still present. It wouldn't necessarily address any of the issues people face • It is a good practice to let gradle wrappers pick their own tool version, so Gradle plugin versioning and core ktlint jar would still have to be independent • Issues with incompatibility with newly released ktlint versions can be resolved by introducing https://github.com/Kotlin/binary-compatibility-validator to the core ktlint project, and it would actually be a benefit for all types of integrations (spotless, maven plugins etc) And don't get me wrong, I see benefits of having an official Gradle integration, I'm just not sure merging them would address any of the issues you're describing here 🙂
p
the recommended gradle integration
If that is the case I suggest to move it to a parent 
ktlint
organization and remove this pin: https://github.com/JLLeitschuh/ktlint-gradle/issues/569
m
plus, if I got that well, you also pointed out there is a delay between ktlint release and the plugins actually supporting newly released version. If that's a pain point in your workflow, I can suggest forking the plugin and setting up dependabot/renovate to bump things for you automatically (that's what I started doing 😅)
p
I would prefer, to not to release ktlint until the main API consumers (Ktlint CLI and ktlint-gradle) are ready for release as well. Incorporating the gradle plugin into the ktlint repository would make the ktlint-gradle a first class citizen like ktlint CLI.
p
It is a good practice to let gradle wrappers pick their own tool version
Usually I do agree, but in this specific case I don't. I think that argument makes sense for dependencies which are consumed by libraries. In this case people want to use the gradle plugin because LFS is causing complexities. I would be totally fine to just apply a 0.5.0 Gradle plugin to get ktlint 0.5.0
Okay but to summarize: Everyone is up for incorporating it to the ktlint library itself? ☺️
m
If that is the case I suggest to move it to a parent ktlint organization and remove this pin:
I agree that sounds like viable option 👍 The ktlint repo could be pulled as a git submodule so it can be compiled with latest code with no extra effort (which would address what Paul wrote above) However, I think the challenge here is that someone would have to take ownership of that proces + it requires collaboration from both sides (and I don't think I'm personally one of the sides here 😅)
p
Wouldn’t it be more clear to work from the 
ktlint
repository and move 
ktlint-gradle
to a submodule of the base project (comparable to`ktlint-cli`). Working from one repository will more naturally lead to collaboration. But I can not maintain it by myself because I have too little knowledge of gradle.
and I don’t think I’m personally one of the sides here
Does this mean, you don’t’ want to maintain the plugin in a new setup?
m
Wouldn’t it be more clear to work from the ktlint repository and move 
ktlint-gradle
to a submodule of the base project (comparable to`ktlint-cli`)
it would work too, I think 👍 It would be even better from your perspective, since you'd get immediate feedback if the gradle plugin still works fine + refactoring options would automatically udpate plugin updates. I proposed having a separate repository, because it felt like safer option if maintaining compatibility with older versions would be still a requirement (every adjustment to new ktlint apis would have to be explicitly added and tracked for future use).
Does this mean, you don’t’ want to maintain the plugin in a new setup?
I've tried to start contributing to 
ktlint-gradle
in addition to 
kotlinter
, but it turned out some of the decisions that had been made cause extra effort needed to provide proper support. I sticked with kotlinter, but then, given things weren't moving at a pace I would want to, I created my own, more lightweight plugin (https://github.com/usefulness/ktlint-gradle-plugin) which focuses more on performance rather than supporting all possible cases/workflows. So, as a conclusion, since the discussion here is to move 
ktlint-gradle
code, I don't think I should have much input here 😅 I'll definitely will try to contribute, but since it will be Jonathan's and John's code, I'd assume the direction how the plugin should move forwards will be up to them, and they'll lead the project 🙂
p
Ah, sure I keep mixing up who is working on which plugin. I would advocate that you and @wakingrufus would team up to decide how to create a ktlint native gradle plugin. I don’t care about whether which repository would be taken as starting point or even if an entire new plugin is created.
m
I can try to put down my thoughts into a Github issue + come up with base setup for the plugin (living within base ktlint repository, as you suggested). I'm thinking of just the bare bones, that would allow testing if the setup works for everyone, and would enable us to reuse existing code or writing new one, whatever is preferred later. I should have some free time in 1-2 weeks, so somewhen around then 🙂
btw. I just noticed this issue: https://github.com/pinterest/ktlint/issues/2111 ❤️ This is a great idea, and will definitely be helpful for all types of integrations 🚀
p
I can try to put down my thoughts into a Github issue + come up with base setup for the plugin (living within base ktlint repository, as you suggested). I’m thinking of just the bare bones, that would allow testing if the setup works for everyone, and would enable us to reuse existing code or writing new one, whatever is preferred later.
I should have some free time in 1-2 weeks, so somewhen around then 🙂
Cool and tnx in advance
w
Hi there, sorry I'm late to the conversation, I was on vacation. My recent refactors to ktlint-gradle make adding compatibility shims for specific ktkint versions far easier. For the last ktlint release, I was able to get a compatible plugin released within 1 business day. Now the bottle neck is spending time reproducing errors people are getting with very specific setups, but in general, ktlint version upgrades are in a much better state. One improvement I could see is perhaps an easy to run github action to run ktlint-gradle's integration tests against a ktlint snapshot. I'd be happy to collaborate on that. We have debated the "merge to a separate repo" issue to death before, always with the conclusion that there is not actually much benefit to that, for the reasons @mateusz.kwiecinski summarized above.
Also, on a nontechnical level, I've been working more closely with the ktlint team @Sha Sha Chu and @Paul Dingemans now.
p
Tnx for responding @wakingrufus. I am not sure whether I agree on:
We have debated the “merge to a separate repo” issue to death before, always with the conclusion that there is not actually much benefit to that, for the reasons @mateusz.kwiecinski summarized above.
I do believe there is a benefit to integrate a ktlint gradle plugin in the project. But that is only feasible if a maintainer / developer is joining me in the project to work on it. It might be a solution is to expand ktlint with a ‘simple’ gradle plugin which is strongly coupled with the project (e.g. it only supports the current version of the KtlintRuleEngine). For other use cases, the existing plugins can be used (provided that they are still maintained).
p
Thats a smart idea! Then adjustments will also be simple and be out of the box assisted by IDE refactorings. Then we’d get the best of both worlds. Developers who would like to have a plugin that supports a lot and many different ktlint versions can use one of the existing plugins. I actually really like this one as a baseline: https://github.com/usefulness/ktlint-gradle-plugin @mateusz.kwiecinski Would you be willing to maintain this inside the upstream repo?
w
Another option is to have a "ktlint" github org and have ktlint and the gradle plugin repos both exist in that org. also, going back: i did not mean to imply that we had a consensus on repo-merging, but that I had not yet seen an good argument made for what benefits it would bring. many people have brought up this idea, but each time, the benefits that are expected would not actually be realized by simply merging the code bases. all the same work needs to be done either way. also, the 2 teams have their own specializations and wouldnt be able to help each other that much... i dont know much about ktlint internals, and the ktlint team (from what i understand) does not have much experience with writing gradle plugins. meanwhile the repo is bigger, the build takes longer, the project is more complex... what problem, specifically are we trying to solve here? lets start there and come up with a solution, and not just assume that merging repos is a silver bullet that will solve all our problems. But as it stands: I think the relationship between the ktlint project and ktlint-gradle project has developed really well over the past few months, and I think things are going much more smoothly now. There is still a lot of catch up to play in ktlint-gradle since it was not really maintained for a year or more, but I am slowly working through it. Maybe the benefits of this are not yet apparent to our users, and that is why there are so many people with these opinions of what needs to change?
p
Statrting a “ktlint” github org would mean that we also have to build a new governance structure which now is being handled by pinterest. I am not favour to take this upon me. A benefit of integrating in a single repository is that ktlint and its internal gradle plugin would be in sync by definition as soon as it is released. External plugins always suffer some lag between release of ktlint and the release of the plugin. By eating our own dogfood (e.g. applying the plugin on ktlint itself) it is also easier to detect problems in an early stage when changing the API. Also it can help in evolving the API as we can try out new features without having to release the API. I don’t know a lot of Gradle as I only use maven in my professional job. But I am willing to learn about it as sidekick of a proficient Gradle developer. Of course, I hope it also works vice versa 😉 Another benefit of merging would be that the developers will be more closely related as they share the same issue tracker but more important share the same release. I expect that a shared goals leads to better collaboration.
w
Ok those are good points. I'll expand a little on each: For testing ktlint snapshots with the plugin we could create a github action that runs the plugin's integration tests against a snapshot. That would be pretty easy and im open to working on that. For more tighter integration in the development cycle: this is harder than it sounds. Gradle plugins resolve external tools via artifact repositories. So running the plugin with a local-built ktlint in the same project that hasn't been published anywhere yet would involve some considerable custom code and/or processes. For streamlining issue tracking, i see the benefit there for sure. We do get issues on the plugin issue trackers that are about ktlint internals fairly regularly
By the way, what is Pinterest's governance model? ktlint-gradle's governance has not been formalized yet. I'd 100% be up for transitioning to a community governance model, but tbh, id very cautious of a corporate oss governance. And @jlleitschuh is the current owner so his opinion on this would be most important
p
I have no clue what the governance model is of pinterest. Maybe @Sha Sha Chu can answer that one. Within the ktlint project there are only just a few things that I can not manage myself. Afaik, both, the owners of ktlint-gradle and kotlinter have indicated that they are open for merging the repositories or handing over the ownership because they don’t have time to be involved themselves.
w
In the end: i have a lot of work to do still in getting ktlint-gradle up to par and addressing the bug reports, so I am inclined to prioritize that work over the work it would take to merge the projects. I think a better time to discuss the merge is when the plugin is in a better place maintenance-wise
p
Yeah that sounds fair to me. If you are willing to work on ktlint-gradle as separate project, your users will likely very much appreciate that. Also, as ktlint maintainer I do appreciate that. Let’s keep in touch. Having said that, this does not mean that no work will be done on integrating another gradle plugin into the ktlint project.
w
yeah, if you want a "reference" implementation or whatever in the project yourself, go for it. maybe it doesnt cover all the use cases our plugin does, but its enough to prove out basic functions, and there is value in that. more power to you. you already have a nice guide for other integrations to point people at the more feature rich implementations: https://pinterest.github.io/ktlint/0.50.0/install/integrations/ and I think that is good.
basically you could just replace https://pinterest.github.io/ktlint/0.50.0/install/integrations/#custom-gradle-integration with a plugin that does the same
s
I'm on vacation until the end of the week but Paul's right that he has near-admin access but a Pinterest employee does need to be involved in the releases atm. I'm not against discussing moving ktlint to be completely community supported (eg to GitHub/ktlint) because if we're being honest it's been pretty much community supported even after it moved under Pinterest
j
> the recommended gradle integration
>
If that is the case I suggest to move it to a parent 
ktlint
organization and remove this pin:
https://github.com/JLLeitschuh/ktlint-gradle/issues/569 I'm more than happy to move to one common organization, but I really don't think the single repository makes sense. Also, I'm a 👎 on git submodules. I've heard some haneous stories around how they complicate dependency management significantly. Instead, I'd propose that every merged commit to 
ktlint
publish a snapshot version, that the main branch of 
ktlint-gradle
depends upon until a release, at which point the release would be pinned to a non-snapshot version number. The biggest problem is, I truly don't know what this will change regarding development/release cadance. Just because these two repositories are in the same organization doesn't immediately mean that the people developing 
ktlint
will know how to fix or add features to 
ktlint-gradle
. The knowledge about how Gradle, and it's API's work won't immediately transfer to those people, just because the repo changed orgs.
That being said, I'm barely contributing to 
ktlint-gradle
anymore, beyond PR reviews. We are looking for more interested contributors, and if this change means more people will likely step up to assist with maintenance, I'm happy to support that transition
I'm also on Vacation till next Monday
To use a coined phrase, I want to make sure we're not committing to work that is just "changing seats on the titanic"
p
Multiple repositories under the same organisation will indeed change nothing. Multiple modules inside a repository at least stress the importance that all modules should work together before you can release. But it will only work when we have enough contributors. So if an interested contributor want to onboard in the ktlint project and is willing to create/maintain a ktlint gradle plugin, then I will welcome this. Otherwise, we will just keep status quo as it is right now and have separate projects.
3 Views
Open in Slack
PreviousNext
Powered by