YML format - We already have `.gradle`, `.kts`, `....

# amper

[handstandsam]

YML format - We already have .gradle, .kts, .toml. Amper is solving real problems, but do we need another format? Why can't this be a subset of Kotlin or a DSL so we can leverage all existing tooling? The Kotlin PSI tooling can parse this amper.kts file that only allows a specific DSL. This would also allow us to leverage existing tooling, etc.

There is a lot of advanced usage and custom convention plugins that won't be able to be described in this format. I'm not suggesting that amper.kts would do this either, but could at least use Kotlin syntax to avoid another language needed to be learned.

[David Herman]

Personally, I think it makes sense that the file format they choose should be a data format, not a code format (so not .gradle, .kts). I've compared yaml and toml before in my own stuff and I found yaml is way better for nested data files.

[mbonnin]

There have been talks about .kd : https://kotlinlang.slack.com/archives/C062WG3A7T8/p1699895404568699?thread_ts=1699885069.375749&cid=C062WG3A7T8
I'd love to see something like this happening.

Not only does this mean I can reuse my "how to write litterals" knowledge but also could leverage the API lifecyle annotations like @OptIn and @Deprecated that I haven't seen in any other data language

[David Herman]

I found my format comparison doc if you want to see yaml vs toml: https://docs.google.com/document/d/1kT_AF54lf_ttNBVubauOe0AUvagWYRN4ZeOJVHIBL7k/#heading=h.5etrlgkl1jcb

I ultimately went with yaml

[handstandsam]

As a Kotlin/Gradle developer I'd love to avoid using any yml or toml. But that's based on what I already know.

[David Herman]

Do you use libs.versions.toml?

[handstandsam]

I do use toml but I was basically forced to. I wouldn't have chosen that, but I'm sure there are many reasons.

[David Herman]

Ah gotcha. I actually think toml was a good choice for that file because it's a data format useful for something that's relatively flat

[David Herman]

That said I'm really curious about kd. If there's a data format that looks like Kotlin I think that'd be neat.

[amakeev]

@handstandsam Could you elaborate, what existing Kotlin tooling would you like to leverage, how and for what purpose?

I completely understand the argument about familiarity and the additional language to learn

[handstandsam]

With build.gradle.kts you can have some beautiful DSLs to orchestrate many plugins behind the scenes. One open source plugin is the https://slackhq.github.io/slack-gradle-plugin/ that allows you to do that kind of stuff. Square and Dropbox both have somewhat similar convention plugins.

You could have a really nice block under the kotlin {} extension (even though it wouldn't be in the kotlin plugin itself, that could allow you do configure all these things.

Your editor could have special logic for this part of the file to do additional validations, etc, but make this configuration as simple as possible.

[handstandsam]

You achieve the same sort of simple, easy to read and less plugins to find while keeping things "kotlin".

[handstandsam]

Also, I'm not requiring you'd use the existing kotlin {} block, but you already own it, and they are extensible "gradle extensions" and just a DSL definition (meaning they can be used by any plugin)

[handstandsam]

One possible issue is versioning of the plugins you reference, but that is more advanced and could be configured in the. libs toml or somewhere else

[amakeev]

I see, thanks. That totally makes sense, but I’m still not sure what you meant by leveraging the tooling. Did you mean that using Kotlin DSL would allow reusing existing IDE features as opposed to writing the again for another configuration language? Ir did you mean that there is other tools exists that could work with Kotlin DSL (linters, in-shop checkers etc) and they could continue working?

[handstandsam]

Both. And no language switching for Korlin devs.

Korlin DSLs are concise and powerful, and much better than YAML in my opinion

[amakeev]

Got it, thanks for the feedback! We’re going to take it into account for the final language choice

[David Herman]

One thing this thread had me thinking is: what data formats should be considered part of a healthy programmer's diet? I don't think any programmer would bat an eye at being expected to know html/xml(*), md, and json. Perhaps we can throw .properties in there for this crowd since it's so simple and a foundational part of the Java ecosystem.

(*) I've seen devs complain a ton about using xml in the past, but it was always about its verbosity, not about their lack of familiarity with it.

Personally, I thought yaml and toml were also on that list. Their usage is relatively widespread and they are generally quite useful. I've seen them both around in other projects for years now and thought they were kind of ubiquitous at this point.

Obviously from this thread, it seems like my assumption was wrong. It could be a really interesting opinion for the Amper team to take a pulse of. Do developers think of yaml and toml as standard formats or not?

[mbonnin]

My hunch is there are 2 population bubbles here:
1. people who know Kotlin/Gradle enough that they don't really need amper/yaml and amper/yaml might even be counter productive as it fragments the ecosystem.
2. people who know little about Kotlin/Gradle and coming from other places where yaml is a lot more widespread. amper/yaml is great for them.

[mbonnin]

The problem is that the learning curve has a huge step. If you need to add imperative logic to your build (which is bound to happen in a multi-module/convention plugins world) then you have to assimilate Gradle + Kotlin DSL all at once.

[mbonnin]

A language based on Kotlin DSL would be hopefully simple enough that it's easy to get started and still make the upgrade to a turing complete build logic "easier"

[amakeev]

@David Herman Thanks, that’s an interesting perspective. It indeed seems that YAML/TOML are much farther than XML, json for JVM developers. But the at the same time much closer to devs in say, Rust or Flutter.
I wonder what Ktor developers think about the configuration format

[Javier]

Not related to Amper itself but about the bad decision, in my opinion, of picking a configuration language.

In general working with GitHub Actions and Yaml is in the best case duplicating code if you want to simulate an if else and, overall, deferring everything to Gradle to not throw me through the window.

I have tons of repositories, and each repository has its main branch protected with specific jobs, as I use a Matrix to build the projects with the three OS, those jobs are called: build / macos-latest, build / ubuntu-latest, and build / windows-latest.

What is happening now? I just want to change from macos-latest to macos-14 to use the new M1 runner, and as I am using reusable workflows, I can edit only a single Yaml, and all repositories will start to use it... but now the job name is no longer called macos-latest. I need to go to dozens of repositories to change manually the job that protects the branch, if not, I cannot merge any pull request.

So I was thinking of a way to avoid this in the future, and that's it, instead of using a matrix with the exact names (suffix -latest or -14) I can go with the OS name and, with "a simple condition", add the suffix, -latest if it is not macos, -14 for macos. If I want to test a specific version in the future, I would not need to change the jobs used to protect the branch in all repositories. As this condition is only appearing in the reusable workflow, it is a one-time change.

But here is when it appears again, my old friend Yaml, losing my time in a solution that would take only 1 second with any real language, even the most garbage language that is not a "configuration language". It can be transformed into hours or end up delegating to Gradle tasks.

Well, it should be a simple if else... Well, Yaml is here, and yes, I have had to duplicate multiple jobs in the past with if: condition and another after it with if: !condition, but at least I get something working... right? I cannot do that here!

I cannot do a simple if else in the runs-on, as the language does not allow that, and GitHub Actions authors didn't provide any workaround inside the ${{ }} template. The fact that you need to write conditions and expressions inside ${{ }} is already a signal that choosing Yaml was not a very good idea, as you were already patching the limitations of this language. GitHub Actions expressions are pretty well documented, this documentation could be totally dropped with a real language.

I haven't fixed it yet, but, a missing if else expression is going to provoke I need to add a previous job or whatever to build the job name somehow as I cannot delegate fixing, once again, limitations of Yaml with Gradle. Other solutions? Going to all repositories to change the name and praise I do not have to change it again. Or reading the GitHub API to do it with requests... A lot of things, but none is 1 second solution.

None is as simple as an if else.

I have spent/wasted thousands of hours in tooling, not only with Gradle, building tooling libraries, and plugins. In open source, and internally. I would not spend any second of my life building any tooling/wrapper library to fix the limitations of a Yaml build tool knowing beforehand it is a problem: https://github.com/typesafegithub/github-workflows-kt

Doing tooling is probably being masochistic, to be honest. But Yaml... Oh my, I'm not that masochistic. Yaml would win me, easily, something Gradle could not do after thousands of hours.

This is ragging a lot, I know. But not directly proportional to the frustrating time I have lost over the years due to the limitations of Yaml. Every second lost with Yaml remembers me to Interstellar planet.

Each tool has its utilities, I understand that... I can use a lot of different tools to move from A to B, depending on the distance it can be debatable if it is better a bicycle, a scooter, a car, or even walking... But sorry, Yaml is a stationary bicycle, there is no debate for me, I cannot move from A to B with it, and I will not build wheels for it. They already exist... Hi Kotlin!

Sorry for the long message and have a good weekend!

[David Herman]

@Javier keep in mind Rust has done this already, and last I checked, Cargo was one of the few approaches to dependency management that is actually adored. If you've never read about cargo, take a skim through their docs and see what you think. (Cargo uses toml)

I don't think it's fair to compare this to GitHub yaml because GitHub doesn't provide a code escape. I agree with you if Amper starts trying to add weird data declaration constructs to work around dynamic features that are expressed more naturally in code, they're probably doing it wrong.

But if they use a data file for very common, structured data declarations, like libs.versions.toml does, I think that could be more amazing than most people realize. (And based on the demo code I've seen for Amper, it's been solid so far).

Do you remember the nightmare it was declaring shared versions across Gradle build scripts before libs.versions.toml existed? Every approach had some disadvantage, and soon everyone had their own unique approach of where to put things, often with projects having multiple different approaches at the same time, which was even worse.

[Javier]

Rust can use Rust files when Cargo is not enough. And this is not difficult to happen, the first project I created in Rust was a toy app with gRPC and protocol buffers and Cargo was not enough, you can check the official grpc tutorials for this.

We already have something similar, Version Catalogs and Gradle to the above.

Create a build tool from scratch using Kotlin (Amper), or create a Kotlin Gradle wrapper (Amper), or fix KGP DSL, or fix Gradle. But please, do not create a Yaml wrapper.

About the dependency management nightmare, it is unrelated to the language. It was a missing feature that was added with version catalogs in toml and/or kts. As you consume it on Gradle, if you want to do an if else with a dependency, o modify any part of it based on some condition, you can.

For example, I build compiler plugins and I am replacing the Kotlin version from the version catalog in Kotlin. So I can test dev versions from the bootstrap repository.

if (kotlinVersion != null) {
    version("kotlin") {
        strictly(kotlinVersion)
    }
}

The above solution could be a real nightmare with a CI and a build tool belonging the Yaml world.

Amper will end up in another layer which will only have sense in very, very, simple projects due Yaml. In the rest of projects you will have a mix of Amper and Gradle files (even the image of Amper in the blog post shows a Gradle file because it cannot apply plugins).

Clean architecture layering vibes.