It would be great if we could set up open source l...

# compiler

[Marc Knaup]

It would be great if we could set up open source libraries and projects for automated unit testing against Kotlin compiler changes.
That way any changes are automatically checked against a wide range of code 🙂
It should definitely check incremental builds too as they always cause issues.

[raulraja]

I was wondering the same here https://kotlinlang.slack.com/archives/C7L3JB43G/p1578951482008800

[dmitriy.novozhilov]

We want to release stable a stable compiler API, which will be backward compatible between compiler versions (at least between minor ones) with proper deprecation and other things, but it's very far from now (and I'm not sure that we will manage building such API)

[dmitriy.novozhilov]

But for know such tool could be really helpful for compiler plugins developers. And is someone creates one, it will help us to understand which parts of compiler people are interested in

[Marc Knaup]

@dmitriy.novozhilov my request is not about compiler API or plugins, but compiler implementation.
There’s rarely a Kotlin release where I don’t run into new compiler issues.
These issues could be detected earlier if compiler changes are unit tested against open source libraries and projects.
That would detect more issues before they end up in a Kotlin release.

[dmitriy.novozhilov]

Oh, I get it. Actually we test compiler builds against different third party projects, internal JB ones and some open-source (there are something about 20 of them) and we check that all of them can be successfully compiled with new compiler build
If you have any big open-source projects with rich codebase please share them, so we can add them to our projects set

[dmitriy.novozhilov]

cc @Victor Petukhov

[Marc Knaup]

It would be nice if we could simply
• prepare a project for “Kotlin compiler testing” (e.g. by making the Kotlin version a variable in build scripts)
• creating a special branch for it
• register the repo & branch somewhere with JetBrains
• have its unit tests automatically run against new compiler versions.

Or maybe the other way round.
Have some kind of webhook that triggers CI with a new Kotlin prerelease.
But there needs to be a way to test incremental builds as well as IDE Gradle syncs. Both are quite fragile.

[dmitriy.novozhilov]

It will be very hard to understand why some test fails, because each project has it's own specific and we don't have enough human resources to deep into each user project. And also it may be quite expensive to JB run all unit tests from different projects on CI
But I see a compromise: we can add some projects to our builds to test only compilation, and authors of project may add configuration on their CI which uses compiler from last build from release branch (for example) and runs all tests in project. And if this configuration fails author can report it to us, so we can quickly fix the problem

[Marc Knaup]

Yeah I think the latter is a good compromise. Kotlin could provide an easy way to compile against the latest compiler build (and libs).
Running that with each CI of open source libraries themselves would surface issues pretty quickly.

[dmitriy.novozhilov]

I will prepare some document with instructions how to depend on nightly build on next week

[Marc Knaup]

Sounds awesome, thank you 🙏

[dmitriy.novozhilov]

@Marc Knaup we discussed this internally, and decided to try next scheme:
We advance bootstrap kotlin compiler (build of compiler which is used for compiling kotlin project) each two weeks (using fresh master build), and this boostsrap build is published to public maven repo on Space packages. So users who want to test their projects with freshest compiler version may use those builds.
This option is much simpler than publishing snapshot builds for each push to master/each day, so for now we can try this one as a start decision. In future we can reconsider this decision after you and other users collect some feedback.
cc @raulraja

[dmitriy.novozhilov]

You can simply add this repository with

maven {
    url = uri("<https://maven.pkg.jetbrains.space/kotlin/p/kotlin/bootstrap>")
}

in your build.gradle.kts and pluginManagement section of settings.gradle.kts
Last published version is 1.5.20-dev-814 (published today)

You can track new builds in public kotlin TC (see builds with boostrap tag): https://teamcity.jetbrains.com/buildConfiguration/Kotlin_KotlinPublic_BuildNumber?mode=builds&tag=bootstrap (1.5.20-dev-814 is not here yet, but I will add it soon)
If there will be some other people how interested in such tests then we can create new slack channel where I will notify about new bootstrap builds (it's OK for me because it's me a man who responsible for bootstrap updates)

[Marc Knaup]

Thank you very much @dmitriy.novozhilov.
I’ll add additional unit testing using the latest compilers to all my libraries once I update them 👍

[Marc Knaup]

@dmitriy.novozhilov is there any way to automatically and easily test incremental compilation?

[dmitriy.novozhilov]

No, we even don't have our own complex integration tests for IC, only simple ones
It's quite difficult to setup such tests, which need to build project, change something in code (not too much, but it should affect recompilation of other modules and keep code semantics), recompile it with IC and than compare that all affected files recompiled
All our tests for IC do it on simple projects and devs manually track which code was recompiled and compare it with expectations

You can check tests here: https://github.com/JetBrains/kotlin/tree/master/compiler/incremental-compilation-impl