We're close to the runtime limit for box tests on ...

# python-contributors

[Piotr Krzemiński]

We're close to the runtime limit for box tests on GitHub (6 hours, we're at ~5). Worth considering parallelizing them - I'm thinking about it already. AFAIK, GitHub Actions allow parallel execution on multiple machines, the main difficulty will be telling Gradle/JUnit to run a specific subset (map), and then join the results (reduce).

[SerVB]

That's strange why they are made bigger in 3 times after my last PR...

Will be tricky to split them...

I had some ideas regarding the performance of tests themselves (1, 2). I can check the next week or you can do it if you want:

[SerVB]

Also it's worth checking how much threads/processes Gradle uses on GH workers to run our tests. AFAIR there are two vCPUs, so if it uses only 1, it's not effective

[Piotr Krzemiński]

I think the more IR stuff we translate, the more time it takes for the compiler to produce the output. But true, this increase is suspicious. Whoever has an opportunity to work on it, let's check it out :)

[SerVB]

How much time do box tests take on your machine, by the way? After we've updated to Kotlin 1.5.21 and I've switched to 24-threaded CPU, they only take 8-10 minutes on mine 😅

[Piotr Krzemiński]

Regarding StringBuilder, Kotlin's string interpolation uses StringBuilder under the hood

[SerVB]

Kotlin's string interpolation uses StringBuilder under the hood

I doubt it works outside of a single function... Inside a single one – definitely of course

[Piotr Krzemiński]

I haven't measured, but I think around 1-2 hours. For sure not 4 :D. Will check it next time

[SerVB]

Wow, that's a lot. Do you use ramdisks?

[Piotr Krzemiński]

I once tried to use them, but then the tests didn't work at all. I didn't have time to investigate back then.

[Piotr Krzemiński]

Just one more thought before I have to leave: worth profiling before deciding to optimize something :)

[SerVB]

We can also try enabling ramdisks for GH Actions

[Piotr Krzemiński]

Good idea, also thought about it

[Piotr Krzemiński]

Can we hook your machine to GitHub Actions? 🤣

[SerVB]

We can consider it too 😉

The problem is I don't run Linux usually, only for Kotlin

[SerVB]

In the worst case, I think I could boot up to Linux and run GH agent for build to pass 😅

[Piotr Krzemiński]

No worries, I meant it as a joke only :D we'll figure something out so that I won't have to ask you to run your machine so that my PR gets validated ;)

[SerVB]

I hope 😉

[CLOVIS]

I don't have any experience with GH actions but I'm fairly knowledgeable of GitLab CI, I have no idea if that's useful for you, but if it is please ping me, I'd love to help a bit on this project

[Piotr Krzemiński]

Are you willing to learn GitHub Actions? Any help to optimize the CI would be great

[Piotr Krzemiński]

Some solutions may be actually unrelated to the CI platform, but e. g. Gradle or using ramdisks, like mentioned above. Feel free to fork and tinker around :)

[bashor]

I’d suggest profile tests first to understand which parts are heavier in your case

[bashor]

There is a high probability that running generated code is a slower part and you can apply some tricks to speed up it.

[bashor]

usually it’s enough to profile only part of test — 10-50 at once

[SerVB]

@bashor, it seems profiling doesn't work for me for test tasks. I've mentioned it here: https://kotlinlang.slack.com/archives/C0289CS37AS/p1627654423005300

Does it work for you?

I've also asked IntelliJ folks about it and already received an answer, will check on Friday

[bashor]

Yes, it worked for me, I think even from gradle in last few major versions.

[bashor]

As workaround you can start from `println`s with timings for different stages

[SerVB]

JFTR, I was asked to file a ticket by IntelliJ folks about unavailable profiler: https://youtrack.jetbrains.com/issue/IDEA-275500

[bashor]

🤷‍♂️ it works for me