The main problem I have when adding native targets...

# kotlin-native

[altavir]

The main problem I have when adding native targets is the speed of tests. For some reason they take tremendous amount of time. For example, I am recomiling kotlinx-io fork and tests take more than 1 hour (JVM and both JS take about 2 minutes alltogether). Are there ways to fix that?

[Arkadii Ivanov]

In all my projects Native is ~2-5x slower, not that slower as in your case. Maybe enable gradle cache would help, if it's not enabled already....

[Elena Lepilkina]

Tests are compiled without optimizations in order to provide the consistent debug info and better compile time, because main idea is that tests are compiled and debugged very often during development.
It seems that full tests set is run mostly on some CI. There is possibility to add -opt for tests binaries.
Also there can be some problems both in compiler and library itself. We had some questions similar to this from ktor team some time ago. We've managed to resolve them (there were patches in library and K/N compiler).
You mentioned here kotlinx-io , we didn't get any questions connected with this one library from library team.
To answer we need to have a look at exact example. Better if it's formed as an issue.

[altavir]

What I have in mind for our own libraries is to be able to run common tests only on JVM since we rarely have platform-specific logic. And run full set of native tests only during releases. It won't help in case of kotlinx-io, since there is some native-specific logic there.

[Elena Lepilkina]

There is a separate task to run only jvmTests (if I remember right jvmTests ). What is problem to use it?

[altavir]

The problem is that I want to run platform-specific tests if they are defined. Also build relies on allTests , which means that I will have to change some logic in it. It is doable on our side, but there is still a small problem here.

[Arkadii Ivanov]

Please keep in mind that even plain common code can produce different output in different targets. One of the examples:

val original = Float.MAX_VALUE
val bits = original.toRawBits()
val result = Float.fromBits(bits)
assertEquals(original, result)

This test fails in JS only.

[altavir]

Yes, I understand that problem. And JS is no big deal since testing time is only marginally slower than JVM, but Native is a problem, ispecially remembering that there are at least several native targets always.

[Arkadii Ivanov]

But same is applicable to Native as well. You can't be sure unless you test. Specially x86 vs x64 may have different behavior with conversions etc.

[Arkadii Ivanov]

So depending on a particular case it might be too late to fix the problem if you are already preparing a release.

[Elena Lepilkina]

What I have in mind for our own libraries is to be able to run common tests only on JVM since we rarely have platform-specific logic.

The problem is that I want to run platform-specific tests if they are defined.

Sorry, I can't get what you want to configure? Run native tests only when changes are in native source sets?
In general, it isn't expected such a big difference in tests performance (as I got right ~60 times)

[altavir]

It is OK to go through the whole build process during release, but in regular development environment, no so much.

[altavir]

@Elena Lepilkina No, I mean run only nativeMain tests on native. Do not run common tests on native unless it is a special releas build.

[Elena Lepilkina]

Also build relies on allTests , which means that I will have to change some logic in it.

Why do you need build task always during development?

[altavir]

Well, it is a good question. Maybe it would be better to use custom build task. But dev deploy activities like publishing to maven local rely on build.

[Elena Lepilkina]

Do not run common tests on native unless it is a special releas build

Common tests mean that they should be checked everywhere. Even one code in common can have problems in different targets. At least there are three separate compilers and they can have different problems and of course targets specific. You can't be aware of all details of all platforms. And when you think that all code in common part should work fine, it can be false.

[altavir]

I understand the problem. But the build time is still an issue. I do not see win-win solution for that.

[Elena Lepilkina]

Maybe it would be better to use custom build task.

If you want to check tests you can call jvmTests task, it'll rebuild everything you need to run tests.

publishing to maven local rely on build.

I amn't sure about default behaviour, but at least our libraries don't run all tests during publishing.

[Elena Lepilkina]

I do not see win-win solution for that.

Could you then provide exact scenarios when you run build task and can't escape it?

[altavir]

Publish to maven local or teporary repositry. But you are correct, it may be easier to circumvent.

[Elena Lepilkina]

To answer we need to have a look at exact example. Better if it's formed as an issue. (

This is still appriciated, if you can share your code where is such a great difference. It can help and fix some performance problems.

Publish to maven local or teporary repositry

It's strange, it should depends on assemble task not build one by default. Tests aren't run during publishing.
cc @ilya.matveev

[altavir]

Right now it is actually your code: https://github.com/Kotlin/kotlinx-io, I am maintaining a public fork of it.

[altavir]

Maybe you are correct about publishing. I just was really frustrated after a hour of tests.

[Elena Lepilkina]

Did you notice what exact tests are very slow? ktor seems to contain same module, may be it's already fixed. If you can't remember tests, I'll have a look later

[altavir]

The kotlinx-io is configured to run native compillation and tests only in non-idea environment (the thing I will probably adopt for my own projects). So from command line I could only tell the number of test being done and it took 40 minutes after compillation was complete, so I just switched to other tasks.

[ilya.matveev]

It's strange, it should depends on assemble task not build one by default. Tests aren't run during publishing.

Yes, this is the case. Publishing doesn't trigger running tests by default and AFAIK kotlinx-io doesn't change this default in its buildscripts.

So from command line I could only tell the number of test being done

Gradle stores an HTML test report in build/reports/tests. Could you please share it?

What I have in mind for our own libraries is to be able to run common tests only on JVM since we rarely have platform-specific logic. And run full set of native tests only during releases.

You can achieve this by using test filters. You can move native-specific tests into a separate package and then configure a separate test run to execute tests from this package only:

kotlin.macosX64("macos") {
    val short by testRuns.creating {
        filter {
            includeTestsMatching("org.sample.native.*")
        }
    }
}

This snippet adds a separate task macosShortTest which runs only tests from the org.sample.native package. The default macosTest task is still available and runs all tests.

You also can configure filters of the default test task by accessing the task itself

val macosTest by tasks.getting(KotlinNativeTest::class)

or by configuring the corresponding test run:

kotlin.macosX64("macos") {
    val test by testRuns.getting
    with(test) { ... }
}

[altavir]

@ilya.matveev Thank you, the remark about test filters is really useful. The build output right now does not include tests results for native, I've probably overwritten it. I will send it to you the next time I will run full test suite.