I'm trying to compile and test the kotlin compiler...

# compiler

[Claude Brisson]

I'm trying to compile and test the kotlin compiler, but I alsways get plenty of tests failing after a while with StackOverflowErrors, I'm guessing it is a memory exhaustion problem but I don't know how to overcome it. I tried running the tests in sequential mode with things like:

org.gradle.workers.max=1
kotlin.test.junit5.maxParallelForks=1
kotlin.parallel.tasks.in.project=false
org.gradle.parallel=false

in my gradle.properfies file, but it just made the tests much slower and the same errors appeared.
The command I am running is: ./gradlew -Porg.gradle.java.installations.auto-detect=false compilerTest. Maybe it's too wide a task, but how to refine it? Especially, how would I specifically test tweaks to the lexer and the parser? Thanks for your help.

[dmitriy.novozhilov]

1. Can you provide stack trace of those exceptions?
2. The easiest way to run specific tests is just run them from IDEA. Tests for lexer and parser lays in compiler/tests-gen/org/jetbrains/kotlin/lexer/kotlin/KotlinLexerTestGenerated.java and compiler/tests-gen/org/jetbrains/kotlin/parsing/ParsingTestGenerated.java. If you are really want to run them from terminal then you can ran test task from specific module with corresponding test filter:

./gradlew :compiler:test --tests "org.jetbrains.kotlin.lexer.kotlin.KotlinLexerTestGenerated*" --tests "org.jetbrains.kotlin.parsing.ParsingTestGenerated*"

[Claude Brisson]

All the 299 failing tests except the first happen in org.jetbrains.kotlin.cli.CliTestGenerated$Js.

> Task :compiler:test
*** java.lang.instrument ASSERTION FAILED ***: "!errorOutstanding" with message transform method call failed at JPLISAgent.c line: 844

org.jetbrains.kotlin.integration.CompilerSmokeTest > testBuildFile FAILED
    java.lang.IllegalStateException at KotlinTestUtils.java:445
        Caused by: java.lang.StackOverflowError at StringCoding.java:304

org.jetbrains.kotlin.cli.CliTestGenerated$Js > testNonExistingSourcePath FAILED
    java.lang.StackOverflowError at StringCoding.java:304

org.jetbrains.kotlin.cli.CliTestGenerated$Js > testLibraryDirNotFound FAILED
    java.lang.StackOverflowError at StringCoding.java:304

org.jetbrains.kotlin.cli.CliTestGenerated$Js > testSourceMapRootMissing FAILED
    java.lang.StackOverflowError at System.java:716

org.jetbrains.kotlin.cli.CliTestGenerated$Js > testSourceMapRootAuto FAILED
    java.lang.StackOverflowError at System.java:716

org.jetbrains.kotlin.cli.CliTestGenerated$Js > testKotlinPackage FAILED
    java.lang.StackOverflowError at System.java:716

org.jetbrains.kotlin.cli.CliTestGenerated$Js > testJsExtraHelp FAILED
    java.lang.StackOverflowError at StringCoding.java:304

org.jetbrains.kotlin.cli.CliTestGenerated$Js > testSimple2js FAILED
    java.lang.StackOverflowError at StringCoding.java:304

org.jetbrains.kotlin.cli.CliTestGenerated$Js > testKotlinHomeWithoutStdlib FAILED
    java.lang.StackOverflowError at StringCoding.java:304

org.jetbrains.kotlin.cli.CliTestGenerated$Js > testModulesWithSameNames FAILED
    java.lang.StackOverflowError at StringCoding.java:304

org.jetbrains.kotlin.cli.CliTestGenerated$Js > testEmptySources FAILED
    java.lang.StackOverflowError at StringCoding.java:304 > Task :compiler:test
*** java.lang.instrument ASSERTION FAILED ***: "!errorOutstanding" with message transform method call failed at JPLISAgent.c line: 844

org.jetbrains.kotlin.integration.CompilerSmokeTest > testBuildFile FAILED
    java.lang.IllegalStateException at KotlinTestUtils.java:445
        Caused by: java.lang.StackOverflowError at StringCoding.java:304

org.jetbrains.kotlin.cli.CliTestGenerated$Js > testNonExistingSourcePath FAILED
    java.lang.StackOverflowError at StringCoding.java:304

org.jetbrains.kotlin.cli.CliTestGenerated$Js > testLibraryDirNotFound FAILED
    java.lang.StackOverflowError at StringCoding.java:304

org.jetbrains.kotlin.cli.CliTestGenerated$Js > testSourceMapRootMissing FAILED
    java.lang.StackOverflowError at System.java:716

org.jetbrains.kotlin.cli.CliTestGenerated$Js > testSourceMapRootAuto FAILED
    java.lang.StackOverflowError at System.java:716

org.jetbrains.kotlin.cli.CliTestGenerated$Js > testKotlinPackage FAILED
    java.lang.StackOverflowError at System.java:716

org.jetbrains.kotlin.cli.CliTestGenerated$Js > testJsExtraHelp FAILED
    java.lang.StackOverflowError at StringCoding.java:304

org.jetbrains.kotlin.cli.CliTestGenerated$Js > testSimple2js FAILED
    java.lang.StackOverflowError at StringCoding.java:304

org.jetbrains.kotlin.cli.CliTestGenerated$Js > testKotlinHomeWithoutStdlib FAILED
    java.lang.StackOverflowError at StringCoding.java:304

org.jetbrains.kotlin.cli.CliTestGenerated$Js > testModulesWithSameNames FAILED
    java.lang.StackOverflowError at StringCoding.java:304

org.jetbrains.kotlin.cli.CliTestGenerated$Js > testEmptySources FAILED
    java.lang.StackOverflowError at StringCoding.java:304

[Claude Brisson]

I don't have a more detailed stacktrace, even with the --stacktrace gradle option.

[dmitriy.novozhilov]

Usually gradle generates html report which contains detailed stacktraces for exceptions in each test. Link to this report should be somewhere at the end of gradle output

[dmitriy.novozhilov]

Also note that compierTest task includes more than 100k tests, including codegen tests for five (or more) combinations of frontends and backends, so I highly recommend you to run only those tests suites which you are really interested in

[Claude Brisson]

Here's the final report. It looks like circular calls to get the JDK home

index.html

[Claude Brisson]

Oops, sorry. You'd need the dependent html files... I prepare an archive

[Claude Brisson]

Here you are.

test.tgz

[Claude Brisson]

so I highly recommend you to run only those tests suites which you are really interested in

Precisely. But I'm not really aware of the pertinent test suites I shall use to test parsing changes, apart from the lexer and parser ones you mentioned.

[dmitriy.novozhilov]

You also can run DiagnosticTestGenerated and BlackBoxCodegenTestGenerated. They cover most of possible cases

[Claude Brisson]

Thanks.

[Claude Brisson]

You can see the result of my investigation in the #kontributors channel...