Hi, I'm trying to speed up my tests by using parallelism Kotest feature. I distribute databases instances by using LinkedBlockingQueue and

beforeSpec { queue.take() }

,

afterSpec { queue.add(db) }

Unfortunately I don't see a big improvement of performance. Normally my tests take around 30s. With

parallelism=16

only a few seconds better. My CPU has 32 threads. Database server in the container also has access to all 32 threads. I tried different combinations of

parallelism

setting and number of available databases but I didn't notice significant difference. IntelliJ shows that all my 200 tests take 10s but this time doesn't include time between tests and time taken by building project. What's wrong with my approach?

Parallelism will mostly just mean multiple specs in parallel

My 200 tests cases are of course divided into multiple specs.

Try increasing concurrentSpecs too

Setting

concurrentSpecs=16

didn't make difference but

concurrentSpecs=1

works 20s slower

Does your queue of db instances hold enough entries to support that number of parallel tests?

It does. I tried different configurations. With the same number of dbs as

parallelism

, less dbs than parralelism, more dbs than parallelism. I don't see difference. For example there is no difference between

parallelism=16

with 10 dbs and

parallelism=8

with 8 dbs.

IntelliJ might not be the best way to benchmark this. It injects some build config so that it can detect and display the test results. Do you see the same results if you run the tests via the command line?

There is no difference in total time when I run it via command line

I use code like this to measure the level of concurrency that my tests achieve:

private val operatingSystemMXBean =
    ManagementFactory.getOperatingSystemMXBean() as com.sun.management.OperatingSystemMXBean
private val runtimeMXBean = ManagementFactory.getRuntimeMXBean()

    val uptime = runtimeMXBean.uptime
    val cpuTime = operatingSystemMXBean.processCpuTime / 1000000
    val percentage = cpuTime * 100 / uptime

if you run that at the end of your suite you will see how much load you generated. optimal percentage would be 100*cpus, but thats more a theoretical value.

@christophsturm do you run it after each spec or after entire project?

do you run it after each spec or after entire project?

this displays a summary of cpu load over the whole test suite so I run it after the entire project.

this displays a summary of cpu load over the whole test suite so I run it after the entire project.

I run it after entire project and it shows around 1800 with

parallelism=16

. With

parallelism=1

value is similar.

interesting. your graphs for parallism=1 and paralellism=16 look very similar too, so maybe cpu load in your case is caused by something else?

There is also SQL Server container used by tests. Maybe IntelliJ do this. I will check it.