Another one :smile: <https://github.com/kotest/kot...

# kotest

[wasyl]

Another one 😄 https://github.com/kotest/kotest/commit/a0ea126434401074dc8d1839fb986c23a74780c1 this commit didn’t really change anything because I was running tests with Gradle, and this discovery path is not even hit — Gradle (Junit?) passes all requested tests classes already, Kotest just filters them to keep Spec instances. The 4GB of memory that ClassGraph takes is triggered by detectConfig(). Disabling jar scanning for it is not reasonable, because jars may provide listeners.

So while config discovery is pretty useful, would you consider some other mechanism for that? I know that scanning can be disabled, but then modules wouldn’t be able to provide useful listeners (I think I even requested discovery at one point, to avoid having to declare a merged project config in each module). One thing that comes to mind is allowing everything that can be AutoScanned now to be provided via ServiceLoaders. Locally I just hardcoded our listeners and got that peak memory down from 4GB to 1GB.

Bottom line: ClassGraph has significant memory impact, at least with larger projects. It’d be nice to have a way to completely bypass it. Btw in addition to memory usage, it also spawns several threads (in my case 18!) that it never releases.

[wasyl]

Out of curiosity, when are tests discovered via classgraph? When running Kotest directly on jars from command line somehow, or from junit maybe?

[sam]

When classes aren't specified

[sam]

So probably never?

[sam]

As gradle specifies them and so does the intellij plugin

[wasyl]

Makes sense, I wonder if someone managed to run Kotest without specifying test classes 😄

[sam]

Lol i think the package run in intellij does that

[wasyl]

I’ll be honest I almost never use that 😄 And I admit I’m mostly looking for optimisations in my workflow, which is running entire module tests via Gradle and maybe running single test from IDE later on

[wasyl]

I can draft service loader support later on but I’m not very familiar with Kotest tests suite so that might take a while

[sam]

If we limit scanning to classes only it should work

[sam]

Although the ones shipped with kotest itself are in jars

[sam]

Service loader is JVM only too so must be opt in

[wasyl]

Right, I forgot 😕

[sam]

You don't need autoscan. You can already disable it

[wasyl]

If we limit scanning to classes only it should work

We also have listeners in other modules, and the case for scanning was to allow modules to contribute listeners just by adding a dependency on them

[sam]

Right

[wasyl]

And that’s why we can disable autoscan, but will have to register those contributed listeners in each module specifically. Not terrible but still some overhead

[sam]

Service loader as an alternative is absolutely fine

[wasyl]

I’ll see if I can figure out something multiplatform-friendly, or maybe Kotlin folks are working on a Kotlin-specific alternative

[sam]

The include packages setting might be sufficient

[wasyl]

I think I tried that and didn’t see much improvement, but I’m not sure now. I’ll check again, you mean to add option to specify acceptPackages argument?

[wasyl]

Actually, another option would be to specify listeners/extensions classes names that may be on the classpath and use them if they are. That’s pretty inflexible but not terrible either (I’m just looking for ways to sidestep ClassGraph completely)

[sam]

If we use service loader then the JDK still has to open up the jars to look for the descriptor files

[sam]

Assuming that classgraph is efficient, using the acceptPackages argument would skip a ton of class scanning

[sam]

I can add that option in like 5 minutes if you want to try a build

[wasyl]

I already figured out how to prototype things quickly locally 🙂 Got a local build and publish whatever I change

[sam]

Ok, and it made no diff ?

[wasyl]

I don’t recall, I’ll check it later today, gotta afk for a while

[sam]

I'll push a change to close down the classgraph threads at least

[wasyl]

Also I still want to hunt down that 1GB of memory early on, I have a feeling is a bit more than needed, but haven’t found it yet. But yeah I’ll try some variants and let you know the results 🙂

[sam]

and when you're back later - classgraph is used in two places, so make sure you put acceptPackages in both

[wasyl]

But one is not hit when running from Gradle, right? Anyway I specified packages in both places, with no difference in memory usage, still 4GB

[wasyl]

I’ll push a change to close down the classgraph threads at least

That’s my bad, the threads are closed right after scanning, the missing scanResult.close() in detektConfig.kt is just to clean up some files. I looked at the profiler wrong

[sam]

ok I did update the code to call close in detect anyway