I'm running into a problem with mockk - on machine...

# mockk

[Jonathan Lennox]

I'm running into a problem with mockk - on machines with small memory, when I use kotest 5.3, my unit tests consume nearly all of memory, and when I look at heap dump of the test program, I see that io.mockk.impl.stub.SpyKStub has a retained size of 1,930,015,280 bytes.

[Jonathan Lennox]

This is particularly a problem on machines with small memory, where the JVM autoconfigures itself to have a small heap - notably the Azure instances that GitHub workflows use only have ~7GB total memory, so the JVM gets less than a 2 GB heap.

[Jonathan Lennox]

As a result my tests take forever because the JVM is constantly GC'ing.

[Emil Kantis]

Is there any particular spying/mockking you're doing? Do you clear your mocks between tests?

[Jonathan Lennox]

The code uses spying as a hack to instantiate abstract objects (there's a fake ScheduledExecutorService which didn't bother implementing all the methods, just the ones the test needs). And no, I don't know about clearing mocks.

[Jonathan Lennox]

It seems likely that's an awful hack and there's a better way to do that?

[Emil Kantis]

Not sure about the spy hack, but for clearing you can simply invoke clearAllMocks() . That might free up a lot of old recorded calls etc

[Jonathan Lennox]

Where in kotest do I invoke that? Just at the end of the test?

[Emil Kantis]

Depends a bit on your lifecycle, and how you re-create the mocks 🙂 You could add something like

beforeEach { mockExecutor = spy<ScheduledExecutorService>() } 
afterEach { clearAllMocks() }

in the beginning of your spec

[Emil Kantis]

that would reset everything between each test

[Jonathan Lennox]

I'll give it a try, thanks!

[Emil Kantis]

np, good luck 🙂

[Jonathan Lennox]

Where should I clear a mock if it's declared as a direct member of a ShouldSpec test? Is there a way to clear it when the whole test is done?

[Emil Kantis]

afterSpec { clearAllMocks() // or clearMocks(mockExecutor, [... other mocks]) }

[Jonathan Lennox]

Cool, thanks

[christophsturm]

be careful with clearAllMocks because it does not work with multithreaded testing. better only clear the mocks that you created

[Jonathan Lennox]

Does the afterSpec solution work if I'm doing IsolationMode.InstancePerLeaf? It says it re-instantiates the Spec each time so that's unclear to me.

[christophsturm]

i think you have to use afterTest then

[christophsturm]

if you use instance per leaf you can just set up the mock where you declare it, and don’t have to use beforeEach

[christophsturm]

and clear it in afterEach. I’m never sure how its called in kotest.

[Emil Kantis]

I believe afterSpec would be invoked after each test in that case, and the spec would be re-created after each test.. I usually use the default isolation mode so I'm not 100% sure though

[Jonathan Lennox]

Is there a reason mocks can't just get garbage collected like every other Java/Kotlin object?

[christophsturm]

I have always just relied on them being garbage collected but my mocks don’t do much

[Emil Kantis]

I think it's just keeping a list with all invocations of the mock for verification purposes. If you never clear it, it can grow quite big I suppose. I'm not sure if there's a global registry of all mocks, would make sense, and in that case there'll always be references to your mocks once created, so they cant be GCd.. I had similar problems when I forgot a static mockk that ended up living in my entire test suite once.

[christophsturm]

there must be a list somewhere to make clearAllMocks work

[Emil Kantis]

exactly 😁

[christophsturm]

anyway if you love fast test suites you can also try my test runner that works great with mockk and is much easier to understand than kotest (at least for me), because it has zero configuration and always uses full test isolation and parallel execution: https://github.com/failgood/failgood

[christophsturm]

but clearAllMocks is a total no no there because any shared state messes up the parallel execution

[Jonathan Lennox]

Looking over the code, it looks like the global uses weak references, but nonetheless doesn't clear values when the strong references go away. Probably on JVM mocks it'd be a good idea if mocks had finalizers/Cleanables that cleaned up the global table, though I don't know how portable that'd be to the other platforms.

[sam]

@christophsturm this isn't a kotest issue, but mockk keeping an internal list. The other solution will work (clearing after each spec). If you use prepareSpec and finalizeSpec it will clear up everything once all instances have completed.

[christophsturm]

Yes sorry I did not think that its a kotest bug. Not knowing what callback is called exactly when in combination with different isolation modes is maybe an issue with kotest’s complexity. I like simple things and for me it always was confusing and a reason for writing my own instead of contributing to kotest. 🙂

[thanksforallthefish]

tbh (on the topic of complexity) I am always wondering what is the use case. all our tests uses the default isolation mode for example, and I am wondering if supporting complex tests that would require messing around with the isolation mode is not incentivizing bad practices (but might be missing or not seeing good reasons for increased complexity). and now back to my lair :)