https://kotlinlang.org logo
Title
j

Jonathan Lennox

05/31/2022, 7:07 PM
I'm running into a problem with mockk - on machines with small memory, when I use kotest 5.3, my unit tests consume nearly all of memory, and when I look at heap dump of the test program, I see that io.mockk.impl.stub.SpyKStub has a retained size of 1,930,015,280 bytes.
This is particularly a problem on machines with small memory, where the JVM autoconfigures itself to have a small heap - notably the Azure instances that GitHub workflows use only have ~7GB total memory, so the JVM gets less than a 2 GB heap.
As a result my tests take forever because the JVM is constantly GC'ing.
e

Emil Kantis

05/31/2022, 7:27 PM
Is there any particular spying/mockking you're doing? Do you clear your mocks between tests?
j

Jonathan Lennox

05/31/2022, 7:28 PM
The code uses spying as a hack to instantiate abstract objects (there's a fake ScheduledExecutorService which didn't bother implementing all the methods, just the ones the test needs). And no, I don't know about clearing mocks.
It seems likely that's an awful hack and there's a better way to do that?
e

Emil Kantis

05/31/2022, 7:41 PM
Not sure about the spy hack, but for clearing you can simply invoke
clearAllMocks()
. That might free up a lot of old recorded calls etc
j

Jonathan Lennox

05/31/2022, 7:41 PM
Where in kotest do I invoke that? Just at the end of the test?
e

Emil Kantis

05/31/2022, 7:42 PM
Depends a bit on your lifecycle, and how you re-create the mocks 🙂 You could add something like
beforeEach { mockExecutor = spy<ScheduledExecutorService>() } 
afterEach { clearAllMocks() }
in the beginning of your spec
that would reset everything between each test
j

Jonathan Lennox

05/31/2022, 7:43 PM
I'll give it a try, thanks!
e

Emil Kantis

05/31/2022, 7:45 PM
np, good luck 🙂
j

Jonathan Lennox

05/31/2022, 7:59 PM
Where should I clear a mock if it's declared as a direct member of a
ShouldSpec
test? Is there a way to clear it when the whole test is done?
e

Emil Kantis

05/31/2022, 8:01 PM
afterSpec { clearAllMocks() // or clearMocks(mockExecutor, [... other mocks]) }
j

Jonathan Lennox

05/31/2022, 8:02 PM
Cool, thanks
c

christophsturm

05/31/2022, 8:03 PM
be careful with clearAllMocks because it does not work with multithreaded testing. better only clear the mocks that you created
👍 1
j

Jonathan Lennox

05/31/2022, 8:08 PM
Does the
afterSpec
solution work if I'm doing
IsolationMode.InstancePerLeaf
? It says it re-instantiates the Spec each time so that's unclear to me.
c

christophsturm

05/31/2022, 8:09 PM
i think you have to use afterTest then
if you use instance per leaf you can just set up the mock where you declare it, and don’t have to use beforeEach
and clear it in afterEach. I’m never sure how its called in kotest.
e

Emil Kantis

05/31/2022, 8:16 PM
I believe afterSpec would be invoked after each test in that case, and the spec would be re-created after each test.. I usually use the default isolation mode so I'm not 100% sure though
j

Jonathan Lennox

05/31/2022, 8:29 PM
Is there a reason mocks can't just get garbage collected like every other Java/Kotlin object?
c

christophsturm

05/31/2022, 8:35 PM
I have always just relied on them being garbage collected but my mocks don’t do much
e

Emil Kantis

05/31/2022, 8:38 PM
I think it's just keeping a list with all invocations of the mock for verification purposes. If you never clear it, it can grow quite big I suppose. I'm not sure if there's a global registry of all mocks, would make sense, and in that case there'll always be references to your mocks once created, so they cant be GCd.. I had similar problems when I forgot a static mockk that ended up living in my entire test suite once.
c

christophsturm

05/31/2022, 8:42 PM
there must be a list somewhere to make clearAllMocks work
e

Emil Kantis

05/31/2022, 8:42 PM
exactly 😁
c

christophsturm

05/31/2022, 8:44 PM
anyway if you love fast test suites you can also try my test runner that works great with mockk and is much easier to understand than kotest (at least for me), because it has zero configuration and always uses full test isolation and parallel execution: https://github.com/failgood/failgood
but clearAllMocks is a total no no there because any shared state messes up the parallel execution
j

Jonathan Lennox

05/31/2022, 8:46 PM
Looking over the code, it looks like the global uses weak references, but nonetheless doesn't clear values when the strong references go away. Probably on JVM mocks it'd be a good idea if mocks had finalizers/Cleanables that cleaned up the global table, though I don't know how portable that'd be to the other platforms.
s

sam

06/01/2022, 2:05 AM
@christophsturm this isn't a kotest issue, but mockk keeping an internal list. The other solution will work (clearing after each spec). If you use prepareSpec and finalizeSpec it will clear up everything once all instances have completed.
c

christophsturm

06/01/2022, 6:55 AM
Yes sorry I did not think that its a kotest bug. Not knowing what callback is called exactly when in combination with different isolation modes is maybe an issue with kotest’s complexity. I like simple things and for me it always was confusing and a reason for writing my own instead of contributing to kotest. 🙂
t

thanksforallthefish

06/02/2022, 9:05 AM
tbh (on the topic of complexity) I am always wondering what is the use case. all our tests uses the default isolation mode for example, and I am wondering if supporting complex tests that would require messing around with the isolation mode is not incentivizing bad practices (but might be missing or not seeing good reasons for increased complexity). and now back to my lair :)