Is anybody else having big memory usage spikes when running the browserTest task?

# javascript

[Rob Murdock]

Is anybody else having big memory usage spikes when running the browserTest task?

[Rob Murdock]

I’ve been struggling with 137 errors in a limited memory CI container, and I’m realizing now that switching to using the browserTest task (from my custom karma setup) is a big culprit.

It could be specific to my situation, but thought I’d throw out my experience in case others are finding something similar.

Log-wise, the task appears to successfully finish webpacking, and then connecting to a browser before quitting the node process with a 137.

[anton.bannykh]

^ @Ilya Goncharov [JB] this is more or less expected, right? Do you have a rough estimate how much RAM should this task consume?

[Ilya Goncharov [JB]]

I would like to see config or maybe CI project, default limit for node.js processes is 512 mb
I don’t think that our build doing something special unless doing webpack for the whole application and run karma (with declared browsers), and it seems that similar stuff should be done previously for your build, isn’t it? Do you have in your custom karma, webpack stage for your files?
Maybe Chrome Headless (default test browser) consume all memory

[Rob Murdock]

It could be related to the sheer size of the modules getting pulled in here. My previous setup had actually used a vendor library to pre-compile to javascript a bunch of modules, and that may have been helping avoid the spike (or maybe not).

[Rob Murdock]

switching to the custom setup with NO vendor file, I produces this stats read that has… shockingly large modules to my eye (maybe this is normal in test runs?).

The only other wild card is the idea that the gradle build is involving more independent processes, with independent RAM caps that add up to more than what’s available. I’ve toyed with changing all the jvm settings to pull down caps and none of them seem to affect the issue.

[Rob Murdock]

I’m investigating whether or not disabling source maps will help at the moment.

[Rob Murdock]

sadly I can’t easily produce a stats file for the kotlin-build configured webpack that goes to karma AFAIK

[Ilya Goncharov [JB]]

packages_imported is so large, because there is fully js and source maps, which can spend a lot of space

[Rob Murdock]

yeah it looks like it doubles it. Sadly I’m still getting the 137 even with source maps disabled

[Rob Murdock]

I’m investigating switching to a node.js test run instead. Setting up jsdom or some such, to see if it’ll sidestep the problem.

[Rob Murdock]

Oh, there doesn’t appear to be a mocha.config.d setup 😅

[Rob Murdock]

And to compound things, there doesn’t appear to be a way to use webpack + mocha (as a frontend module it references things like css, etc that the node require system won’t know what to do with). Darn, that makes it a nonstarter for me.

[Rob Murdock]

Just testing this again, and it looks like I still get 137s when using the gradle karma support (rather than a custom command).

Is there any way to output the webpack config that’s being loaded into karma? That could be useful for figuring out what the differences are

[Ilya Goncharov [JB]]

Hmm, you can check build/js/packages/module-test/karma.config.js , it contains webpack configuration for karma inside

[Rob Murdock]

Thanks, I’ll take a peek