Just wondering did anyone try running Gradle in memory Like kotlinlang #gradle

Just wondering, did anyone try running Gradle in memory? Like instead of having buildDir on a drive (HDD or SSD), having it in the memory. With 64+ GB of memory you should easily be able to donate 10 GB to the build, right?

Emil Kantis

05/06/2023, 7:10 AM

https://stackoverflow.com/questions/29116000/gradle-make-use-of-ramdisk

mbonnin

05/09/2023, 12:51 PM

I would expect the OS to cache the hard drive much better than what I can do manually

Rok Koncina

05/09/2023, 4:29 PM

It's not about caching, it's about avoiding the hard drive altogether. Ram is supposed to be 30x faster than SSD. Emil's idea is a good one, I have to try it.

mbonnin

05/09/2023, 4:30 PM

Well, if you cache your hard drive, you're not accessing it anymore

Rok Koncina

05/09/2023, 4:32 PM

I wish Grade would support that directly, then you wouldn't lose speed on the emulation... But I guess sometimes one needs to access intermediate files...

Oh, I see what you mean. Can you do that? Can you do it for a folder or does it have to be the whole drive? I don't have enough memory to cache it all

mbonnin

05/09/2023, 4:35 PM

Linux caches disks reads/writes by default: https://tldp.org/LDP/sag/html/buffer-cache.html

MacOS too. The OS knows better what is accessed and how often it is accessed

You don't want your window manager to OOM for an example by giving too much memory for Gradle build dir

Rok Koncina

05/09/2023, 4:46 PM

Not really. Cache everything except maybe the final output file. Any logic on top of that is a waste of resources. Can I tell the os to behave like that?

mbonnin

05/09/2023, 4:48 PM

You want to persist the intermediate results so that subsequent invocations of Gradle can reuse them

Rok Koncina

05/09/2023, 4:49 PM

OOM is not a problem. Limit 10GB for Gradle, more than enough for the build and dependencies.

mbonnin

05/09/2023, 4:49 PM

The persistence deosn't have to be synchronous though. Your OS will put everything in cache and persist it if memory is scarce/needed for something else

If you really want this I guess you can mount a RAM only file system at

$project/build

but my hunch is that it's a lot of trouble for very little (if any) benefit

Emil Kantis

05/09/2023, 4:59 PM

Tried ramdisk in a multimodule setup.. but I'm just getting loads of issues where gradle incorrectly starts detecting implicit dependencies 🤷

4 Views

#gradle Join Slack