The problem is actually not specific to Java and JVM. There are a lot of articles about it. For example this one:
https://docs.oracle.com/javase/tutorial/java/data/autoboxing.html. In short, if you are using some kind of generic objects in your code (for example, abstract numbers), eventually they be represented in memory like references to heap, not by stack-allocated primitives. It has a strong impact on performance. There are currently no good ways to treat the problem in any language (maybe only in Julia).