https://kotlinlang.org logo
w

wasyl

02/05/2021, 8:49 AM
Originally asked in #coroutines but turns out coroutines are irrelevant 🙂 I have a question about the generated code: why does the compiler generate an additional variable when a variable spanning two lambdas is used? For example this code:
Copy code
fun test() {
    var foo: Foo? = Foo()
    println(foo)
    runBlock { foo = null }
}

private fun runBlock(block: () -> Unit) {
    block()
}

class Foo
So I have
var foo
, which I use in
println()
and then null it out in another lambda. The compiler generates the following (decompiled) code:
Copy code
final ObjectRef foo = new ObjectRef();
foo.element = new Foo();
Foo var2 = (Foo)foo.element;
boolean var3 = false;
System.out.println(var2);
Note the
Foo var2 = (Foo)foo.element;
which is used in
println
. It is somewhat unexpected, e.g. if
Foo
is heavy I may want it to be GC-ed after
foo = null
is executed. It won’t be, because
var2
is keeping it. Why is necessary that
var2
exists, and would it make sense to clear it after it’s used?
i

Ilmir Usmanov [JB]

02/05/2021, 9:07 AM
The compiler, generally, does not know, whether the lambda is stored in a variable or executed right away. So, it has to be conservative - assume the worst-case scenario. It sees, that
foo
is captured in a lambda, which might be stored is variable, so, it assumes, that it is safer to hold a reference until the lambda is executed.
w

wasyl

02/05/2021, 9:10 AM
I’m not sure I follow 🤔 The lambda isn’t even created until after
println(foo)
is executed and the
var2
is never going to be used anymore. Can you give an example of when this
var2
is needed?
For example if I move
println(foo)
after
runBlock { }
, then I still see the variable generated:
Copy code
Foo var2 = (Foo)foo.element;
boolean var3 = false;
System.out.println(var2);
but it still captures
foo.element
right before printing.
i

Ilmir Usmanov [JB]

02/05/2021, 9:16 AM
The compiler uses the same logic for all captured variables, regardless of their kind - whether they are read after the call-with-lambda or not. This is because
ObjectRef
does not consume much memory - it is essentially object header + reference to another object. So, it is not a big deal.
In case of coroutines, however, https://youtrack.jetbrains.com/issue/KT-16222 could be a big deal, since a coroutine used to hold reference to potentially all local variables, which lead to memory leaks.
w

wasyl

02/05/2021, 9:20 AM
I must be missing something crucial, I just don’t understand why compiler can’t simply do
System.out.println((Foo)foo.element);
or
Copy code
Foo var2 = (Foo)foo.element;
System.out.println(var2);
var2 = null
But about clearing the coroutine state, I understand it’s still expected in this case: https://pl.kotl.in/25D0_X5w8?
i

Ilmir Usmanov [JB]

02/05/2021, 9:25 AM
Could you provide the bytecode, since there can be something for debugger, like, we need to unbox the value, so the debugger is able to show its value when we stop at breakpoint on
println
or something?
Yes, it is expected, since coroutines follow the same rules of captured variables.
w

wasyl

02/05/2021, 9:29 AM
The entire code is
Copy code
class Foo

fun test() {
    var foo: Foo? = Foo()
    println(foo)
    runBlock { foo = null }
}

private fun runBlock(block: () -> Unit) {
    block()
}
And the bytecode for the method is
Copy code
public final static test()V
 L0
  LINENUMBER 6 L0
  NEW kotlin/jvm/internal/Ref$ObjectRef
  DUP
  INVOKESPECIAL kotlin/jvm/internal/Ref$ObjectRef.<init> ()V
  ASTORE 0
  ALOAD 0
  NEW com/example/Foo
  DUP
  INVOKESPECIAL com/example/Foo.<init> ()V
  PUTFIELD kotlin/jvm/internal/Ref$ObjectRef.element : Ljava/lang/Object;
 L1
  LINENUMBER 7 L1
  ALOAD 0
  GETFIELD kotlin/jvm/internal/Ref$ObjectRef.element : Ljava/lang/Object;
  CHECKCAST com/example/Foo
  ASTORE 1
 L2
  ICONST_0
  ISTORE 2
 L3
  GETSTATIC java/lang/System.out : Ljava/io/PrintStream;
  ALOAD 1
  INVOKEVIRTUAL java/io/PrintStream.println (Ljava/lang/Object;)V
 L4
 L5
  LINENUMBER 8 L5
  NEW com/example/SpecSpecKt$test$1
  DUP
  ALOAD 0
  INVOKESPECIAL com/example/SpecSpecKt$test$1.<init> (Lkotlin/jvm/internal/Ref$ObjectRef;)V
  CHECKCAST kotlin/jvm/functions/Function0
  INVOKESTATIC com/example/SpecSpecKt.runBlock (Lkotlin/jvm/functions/Function0;)V
 L6
  LINENUMBER 9 L6
  RETURN
 L7
  LOCALVARIABLE foo Lkotlin/jvm/internal/Ref$ObjectRef; L1 L7 0
  MAXSTACK = 3
  MAXLOCALS = 3
i

Ilmir Usmanov [JB]

02/05/2021, 9:33 AM
Copy code
GETFIELD kotlin/jvm/internal/Ref$ObjectRef.element : Ljava/lang/Object;
  CHECKCAST com/example/Foo
  ASTORE 1
This is because of different type. The compiler does not create a 'new' variable. It uses additional slot, so it does not cast the variable every time the variable is accessed.
BTW, notice, it stores the value on stack, without allocating slot for it
Copy code
INVOKESPECIAL com/example/Foo.<init> ()V
without
ASTORE
Edit: I really hate slack visiwig redactor.
w

wasyl

02/05/2021, 9:38 AM
You can still turn it off, I think 🙂
This is because of different type
Okay but if I write
println(foo)
I see generated
Copy code
Foo var1 = (Foo)foo.element;
boolean var2 = false;
System.out.println(var1);
But if I have a function with the same type:
Copy code
fun useFoo(foo: Foo?) = println(foo)
then generated code is not using intermediate variable:
Copy code
useFoo((Foo)foo.element);
but it still does the same amount of casting
(I’m pasting java decompiled code because I’m not all that familiar with bytecode)
Apologies in advance if I’m asking stupid questions, I’m just under the impression that what you wrote about coroutines before:
a coroutine used to hold reference to potentially all local variables, which lead to memory leaks
is applicable in this scenario as well, and we can potentially hold on to massive objects even though looking at the code would suggest that we don’t. If we use
suspend fun
it’s not difficult to imagine
runBlock
running for a long time, in which case
Foo
would be kept as well
i

Ilmir Usmanov [JB]

02/05/2021, 9:52 AM
The compiler uses intermedate variables as it (I'd rather call her 'she') desires. For example,
boolean var2 = false;
has no effect, unless you use a debugger and it shows, that you are inside an inline function - in this case,
println
. I changed
useFoo
to be inline as well and lo and behold
Copy code
GETFIELD kotlin/jvm/internal/Ref$ObjectRef.element : Ljava/lang/Object;
    CHECKCAST Foo
    ASTORE 1
BTW, what is the problem? The compiler does not generate code, which creates additional memory allocations (one ObjectRef is hardly an issue for a generational GC). If you do not want to allocate even this amount of additional memory, you can declare
runBlock
inline. The last piece of advice - take decompiled code with a pound of salt - it never tells the whole truth. The bytecode tells much more. For example, the decompiler never generates correct names for inline variables, but still shows them as
boolean var2 = false
.
it’s not difficult to imagine
runBlock
running for a long time, in which case
Foo
would be kept as well
It's not difficult to imagine
runBlock
stores the lambda parameter and all its closure should live as long as the lambda is stored.
w

wasyl

02/05/2021, 10:01 AM
It’s not difficult to imagine 
runBlock
 stores the lambda parameter and all its closure should live as long as the lambda is stored.
No, but that’s relatively easy to spot when looking at the code, it’s just not surprising, as the same thing happens in plain Java. Just to circle back, I’m not complaining about
ObjectRef
or even that the compiler generates any variables she wants. It’s just that current behaviour broke my assumption about JVM being able to GC an object. In all the code I wrote in Kotlin (and could read), I cleared the references. But the JVM couldn’t GC
Foo
instance because it was kept as a result of what Kotlin compiler did internally.
It’s just the kind of thing one needs to know about, otherwise the code is not behaving as one would expect and it’s pretty difficult to find it out. The fact that lambdas can keep the reference to outer scopes is relatively basic, it’s a known problem. The fact that using an object might prevent it from being GCed after all user-written references are cleared is much less obvious
i

Ilmir Usmanov [JB]

02/05/2021, 10:09 AM
Well, this might be because code blocks and lambdas in Kotlin use the same syntax. Thus, you cannot tell, whether the variable is safely used in a block (of
if
statement) or is captured in lambda. Additionally, there is difference in inline lambdas and ordinary ones.
using an object might prevent it from being GCed after all user-written references are cleared
Clearing an object harms in this particular case, since it forces the variable to be captured.
w

wasyl

02/05/2021, 10:11 AM
Clearing an object harms in this particular case, since it forces the variable to be captured.
I want to be extra clear, I understand that
ObjectRef
is needed 100% and it doesn’t cause any harm here. Because when doing
foo = null
it will do
Copy code
L0
 LINENUMBER 11 L0
 ALOAD 0
 GETFIELD com/example/SpecSpecKt$test$1.$foo : Lkotlin/jvm/internal/Ref$ObjectRef;
 ACONST_NULL
 CHECKCAST com/example/Foo
 PUTFIELD kotlin/jvm/internal/Ref$ObjectRef.element : Ljava/lang/Object;
so there’s no longer any hard reference to
Foo
object and it should be GC-ed
i

Ilmir Usmanov [JB]

02/05/2021, 10:12 AM
It should. And it will. Eventually.
w

wasyl

02/05/2021, 10:12 AM
It will not, if I have
println(foo)
before
runBlock { foo = null }
*It will not as long as we’re in
test()
scope
i

Ilmir Usmanov [JB]

02/05/2021, 10:13 AM
How did you test?
w

wasyl

02/05/2021, 10:15 AM
This will crash as long as
println(foo)
is there, if you comment it out it will pass. Now I understand there might be some optimisations happening but if I understand what the bytecode does it shouldn’t be the case
i

Ilmir Usmanov [JB]

02/05/2021, 10:18 AM
The crash is a bug and belongs to youtrack. Can you, please, file it? I ran the code with captured variable
Copy code
fun test() {
    var foo: Foo? = Foo()
    println(foo)
    runBlock { foo = null }
    System.gc()
    repeat(10) {
        println("Try")
        Thread.sleep(10)
        System.gc()
    }
}
private fun runBlock(block: () -> Unit) {
    block()
}
class Foo {
    fun finalize() {
        println("Cleanup")
    }
}

fun main() {
    test()
}
And got
Copy code
Foo@7cc355be
Try
Try
Cleanup
Try
Try
Try
Try
Try
Try
Try
Try
So GC does what it should - it cleans the object up after just second call of
System.gc()
.
w

wasyl

02/05/2021, 10:22 AM
Apologies I’m not sure I understand. So the code I pasted is indeed a bug? And what’s the difference in what you shown works well?
i

Ilmir Usmanov [JB]

02/05/2021, 10:26 AM
Ah, I see. My bad. Crash usually means for me "Something gone wrong and resulted in incorrect behavior" and not "assertion triggered". Year. Your code is fine. GC does clean the variable up.
👍 1
And yes, turns out, there is a difference between captured variables in coroutines and ordinary code. Can you file a bug?
w

wasyl

02/05/2021, 10:37 AM
I’m sorry, but this all goes way over my head. I’m happy to file a bug, I’m just really not sure we’re on the same page.
there is a difference between captured variables in coroutines and ordinary code
The thing I have is reproducible both with and without coroutines
And your code in fact GCs
Foo
. But if you put the
System.gc(); repeat { }
part in a separate function, it doesn’t anymore:
Copy code
fun test() {
    var foo: Foo? = Foo()
    println(foo)
    runBlock { foo = null }
   	
    gc()
}

fun gc() {
    System.gc()
    repeat(10) {
        println("Try")
        Thread.sleep(10)
        System.gc()
    }
}

private fun runBlock(block: () -> Unit) {
    block()
}
class Foo {
    fun finalize() {
        println("Cleanup")
    }
}
fun main() {
    test()
}
prints
Copy code
Foo@5acf9800
Try
Try
Try
Try
Try
Try
Try
Try
Try
Try
(assuming we can trust
finalize()
)
I opened https://youtrack.jetbrains.com/issue/KT-44733 which is the only thing I still believe behaves unexpectedly. Please let me know if I should file something else
3 Views