Also, I've never actually stopped to actually anal...
# compiler
c
Also, I've never actually stopped to actually analyze the bytecode produced by Kotlin. For this - `class MyClass : AbstractJvmUnitTest(FirTestMetadata().also { with (MyTests()) { it.`do a thing`() }))` - it generated all of this: https://paste.ofcode.org/u7SyZRkzL6a6Nwn7x8CdJa, when this would have worked instead: https://paste.ofcode.org/6zVRA6bzejLbm3MUB9pjdg I'm mainly just confused... Why are 0s being stored in LV 3 and 5 and never referenced again? Why is
this
being copied from LV 0 to LV 6? Is it supposed to do that?
✅ 1
y
The 0s, I believe, are markers that arise from inlining. This makes debugging through
inline
functions work. All of these "inefficiencies" are easily optimized by ProGuard and/or the JIT
c
This makes debugging through
inline
functions work.
Ahhhh, that makes sense. Glad to know it's not as bad as I thought.
Is there a way to view the Hotspot output of some given bytecode? Ideally through something like godbolt, but locally would work too. I'm just curious to see what it looks like.
c
Thanks!
On the topic of Java bytecode: is
INVOKEVIRTUAL
a safe opcode to use for instance methods in general, even ones that aren't actually virtual? I just want to double-check
y
I believe so. IIRC, virtual is for class methods (so it'll use a vtable lookup), interface is for interface methods (so it'll use an itable lookup, which is slower), dynamic is for magic stuff (lambdas, strings, varhandles, etc), and static is for static methods (so no dispatch receiver)
c
dynamic is so frickin cool btw, I looked more into it and it has so much
I can implement structural typing using native jvm processes
you can make traits that you add to third party types
and have it be resolvable just like an interface
it's straight up the same freedom as like a dynamic-linker in C
actually wait no that's literally what it is
it's literally a dynamic linker
y
I don't imagine it's particularly fast, though, which is expected. Java is going with typeclasses (through witnesses) instead of indy shenanigans
c
Actually, it uses the same optimization as interfaces
caching by type and whatnot
at least it did back in 07 when it was introduced
Java is going with typeclasses (through witnesses)
They're doing product types? I never heard about that
Or am I conflating different terms
y
Not sure what you mean by product types (sounds like
Pair
to me), but they're basically doing something like Haskell's typeclass lookup system and Rust's trait impl lookup system.
c
That's... so cool. That's already what I've been wanting to implement for Kotlin for years. Where did you hear about this?
And yeah product types are basically dynamically-formed named tuples
y

https://youtu.be/Gz7Or9C0TpM?si=y4s1DDy9nVYte91G▾

It takes some time, but he eventually gets into the witness stuff
👀 1
I've been wanting to implement it too! It's the part missing from context parameters: a lookup mechanism. Java is only getting the lookup mechanism, and not the context parameters part. I'm hoping Kotlin will follow Java in this part. Interestingly, the Java witness stuff adds nothing new to the bytecode, so it's purely language side. It does rely on project Valhalla stuff, though, with the idea being that the witnesses are inherently
const
. Such a limitation doesn't need to exist in Kotlin, though, but it can be nice to limit the behaviour of the lookup system and make it predictable (e.g. no side effects executed by the lookup).
I recently wrote a post translating Rust's trait intro into Kotlin, and it's painfully obvious that Kotlin needs a lookup mechanism to make things cleaner
c
I've been thinking about using value classes, bringing them the rest of the way towards being full Rust-like traits. All it would need is a compile-time lookup system, some C++-like function templatization, and a way to easily declare the conversion. Then you could just call
obj.into<T>()
on an object and it would be substituted at compile-time with
ThatTraitImpl(obj)
.
Of course value classes are ABI hell though
I really like value classes as traits though, because it's a great way to declare a specific context for an arbitrary item
Using C++ has made me really understand just how powerful compile-time metaprogramming is. Things like
runValueGettingAlgorithm(&OutputWithBehaviorDefiningWhichValueToGet)
where you have the same procedure run to get some arbitrary value no matter which one it is, and then a different output type depending on which value(s) you actually want to extract from it bundled with HOW you get those values, all resolved at compile-time
y
You'd end up with a ton of boxing, though. I think context parameters are nicer here. You declare your traits differently, but you use them in a similar way. It also works way better for static methods that you'd want on a trait. For instance, take a mathematical field. You'd normally write it like this:
Copy code
interface Field<T> {
  operator fun plus(other: T): T
  operator fun times(other: T); T
  operator fun unaryMinus(): T
  fun multInverse(): T
}
But where's the 0 and 1? The context style is nicer and clearer IMO:
Copy code
interface Field<T> {
  operator fun T.plus(other: T): T
  operator fun T.times(other: T); T
  operator fun T.unaryMinus(): T
  fun T.multInverse(): T
  val zero: T
  val one: T
}
In some sense, this is like C++ concepts
c
The problem with the context style is that you end up with a hugely-cluttered intellisense, where all contexts overlap on the same item regardless of whether or not they're applicable to your use-case. Compile-time boxing is super intuitive if the conversion method is concise.
As long as it's not runtime boxing
Screw runtime boxing
y
Can you elaborate further on the cluttered intellisense? Do you mean that like top-level context methods show up, even if their context isn't in scope?
c
Essentially. Or like with Rust where, since traits are a stock replacement for interfaces, every single trait's methods show up on the same object when you do
obj.
.
I'm wondering if we're talking about the same thing though
> Essentially. Or like with Rust where, since traits are a stock > replacement for interfaces, every single trait's methods show up on the > same object when you do
obj.
. The alternative being that you only see those methods when you say "this object is being viewed as x"
Like this: https://github.com/cbrandt77/Kotlin-CompositeAnnotations/tree/main/compiler-plugin[…]plugins/kt/compositeannotations/fir/helpers/annotations/symbols This is more of a conventional use of "value classes as denoting context-specific assertions and the use-cases extending from that", but it's a similar idea
y
I think we are. For instance:
Copy code
context(_: Foo) fun foo()
// somewhere without Foo in scope
fo// intellisense completes foo, but then complains because it isn't in scope
I think that's technically a feature, not a bug, with the intention being API discovery. I agree though that, depending on your API, you don't want all these methods to show up. I think it should be easily fixable on the IDE side though. Simply introduce some
@GetOutOfMyWayAutoComplete
annotation that you can annotate
foo
with so that it only shows up if all its contexts are in scope
So that's a different beast, in my eyes. It reads more like refined/restricted types. Importantly, it doesn't carry with it info about how this type implements some interface, while the context design does. I think some new Kotlin feature would be nice here:
Copy code
restricted type ComposableAnnotation = FirAnnotation 
fun ComposableAnnotation.foo() {}
val FirAnnotation.isComposable get() {
  contract { returns(true) implies this@isComposable is ComposableAnnotation }
  return ...
}
// you can only call `foo` if you check `isComposable`
if (ann.isComposable) { ann.foo() } // smart casted
c
The main use-case for traits that I like is stuff like this, which I implemented when working for NASA (it's open-source dw): https://paste.ofcode.org/b8SEB5zXVhwmxV5Dgt2fNQ
Allowing you to output either its Vec position, its Quaternion orientation, or a PairVec, Quat of both, using the same algorithm, with the "what data it stores and how" behavior defined by the _OutputsTrait
Defining common contextual behaviors performed on a given arbitrary type, at compile-time and inline
Interfaces can handle the common non-contextual behaviors imo
edit: forgot the doc comment to actually explain what's going on https://paste.ofcode.org/34G3p37vGBv7Ri5cnUxHni6
And then you have stuff like this defined: https://paste.ofcode.org/q9FCErutpgJ2D8Y2fzNJVz
idk I'm just particularly proud of it, ignore me
it's something that's sorely needed when you can't embed something in a transparent struct without a heap alloc
What use-cases were you thinking about?
y
I lack a lot of knowledge about C++, so apologies if I misunderstood the code, but I think contexts work nicely here:
Copy code
interface OutputsTrait<T> {
  fun T.zeroOut()
  operator fun T.plusAssign(pos: RelativePosition)
}
// assume I have the bridge methods here somewhere

context(toGet: DataToGetWhenRecursing, _: OutputsTrait<O>)
fun <Out> getOffsetsRelativeToCommonParent(pos1: RelativePosition, pos2: RelativePosition, out1: Out, out2: Out): ErrorCode {
    out1.zeroOut()
    out2.zeroOut()

	val err = if (pos1.nestIndex == pos2.nestIndex) {
		if (pos1.parent == pos2.parent) {
			out1 += pos1
			out2 += pos2
            OK
		} else getOffsetsRelativeToCommonParentBoth(pos1, pos2, out1, out2)
	} else if (Pos1.NestIndex > Pos2.NestIndex) 
        getOffsetsRelativeToCommonParentHigher(pos2, pos1, out2, out1)
	else getOffsetsRelativeToCommonParentHigher(pos1, pos2, out1, out2)

	if (err == OK)
		return OK;

	// on error
	out1.zeroOut()
	out2.zeroOut()
	return err;
}

// impl
object RecursingHandleOutputsTrait: OutputsTrait<Pair<FVector, FQuat>> {
    override fun Pair<FVector, FQuat>.zeroOut() {
        first.set(FVector.zero)
        second.set(FQuat.identity)
    }
    override fun Pair<FVector, FQuat>.plusAssign(pos: RelativePosition) {
        first += pos.relativePos
        second *= pos.orientation
    }
}
As you can see,
OutputTrait
works really nicely as a context
c
Huh!
So they do!
I didn't expect that
y
I didn't use methods from
DataToGetWhenRecursing
, since it's only used, well, when recursing, but it'd work nicely as well, and is passed implicitly to the auxillary methods
c
Yeah
DataToGetWhenRecursing
is just an enum so the right
OutputTrait
specialization is selected
Assuming we have different predefined instances of OutputTrait for the different specializations (e.g.
OutputTrait.Both
,
OutputTrait.Pos
,
OutputTrait.Orient
), that would do the same thing
✔️ 1
y
Ohhhhh, I see! I'd just do this instead:
Copy code
context(toGet: OutputsTrait<O>)
fun <Out> getOffsetsRelativeToCommonParent(pos1: RelativePosition, pos2: RelativePosition, out1: Out, out2: Out): ErrorCode {
    ...
}
object DataToGetWhenRecursing {
  object Both: OutputsTrait<Pair<FVector, FQuat>> { ... }
  object Pos: OutputsTrait<FVector> { ... }
  object Orient: OutputsTrait<FQuat> { ... }
}
c
Yeah precisely
I mean it's still finding the behavior at runtime instead of compile-time, and still requires multiple runtime objects instead of compile-time objects, but yeah it works syntactically
I wish the JVM had stack alloc so I could stop stressing about object allocations
Though yeah that does work better than the alternative of wrapping each individual output parameter in a new
OutputsHandler
instance
➕ 1
y
Here, the lookup mechanism isn't necessary, since you want to be explicit about which strategy you're using, but it'd be nice to have so you can do:
Copy code
getOffsetsRelativeToCommonParent(pos1, pos2, someQuat, someOtherQuat)
and it resolving automagically. > I mean it's still finding the behavior at runtime instead of compile-time, and still requires multiple runtime objects instead of compile-time objects, but yeah it works syntactically I wouldn't worry too much about that, with how smart the JIT usually is. There's some discussion about getting some
constexpr
magic that would ensure that, though. In fact, I'd be very surprisied if marking
getOffsetsRelativeToCommonParent
as
inline
didn't magically get rid of any inefficiency. R8 is a dang good optimizer, too, so it might be able to
inline
this itself, and once inlined, it surely can get rid of the interface entirely, and make it so the methods are statically known
c
I think you've won me over on that front
y
If you must ensure no allocations, though, then simply use
inline
lambdas:
Copy code
typealias ZeroOut<O> = O.() -> Unit
typealias Add<O> = O.(O) -> Unit
inline fun <Out> getOffsetsRelativeToCommonParent(pos1: RelativePosition, pos2: RelativePosition, out1: Out, out2: Out, zeroOut: ZeroOut<O>, add: Add<O>): ErrorCode {
    // any mentions of foo += bar should now use foo.add(bar) instead
    ...
}
object DataToGetWhenRecursing {
  object Both { 
    fun zeroOut(p: Pair<...>) = ...
    fun add(p: Pair<...>, other: Pair<...>) = ...
  }
  object Pos { ... }
  object Orient { ... }
}
// usage
getOffsetsRelativeToCommonParent(pos1, pos2, out1, out2, Both::zeroOut, Both::add)
It's guaranteed to inline at compile time, and statically refer to
Both.zeroOut
. Any optimizer worth its salt can then get rid of the
object Both
completely
c
Though what's funny is that the "automagically" thing you're talking about is exactly what C++'s template specialization does already. I think I could've written it like this:
Copy code
template<class OutputType = Pair<FVector, FQuat>>
struct OutputsTrait {
  static void ZeroOut(OutputType& it)
	{
		it = TPair<FVector, FQuat>(FVector::Zero(), FQuat::Identity);
	}
}

template<> struct OutputsTrait<FVector> {
  static void ZeroOut(OutputType& it)
	{
		it = FVector::Zero();
	}
}

template<> struct OutputsTrait<FQuat> {
  static void ZeroOut(OutputType& it)
	{
		it = FQuat::Identity;
	}
}

template<class _OutType, class _OutputsTrait = OutputsTrait<_OutType>>
void getOffsetsEtc(..., _OutType& Out1, _OutType& Out2) {
  _OutputsTrait::ZeroOut(Out1);
}
And then it could be called just like this:
getOffsetsEtc<FVector>(..., FVector(), FVector())
Kotlin would let the known generic be elided, so you'd get the "automagic"
getOffsetsEtc(FVector(), FVector())
y
So that's the C++ equivalent to the "lookup mechanism" I've been describing, good to know! Have you played around with concepts at all? I think they're pretty close to contexts in this regard. They define a clear interface, using type parameters ocassionally, that then a function may use. Kotlin has the advantage of nice extension function syntax
c
If you must ensure no allocations, though, then simply use
inline
lambdas:
The issue with making the whole function inline is that it's a massive function being called in a lot of places. The code bloat would be enormous.
> Have you played around with concepts at all? I haven't had the chance to. I was stuck on an older version of C++. Government work, etc. I'll have to look into them!
y
The issue with making the whole function inline is that it's a massive function being called in a lot of places. The code bloat would be enormous.
Isn't that what `template`s do? Or are they smart enough to only produce one copy per unique arguments?
c
Yes exactly, only one per set of unique arguments. The "trait structs" don't even exist at runtime, and their methods are resolved statically in the body of the generated function.
That's something I'd love to see with inline functions btw, being able to switch on a reified type parameter at compile-time instead of switching on its class
y
> only one per set of unique arguments. Hmm, you can get pretty close by making your own overloads per type, but I do agree, that's an important disadvantage with
inline
> being able to switch on a reified type parameter this kinda works, but it's not fully at compile time (although sometimes the compiler does optimize this IIRC):
Copy code
inline fun <reified T> foo() {
  if (5 is T) { ... }
  else if ("" is T) { ... }
  else if (FQuat.zero is T) { ... }
}
c
I wish I could do this at compile-time: https://paste.ofcode.org/K4N8N29t3fRpaggP4gcEUw
Or even stuff like this:
Copy code
// ASM opcodes
interface Opcodes { ... }

template<int OPCODE>
inline fun Int.invoke()

template<OPCODE == ALOAD>
inline fun OPCODE.invoke(variableIndex: Int) {
  +VarInsnNode(ALOAD, variableIndex)
}
y
You'd need some example values of each, then you can do:
Copy code
when {
		someAnnotationCall is T -> annotationCallCheckers
		someAnnotation is T -> annotationCheckers
        ...
}
I remember decompiling, and seeing a lot of these branches disappear since the compiler knew they must be false. Even if the compiler doesn't do it, they're statically-false conditions, so optimizer/JIT will get rid of all the branches but the one useful one
c
That ASM API is my white whale btw:
Copy code
fun makeMethodInsns() = InsnList().apply {
  +NEW("my/class/MyClass")
  +DUP
  +ALOAD(1)
  +INVOKESPECIAL("my/class/MyClass", "<init>", "()V")
}
y
Well, if you're willing to accept an enum, then you can have
invoke
in the enum class. I agree, the ASM API is disgusting lol
c
You'd need some example values of each, then you can do:
I was switching on a reified generic type param, which means only the compiler knows it- oh I see what you're doing. That's a hacky as hell workaround, that's so weird
I wish I could just do this:
Copy code
when (T) {
  FirAnnotation -> ...
  FirFunction -> ...
}
Something something type-traits
type-based constexpr eval
Sidenote: is there a way to hijack K2's
IntrinsicConstEvaluation
system? I could do so much with that.
y
That's a hacky as hell workaround, that's so weird
it gets weirder, you can detect whether a reified generic is nullable or not:
Copy code
if (null is T) // do something that requires a nullable T
I wish I could just do this:
Contexts work nicely too, but you just need a lookup mechanism so that you don't suffer:
Copy code
interface AddChecker<T: FirExpression> { fun addChecker(checker: FirExpressionChecker<T>) }
object AnnotationCallAddChecker: AddChecker<FirAnnotationCall> { fun addChecker(...) = ... }
...
addChecker(someAnnotationCallChecker) // should magically bring in AnnotationCallAddChecker as a context
In fact, if you know there's some set of contexts you want, you can bring them all in one fell swoop:
Copy code
inline fun withAddCheckerContexts(block: context(AnnotationCallAddChecker, AnnotationAddChecker, ...) () -> Unit) = block(AnnotationCallAddChecker, AnnotationAddChecker, ...)
// usage
withAddCheckerContexts {
  addChecker(someAnnotationCallChecker)
  addChecker(someAnnotationChecker)
}
contexts are smart enough (I think, at least) to choose the right context based on the type
is there a way to hijack K2's
IntrinsicConstEvaluation
system? I could do so much with that.
I'm not sure, but it depends on what you want to do with it. If it's just about the efficiency of the produced code, you can always use an IR transformer and replace any code you want. FIR has more limited options
c
It's for going down different control flow paths based on the value of the constant-expression. A common sight in C++ is something like this:
Copy code
template<> fun foo(intParam: Int) {
  if constexpr (intParam == 4) {
    println("Param is 4")
  } else {
    println("Param is something else")
  }
}

foo(3)
foo(4)

// compiles to:

fun foo-4(intParam: Int) {
  println("Param is 4")
}
fun foo(intParam: Int) {
  println("Param is something else")
}

foo(3)
foo-4(4)
It becomes insanely powerful when you combine it with type-traits (assertions about a TYPE, like "is integral", "has a method named foo", etc.). Since it's just a source code replacer, you can do anything from structural typing to hotswapping entire APIs without any runtime penalty.
y
What happens if you call
foo(someInt)
when the value is not known at compile time?
c
whoops I oversimplified the example way too much
constexpr only works on essentially const vals
so that int would have to be a template parameter, not a value param
mb
It's like how in Rust you have size params for methods, that allow you to make a "dynamically-sized" (at compiletime) array
Copy code
fn makesArray<const SIZE: i32>() {
  let arr = foo[SIZE];
}
y
Well, you can combine a FIR checker (that ensures the value is constant) and an IR plugin (that splits the method into 2 or more, and calls the apt methods how you want) to get what you want there. I agree though, some hook-in to const evaluation would be nice; I believe that is planned for some future time
c
But like for instance, an
equals
method is sometimes this:
Copy code
operator fun <T> equals(other: T) {
  if constexpr(T != MyType) {
    return false;
  } else {
    // actually perform the checks
  }
}
And then you end up with one
equals
function that always returns
false
and another that actually has a body. Then the one that always returns false would likely get inlined by the compiler
Since C++ doesn't have runtime type-checking
but it does have compile-time type-checking
y
If we ignore the pre-existing
Any.equals
, then you can get close using contexts:
Copy code
interface Eq<T> { infix fun T.eq(other: T): Boolean }
context(_: Eq<T>)
inline fun <reified T, reified R> T.eq(other: R): Boolean = this is R && other is T && this eq other
although that does runtime stuff if
R
is some supertype of
T
. Or instead, purely at compile-time:
Copy code
interface Eq<T> { infix fun T.eq(other: T): Boolean }
context(_: Eq<T>)
fun <T> T.eq(other: Any?): Boolean = false
context(eq: Eq<T>)
fun <T> T.eq(other: T): Boolean = with(eq) { this@eq eq other } // bridge method
c
Interesting. You've given me a lot to think on. I prefer Rust-like explicit trait conversion personally because it hides less from the caller, but I see your idea's use-cases as well. I think there are merits to both. Either way, at least we can both agree it's better than implicit constructors.
y
I don't think Rust technically does trait conversion. In fact, I think it's a lot closer to contexts. My understanding is that when you use trait bounds in rust, it uses the apt implementation at compile-time, which is darn close to a function with
context()
. There's no real "conversion" going on. For trait objects, my understanding, again, is that they pack the trait implementation together with the object that it needs to operate on. I had an example of that in my post (which is probably 20 or so messages ago lol) In fact, Rust traits seemingly don't need to take a
self
parameter for each method, so they're even closer to contexts than they are to conversions on a type
c
In fact, Rust traits seemingly don't need to take a
self
parameter for each method
It's for Rust's version of type-traits (properties of a type), in lieu of a runtime reflection system.
Rust's been building out their type-trait system these past few years
You can use traits like those compile-time template structs
y
That's the vocabulary I'm missing here, I think. I use contexts as type traits (akin to Haskell's type classes).
c
I guess I should clarify: type-traits are things like these:
Copy code
// Contains the aliases for types that mean "T&"
template<typename T>
struct TReferenceType {
  using Ref = T&
  using ConstRef = const T&
}

// Specialization for types that are already a reference
template<typename U>
struct TReferenceType<U&> {
  using Ref = T // U&
  using ConstRef = const T // const U&
}

// Specialization for types that are already a CONST reference
template<typename U>
struct TReferenceType<const U&> {
  using Ref = TRemoveConst<T>::Value
  using ConstRef = T
}

// usage:
template<typename T>
fun acceptsReferenceOnly(param: TReferenceType<T>::Ref)
Copy code
// Assertion of the properties of a type
template<typename T>
struct TIsIntegral {
  enum {
    Value = false;
  }
}

// Specialization for `int` saying that the "int type" is "integral"
template<>
struct TIsIntegral<int> { 
  enum {
    Value = true;
  }
}

// Usage
template<typename T>
fun differentActionIfParamIsIntegral(mightBeInt: T) {
  if constexpr (TIsIntegral<T>::Value == true) {
    // act on int
  } else {
    // act on other type
  }
}
But actually I think you are talking about the conventional C++-style "traits", like I had with my getOffsets example
y
I think the latter "Assertion of the properties of a type" is more so what I'm talking about. I can image you can add methods to that struct, no? So those extra methods can then be extension methods in Kotlin. Your getOffsets example is exactly what I'm talking about as well!
c
Yeah actually I think I get what you're talking about now. You're talking about passing a struct containing behavior for things next to the things it acts on, instead of wrapping them. I feel like that clicked earlier, but it's clicking again. It's not a Rust-style "basically just an interface" trait, but it is a C++-style "behavior-defining parameter" trait
I wonder if it would be better to define those "contextual behavior parameters" in the same breath as type parameters instead of as context params, so they can be elided if unambiguous or explicitly stated if ambiguous
Cause there will be conflicts with multiple specializations of the same trait
Like back to the getOffsets example,
RPOutputsTrait.PAIR = object : OutputsTrait<Pair<Vec, Quat>>
vs
ThirdPartyOutputsTrait.PAIR = object : OutputsTrait<Pair<Vec, Quat>>
It's a thing in C++, where you have multiple implementations of, for example,
std::hash<T>
that defines the hashing method for a type T. Since it's just looked up as "find the implementation of `std::hashThatType`"
Context params are kinda clunky to define if you need to, while type params are a known quantity, easy to define in-line.
Or maybe I'm just clinging to C++'s "all template parameters - both generic-type params and physical trait params - are defined in the same angle brackets"
y
Well, the ambiguity is a concern only if you have a lookup mechanism, which contexts don't. However, it can still arise locally, and in that case, explicit context arguments are becoming a thing. Still, I see the advantage of wanting to define:
Copy code
fun <T: Field> foo() {}
// Desugars to
context(_: Field<T>) fun <T> foo() {}
In fact, Scala has this!
c
Honestly I might implement context parameters in Rust, since it seems like they'd solve its single biggest problem: prop drilling Since you aren't supposed to have a "global state" in Rust, and the language is explicitly designed to make that difficult, so you're forced to explicitly pass a "global state" parameter to every single function even if it doesn't need it, because its child might need it