I need to redirect all calls to one function to an...

# compiler

[Caleb B]

I need to redirect all calls to one function to another function, even across library boundaries. Is there any way to textually point to a specific overload of a function such that the IR would be able to read it from a compiled library, like a function descriptor that I could save to the binary?

I know that there are function descriptors in IR, but that's a deprecated API. What can I use that would achieve the same effect?

[Youssef Shoaib [MOD]]

if the "one function" is being compiled with your plugin applied, then simply you can change its body to call the other function. Otherwise, I'm not sure how you can do this reasonably (it sounds like you'd effectively be shadowing a function, but, to my understanding, that can have issues depending on the platform)

[Caleb B]

It's more using a DSL to fabricate a function, and then redirecting any calls to the DSL function to the actual function when they have different parameters and structure.

[Youssef Shoaib [MOD]]

I'm not sure then what you mean by "across library boundaries". I'd probably use FIR to generate this function, and then in IR, have its body call the real function you want to call.

[Caleb B]

Basically, some library author creates a function using the DSL. Another person depends on that library, calls their DSL function, and has their call to it redirected to the generated one.

[Caleb B]

I'm turning listOf("").asSequence().map(...).downstreamAction() into for (el in listOf("")) { map(el) { downstreamAction(it) } }

[Caleb B]

It's actually come along very nicely, I just need to differentiate between multiple stream operations with the same CallableId, like flatten<Iterable<T>> and flatten<Array<T>>

[Youssef Shoaib [MOD]]

So here's the way you should think about this:
1. You need the code to be accepted by the frontend. The frontend can't magically let users call functions that don't exist. Most errors/diagnostics happen in the frontend, so you need code to compile there. The way to do that is with FIR, which lets you generate functions that'll be visible both in this module, but also to any dependents on this module
2. You need the code to compile to something reasonable by the backend. Usually, the trick here is to use an IR plugin, although you could do bytecode transformation instead if you really want. If you're generating functions, then all you need to do is to simply fill in their bodies in IR, and have them call the original function

[Youssef Shoaib [MOD]]

flatten<Iterable<T>> and flatten<Array<T>>

I think you can access type parameters in IR, so this should be doable. However, I'd instead recommend you create multiple functions in FIR, one for each collection type you support (you'll likely only have 2-3 anyways), and thus their bodies can be trivial.

[Caleb B]

So far, I've found that generating the "actually called inside the loop" functions in FIR (generated from the user-defined "chainable operation" functions) causes more problems than it solves. But even in FIR, I'd still need some way to disambiguate them when one is a compiled library and the other is being actively compiled.
I can easily check the parameters and types, etc., but I was hoping there was something I could do to concretely point to it. For example, on the JVM I could write the function descriptor to an annotation, and I could read that descriptor and know exactly which function it's pointing to.

[Caleb B]

I'd basically like to serialize a function pointer, rather than try to figure out which overload is right by manually analyzing their types.

[Youssef Shoaib [MOD]]

Can you give an example of what you're generating right now? I'm somewhat struggling to understand what your calls look like.

[Youssef Shoaib [MOD]]

I was gonna say, pasting in a code sample will be easier to read than a screenshot :)

[Caleb B]

Eh, it's pretty long, I think the screenshot will work better

[Caleb B]

Like I can check all of the types for each function easily enough, it's just really computationally expensive compared to serializing and deserializing the symbolic reference

[Youssef Shoaib [MOD]]

Btw, this feels like you're doing a list comprehension. Is that kind of the intention here? I have a library that already lets you do that using suspend

Is the problem here that you want to take one look at list.asInlineSequence().drop(1) in IR and immediately have access to the apt inline fun drop?

[Caleb B]

Yeah, exactly

[Caleb B]

And it solves the same problem as list comprehension, but without any intermediate objects.

[Caleb B]

It's just functional (and extremely clean) syntax for a for loop, without any runtime penalty

[Youssef Shoaib [MOD]]

I believe there's a CallableId thing you can use to uniquely identify each generated function, and simply add some serialized version of it to an annotation on the InlineSequence extension. Then, you simply use referenceFunctions or whatever it's called to look up functions by their FQN, and simply filter them by the callable id

[Caleb B]

Yeah, there's a callable id, but that's just owner and name. It doesn't include the parameter types, and referenceFunctions takes in that callable id and returns every single overload

[Caleb B]

It got refactored to use CallableId instead of FqN a while ago

[Youssef Shoaib [MOD]]

Another trick you can do is simply generate some unique ID for every generated function, store it in some annotation on both the generated and original function, then filter based on it.

You might also wanna try out my library, especially the Choose dsl, which gives you list comprehensions and more.

[Youssef Shoaib [MOD]]

In other words, I'm thinking like:

// annotation added in IR
@MyCustomId(1)
fun InlineSequence<T>.drop(n: Int) = ...

// created in IR
@MyCustomId(1)
fun drop(...)

// now consider in a different lib we see this call
list.inlineSequence().drop(1)
// we now have the callable id of `InlineSequence<T>.drop`, so we use that to look up all `drop` functions in the same package
// we also have the id from the annotation, so we can filter the results to only get the single function with the correct ID

[Caleb B]

Oh duh, of course I can add my own metadata! I got so focused on the hard-coded links that I forgot that any functions sharing a callable id will probably be in the same module

[Caleb B]

Though... how are external top-level members handled in KMP targets? When loaded on JVM they're members of the "FileNameKt" class, so I don't have to worry about people adding new overloads of an existing CallableId, but what about in other targets?

[Caleb B]

I'm thinking about like:

// MODULE: lib1
package libpkg;

@MyCustomId(1)
fun drop(x)

// MODULE: lib2
package libpkg;

@MyCustomId(1)
fun drop(y)

// MODULE: main(lib1, lib2)
import libpkg.drop

list.inlineStream().drop(x) // have same CallableId, but also same custom ID because they were compiled separately

[Caleb B]

But I guess I could always encode my own function descriptors

[Youssef Shoaib [MOD]]

Even a hash based on the function's parameter types and such would likely be enough I think

[Caleb B]

Good idea!

[Youssef Shoaib [MOD]]

Also, you can place classes in annotations, so you can create a class for each function you generate. This is somewhat equivalent to this hashing approach anyways because you'll have to give the class a unique name somehow

[Caleb B]

How do you make the IR diff ignore that the hashcode changes each time? Can you set a pattern that it's fine as long as it matches?

[Youssef Shoaib [MOD]]

Ideally, you want compilation to be deterministic, so I'd say try to figure out a consistent way to hash things.