kotlinlang #compiler

I need help by probably someone from JetBrains. I'm still writing a compiler plugin and I'm done with the command line part for now and currently working on ide support. My problem is, my code fails to resolve any classes using

ModuleDescriptor::findClassAcrossModuleDependencies(ClassId)

when running in IDEA. It works fine in the command line though. I really don't have any basis to perform debugging on when it comes to IDEA. All I could figure out so far is, that it uses the

JvmAnalyzerFacade

when running in IDEA, which resolves classes differently than the command line facade. Does anyone know what component or so I am missing in order to resolve dependencies correctly?

raulraja

06/19/2019, 9:27 AM

@dsavvinov thanks I guess the question is more technical and we are happy to help bring that support if that is feasible. At this moment I’m just puzzled by the following: Any random jar you build with Kotlin writing sources ends up in a

.class

. Idea is able to parse that presumably through the metadata annotations or parsing the class structure somehow. Why doesn’t this work for synthetically generated code which also ends up in a class file? What is different from that bytecode and the one a user types in by hand and is properly recognised by the IDE. Now for the human side… For what is worth I’m grateful for everything JB does but the state of meta programming and compiler plugins in Kotlin which has been frequently brought up at conferences and many times publicly does not seem to have a Roadmap or goals for that matter that are publicly known to the community. Instead companies like Google are forking the Kotlin compiler because that is easier since they have control of Android Studio and they have announced a Jetpack Compose, a framework as the future of Android Development which is based on a compiler fork. At this moment Jetpack Compose relies on a bunch of compiler extensions that are not public or have been modified to fit their needs, (many) I have read the sources and checked with their devs. Does that mean that all extensions they rely upon will be accepted upstream in the Kotlin compiler?. I’m not sure forking the compiler is good for anyone but the current state of meta in Kotlin is forcing companies to do just this. It’s not good for everyone when there are multiple versions of the compiler unless they are for KEEPs or enhancing the upstream compiler we all should rely upon.

tschuchort

06/19/2019, 2:48 PM

@raulraja Re metaprogramming: I'm working on an annotation processing library for Kotlin but that is probably not powerful enough for you. In any case, kotlinx-metadata is a lot more stable than the compiler itself.

raulraja

06/20/2019, 8:25 AM

@dsavvinov @dmitriy.novozhilov I have been looking around for solutions of having a compiler plugin automatically register the IDEA plugin needed if there is codegen involved and new synthetic members. For others reading this… Currently compiler plugins shipped from JB or Google have direct access to IDEA features because the IDEA part of the plugin is bundled alongside IDEA or AS. Community plugins like the ones we are writing in Arrow can’t activate automatic recognition of generated synthetic declarations. This makes IDEA show redlines in all generated code unless the user manually installs a plugin. I stumbled upon this https://intellij-support.jetbrains.com/hc/en-us/community/posts/206163949-Doing-automatic-installation-of-plugin which makes me think we can potentially copy the IDE specific jar of a compiler plugin directly to the user plugins folders via a gradle task that is part of the gradle plugin bootstrap. The idea plugin will then watch for new class files added to the output folder and whenever it finds a change it will retrigger parsing so those binaries are accounted for and redlines/autocompletion etc works again. Does this sound like a good approach or are we not seeing the obvious limitations?. The end goal is to not have users install both a gradle plugin and an idea plugin manually in order to user a community based compiler plugin.

:today-i-learned: 1

raulraja

06/20/2019, 12:57 PM

I see in many compiler plugins in their build.gradle:

id("jps-compatible")

. Does anyone know what that is for? Thanks.

lukaville

06/26/2019, 4:04 PM

Hi! Does anyone know any good article or documentation with Kotlin compiler internals overview? Particularly, I’m interested in how JVM interop works. Does Kotlin compiler run javac first? If yes, then how it resolves reference cycles between Java <-> Kotlin? For example, if I have some Android gradle module with mixed java/kotlin source code I can see that Gradle executes

KotlinCompile

task and

AndroidJavaCompile

task (from Android Gradle Plugin). Is

KotlinCompile

task contain java compilation or not?

👍 1

raulraja

06/26/2019, 4:36 PM

When I have more experience I'll gather my thoughts and write up some kind of guide if there is nothing official by then but it's still early for me

👍 1

tschuchort

06/26/2019, 8:06 PM

@lukaville Unfortunately this is all undocumented and this wisdom seems to be mostly passed on through word of mouth; I don't know of any good article that summarizes it. My advice would be to look at discussions on github issues of projects like the Kotlin Gradle Plugin or Buck build system. Sometimes you get lucky and a JB employee left a helpful comment. For a very high level overview you can also look at my kotlin-compile-testing libary which implements a simple build process with Java sources and annotation processors: https://github.com/tschuchortdev/kotlin-compile-testing/blob/master/src/main/kotlin/com/tschuchort/compiletesting/KotlinCompilation.kt Basically the build process is divided into 4 steps: 1. Generate stubs: The Kotlin compiler generates Java stub files which is like a Kotlin-to-Java source-to-source translation except that all the methods are empty (since the annotation processor doesn't care about that) 2. Run KAPT: The KAPT plugin is run which runs Java's APT tool internally (this is why the stub files are needed: APT only understands Java) 3. Compile Kotlin: K2JVMCompiler is run to compile only Kotlin sources. I think it actually does some compilation of Java sources internally, possibly by reusing Javac logic. Otherwise you wouldn't be able to reference Java classes in Kotlin 4. Compile Java: The remaining Java files are compiled regularly using Javac with already compiled Kotlin classes on the classpath. No annotation processing is done. Keep in mind that compiler internals are changing constantly and also heavily depend on the compiler flags. For example you can (theoretically) skip some of the steps by including certain compiler flags that may or may not be broken at any time.

🙏 1

Imran/Malic

06/27/2019, 3:01 PM

How to download the kotlin-compiler fronted common module ? // mavenCentral, etc.

Marc Knaup

07/10/2019, 4:52 PM

inline class Test(private val value: Byte) {
	fun toLong() = value.toLong()
}

Would I get any noteworthy performance benefit from making

toLong()

inline

? In general I'm not sure how functions of inline classes are actually invoked. As static functions with the inline value as parameter?

redrield

07/15/2019, 5:39 AM

Is it possible for a compiler plugin to modify the AST? I've tried PsiElement#replace and PsiElement#astReplace and both of them throw exceptions. The former throws a MissingResourceException, the latter a NullPointerException because PomModel.getModel returns null for the project I'm acting on

shikasd

07/15/2019, 10:21 PM

Hi! I have experimented with the compiler plugin API a couple of weekends ago, and created DI prototype based on some Dagger concepts (that's the most familiar compile-time DI for me). I am planning to continue developing it further, so it would be great to get some general feedback on the idea and execution. The repo is here: https://github.com/ShikaSD/kotlin-compiler-di The idea I have for further experiments is that with compiler plugin we are no longer limited with annotation processing, so the API can be much more fluent/accessible and richer feature-wise.

👍 5

PHondogo

07/20/2019, 6:36 AM

Good day, friends! I have one more question. Is there any Kotlin compiler extension to intercept with new IR transformation. I need to use it in my compiler plugin ComponentRegistrar.

Slackbot

07/23/2019, 3:02 PM

This message was deleted.

Alanna

07/24/2019, 10:58 PM

Hi! Where can I find any work-in-progress code in the kotlin repo that goes from Kotlin -> Kotlin IR -> JVM bytecode? I found test cases for Kotlin -> Kotlin IR but I'm having trouble finding a path from Kotlin IR -> JVM bytecode.

aurimas

07/25/2019, 11:09 PM

Hey there! I am working on a Java and Kotlin API public surface analysis tool (https://android.googlesource.com/platform/tools/metalava/). We used PSI and UAST for this work and I have hit a stumble when trying to check if a KotlinUMethod has a parameter that is of an inline class type. Would anyone have suggestions how to go about it?

PHondogo

07/31/2019, 5:55 AM

Hello! I've registered IrGenerationExtension for Kotlin compiler but it seems to work only for JVM. Does someone knows how to enable it to JS too?

eygraber

08/02/2019, 6:37 PM

What are the pros and cons of

kotlin.compiler.execution.strategy="in-process

? I can't find any documentation that discusses it.

thana

08/08/2019, 2:04 PM

hi, im faced with the kotlinx.serialization compiler plugin generating bad bytecode. any chance somebody here can help me debugging the bytecode the compiler printed out?

Adam Spofford

08/08/2019, 5:47 PM

I seem to have resolved it partly. turns out stdout is not written anywhere ever.

raulraja

08/10/2019, 9:27 PM

I see in the compiler sources several classes related to

fir

. Does anyone know what they are for? thanks.

Adam Spofford

08/14/2019, 10:09 PM

would it be feasible to write a compiler plugin that asserts every non-annotated thing is NonNull, but interprets

Optional<T>

T?

Robert Menke

08/20/2019, 8:40 PM

Sorry if this is a noob question, but why do reified generics require the function definition to be

inline

Matej Drobnič

08/21/2019, 10:45 AM

What would be the best way to start finding the cause for

Non-incremental compilation will be performed: inputs' changes are unknown (first or clean build)

message with Kotlin compiler (with gradle builds)? It seems like when starting to build, kotlin will occasionaly decide to do full build instead of incremental. Enabling build report produces above message.

bloder

08/24/2019, 2:17 AM

Hi all! I'm newbie with compilers / bytecode etc and I'm playing with Kotlin compiler plugin api, Is there any example of generating a new function by another function in a existing Kotlin class using plugins? (is it possible to do that with Kotlin compiler api?)

Derek Berner

08/26/2019, 4:07 PM

Suppose I need a

Map<KClass<T>, (T) -> Unit>

where the value type

(T) -> Unit

is covariant (?) with

. e.g.

map[Int::class]

has type

(Int) -> Unit

. There's no way to do this without casting, right?

raulraja

08/30/2019, 9:24 PM

I have some shared code between a Kotlin compiler plugin and the corresponding companion IDEA plugin. In the compiler plugin I was getting the Message Collector like this:

val messageCollector: MessageCollector = configuration.get(CLIConfigurationKeys.MESSAGE_COLLECTOR_KEY, MessageCollector.NONE)

This does not work on the IDEA side because CLIConfigurationKeys is not there. What is the proper way to obtain the MessageCollector in the IDEA plugin that does not use

CLIConfigurationKeys

? thanks!

Foso

09/01/2019, 4:55 PM

Hi everyone, i've written an annotation processing library that can be used in Kotlin Native/JS/JVM compiler plugins. I'm open for feature requests and if you have ideas to simplify my annotation detection or find bugs, please let me know. https://github.com/Foso/MpApt

👌 5

🚀 5

tschuchort

09/03/2019, 12:20 PM

Since when is it possible to have multiple companion objects? And how does the compiler decide which one is the companion object that gets referenced in the metadata?

raulraja

09/03/2019, 6:20 PM

Hi, I have a

KtTypeReference

for

kotlin.Unit

and it could be any other fqName for a type and I wish to resolve the type for. This is happening inside a function declared inside a class where I’m using the class member declaration scope to resolve the types. Currently I’m using:

val returnType = typeResolver.resolveType(containingDeclaration.scopeForMemberDeclarationResolution, function.typeReference!!, trace, true)

but this returns:

[ERROR : kotlin.Unit]

Is there a better way to go from

KtTypeReference -> KotlinType

than using the type resolver? Is there a different type of scope I should be constructing to resolve the type? Any help is appreciated, thanks.

Title

raulraja

09/03/2019, 6:20 PM

Hi, I have a

KtTypeReference

for

kotlin.Unit

val returnType = typeResolver.resolveType(containingDeclaration.scopeForMemberDeclarationResolution, function.typeReference!!, trace, true)

but this returns:

[ERROR : kotlin.Unit]

Is there a better way to go from

KtTypeReference -> KotlinType

than using the type resolver? Is there a different type of scope I should be constructing to resolve the type? Any help is appreciated, thanks.

shikasd

09/03/2019, 10:30 PM

There is a slice in

BindingContext

for

TYPE

, but it probably not populated before resolution. I am usually resolving the whole function using

ResolveSession.resolveDescriptor

and using it

returnType

. Not sure about efficiency of this, but it works most of the time 🙂

raulraja

09/03/2019, 10:33 PM

thanks I’ll try that. The

BindingContext.TYPE

is not populated yet so I need to resolve it, but I will try with the resolve session and report back

With the

ResolveSession

I get:

org.jetbrains.kotlin.resolve.lazy.NoDescriptorForDeclarationException: Descriptor wasn't found for declaration CLASS
	at org.jetbrains.kotlin.idea.project.IdeaAbsentDescriptorHandler.diagnoseDescriptorNotFound(IdeaLocalDescriptorResolver.kt:44)
	at org.jetbrains.kotlin.resolve.lazy.LazyDeclarationResolver.findClassDescriptor(LazyDeclarationResolver.kt:88)
	at org.jetbrains.kotlin.resolve.lazy.LazyDeclarationResolver.getClassDescriptor(LazyDeclarationResolver.kt:62)
	at org.jetbrains.kotlin.resolve.lazy.LazyDeclarationResolver.getMemberScopeDeclaredIn$frontend(LazyDeclarationResolver.kt:227)
	at org.jetbrains.kotlin.resolve.lazy.LazyDeclarationResolver$resolveToDescriptor$1.visitNamedFunction(LazyDeclarationResolver.kt:124)
	at org.jetbrains.kotlin.resolve.lazy.LazyDeclarationResolver$resolveToDescriptor$1.visitNamedFunction(LazyDeclarationResolver.kt:94)
	at org.jetbrains.kotlin.psi.KtNamedFunction.accept(KtNamedFunction.java:50)
	at org.jetbrains.kotlin.resolve.lazy.LazyDeclarationResolver.resolveToDescriptor(LazyDeclarationResolver.kt:94)
	at org.jetbrains.kotlin.resolve.lazy.LazyDeclarationResolver.resolveToDescriptor(LazyDeclarationResolver.kt:91)
	at org.jetbrains.kotlin.resolve.lazy.ResolveSession.resolveToDescriptor(ResolveSession.java:324)
	at arrow.meta.qq.QuoteKt$classOrObject$$inlined$quote$2$1.invoke(Quote.kt:221)
	at arrow.meta.qq.QuoteKt$classOrObject$$inlined$quote$2$1.invoke(Quote.kt)
	at arrow.meta.extensions.MetaComponentRegistrar$syntheticResolver$1.generateSyntheticMethods(MetaComponentRegistrar.kt:407)
	at arrow.meta.extensions.MetaComponentRegistrar$registerSyntheticResolver$1.generateSyntheticMethods(MetaComponentRegistrar.kt:596)
	at org.jetbrains.kotlin.resolve.extensions.SyntheticResolveExtension$Companion$getInstance$1.generateSyntheticMethods(SyntheticResolveExtension.kt:86)

I’m calling this from the

SyntheticResolverExtension.generateSyntheticMethods

shikasd

09/04/2019, 1:01 AM

I checked for a brief minute, it looks like idea has its own set of extensions for resolution ¯\_(ツ)_/¯ I am not sure if it is suitable in your case , but you can try it in the idea environment. https://github.com/JetBrains/kotlin/blob/master/idea/ide-common/src/org/jetbrains/kotlin/idea/caches/resolve/resolutionApi.kt#L65

kralli

09/04/2019, 6:03 AM

Well you can try something like this to resolve a general class

thisDescriptor.module.findClassAcrossModuleDependencies(ClassId.topLevel(fqName))

And for BuiltIns

thisDescriptor.module.builtIns.unitType

(this would be from within

SyntheticResolveExtension.generateSyntheticMethods

)

raulraja

09/04/2019, 7:03 AM

Thanks @kralli, will try that out.

View count: 12