I was using the `generatedByKsp` sourcesets in my project for code that I generate externally that s...

# ksp

[David Herman]

I was using the generatedByKsp sourcesets in my project for code that I generate externally that should not get processed by ksp. That just went away in 1.9.20-RC2. Any advice for migration steps?

[David Herman]

To make things a bit more nuanced (in case it matters), the code that gets generated externally is not generated by ksp but ksp is run first in order to process the user's code and then generate some metadata that I use to generate that additional code. Basically a tag team between ksp and Gradle.

[Ting-Yuan Huang]

Can you try to add them directly to compileKotlin tasks? KSP takes source set from KGP, not compileKotlin , so sources direclty added to compileKotlin won't get processed by default.

[David Herman]

We'll try, thanks!

[David Herman]

Any guidance on how to add source files to the compileKotlin tasks?

We tried a few options but are still not sure how we're supposed to add them. For example, there is a "source" function but when we try to use it we get "The value for this file collection cannot be changed". Any example code you can link to by any chance?

[Ting-Yuan Huang]

May I know when you call source ? IIUC it should be fine in the configuration phase of Gradle.

[Ting-Yuan Huang]

Here is an example when KSP cannot add generated source to the source set, and it has to add to the compileKotlin directly: https://github.com/google/ksp/blob/main/gradle-plugin/src/main/kotlin/com/google/devtools/ksp/gradle/KspSubplugin.kt#L539

[Ting-Yuan Huang]

A common cause that source cannot be changed is that it is already resolved accidentally at configuration time.

[David Herman]

We are setting source at configuration time. Here's the rough code (a bunch of noise ellided):

Plugin<Project> {
  override fun apply(project: Project) {
    project.extensions.getByType<KotlinMultiplatformExtension>().targets.withType<KotlinJsIrTarget>().configureEach {
    project.tasks.withType(Kotlin2JsCompile::class.java).configureEach {
      this.source(genTask)
    }
  }
}

where genTask is a Gradle task that is associated with an output source dir.

[David Herman]

We're of course not confident yet, but we wonder if you have access to an earlier point in the Kotlin compilation lifecycle since you're overriding a method given to you from the Kotlin Compiler plugin that we don't have access to when registering with Gradle.

[Ting-Yuan Huang]

this.source(genTask) looks fishy to me. Should it be it?

[David Herman]

Sorry, that is code we're trying to simplify

[David Herman]

genTask is a task we registered ourselves elsewhere, but we didn't include it in the example above

[Ting-Yuan Huang]

I mean, should it be it.source(genTask)

[David Herman]

Oh, that's what you mean

[David Herman]

Sorry

[David Herman]

Let me check

[David Herman]

Ah, it's this and not it because we're using the kotlin-dsl plugin. this is Kotlin2JsCompile

[Ting-Yuan Huang]

but isn't the parameter of configureEach it ?

[David Herman]

The type is

void configureEach(Action<? super T> action);

// where...
@HasImplicitReceiver
public interface Action<T> {
    void execute(T t);
}

[David Herman]

where I believe the @HasImplicitReceiver is what results in the value being a "this" and not an "it"

[David Herman]

You can see the "this" annotations in the screenshot here

[David Herman]

(Pretty sure thanks to this: https://kotlinlang.org/docs/sam-with-receiver-plugin.html)

[Ting-Yuan Huang]

I see. Sorry for the silly question.

[David Herman]

No, not silly at all. I had to dig in and check. Sometimes the fix is simple stuff like that so I'm glad you confirmed.

[Ting-Yuan Huang]

Another thing: project.tasks.withType(Kotlin2JsCompile::class.java) also returns ksp tasks. You may need to filter them out: https://github.com/google/ksp/blob/main/gradle-plugin/src/main/kotlin/com/google/devtools/ksp/gradle/KotlinFactories.kt#L242

[David Herman]

Ah so yeah we might be setting source on irrelevant tasks. We'll look into that.

[David Herman]

OK looks like we're working on our end. Setting source was the way to go, and a lot easier than what we thought we had to do. Thank you!

[David Herman]

Sorry to follow up on this, but even though I thought things were working for me, I tried using Kotlin 2.0.0-Beta1 to see if things would work, and our KSP fix is failing with it.

(For a reminder, this thread is about marking generated code as something that should not get processed by KSP)

Code is here for how we're currently doing this (by setting source)

In Kotlin 2.0.0, it looks like this approach is going to stop working. I'm seeing the error Source '...\kobweb\playground\site\build\generated\kobweb\app\src\jvmMain\kotlin\ApisFactoryImpl.kt' does not belong to any module

Have you dealt with this by any chance and already figured out how to resolve it? Does it make sense for KSP to expose an API where we can tell it to exclude processing some code?

[Ting-Yuan Huang]

Is that error message produced by some ksp processors?

I'd also make sure allowSourcesFromOtherPlugins isn't turned on; It causes ksp to get sources directly from compilation tasks, instead of source sets.

[David Herman]

The error message comes from Kotlin

[David Herman]

It doesn't like what we're doing with source apparently

[David Herman]

https://youtrack.jetbrains.com/issue/KT-63753/K2-File-does-not-belong-to-any-module-when-it-is-generated-by-registerJavaGeneratingTask-in-AGP looks relevant

[David Herman]

Sorry to revive a three month old thread, but there's an update to my comment just above here.

Basically, the IJ folks fixed the bug by making sure they connected sources to the target sourceset. However, that means, we're once again going to be in a situation where we're generating code that we don't want ksp to process.

I'm wondering if this means it should fall on KSP again for the API to allow us to indicate some way that we have sources that we don't want KSP to process.

[David Herman]

If KSP doesn't provide a way for us to do this, we think we're going to have to jump through hoops in our own code to create a custom sourceset that should be skipped by KSP but compiled in the final product.

Initial experiments are not working however. So far we either stumble back into the "does not belong to any module" error or a circular dependency error.

[Ting-Yuan Huang]

@David Herman What kind of APIs are you thinking about? If it is something like this, it'd require each project that applies KSP to specify.

[Ting-Yuan Huang]

or you'll have to make sure KSP is applied before your plugin, and configure it before ksp use it.

[Ting-Yuan Huang]

Also what do you mean by "not process it"? 1) they are not handed to processors, but sources that are processed can still reference to them, or 2) they are completely out of KSP's reach.

[David Herman]

Let me get concrete.

I have a project with a bunch of Kotlin code in it, some of the files including methods annotated with the @Page annotation. We do not actually generate source in our KSP processor, but we generate resources (json files describing important entries in the code). I'll attach an example json file to this comment.

Next, our Gradle plugin discovers these json files in the project, using them to generate code. Let's call this task "generateSite"

In our mind, this would be ideal:

• Run KSP processor, generate resources
• Run "generateSite", which processes the resources and generates code
However, KSP's current design (as far as I can tell) wants all source to be ready before it runs. It either wants the source to be there or, if generated, is generated by it itself.

We get an error when we try to add the output of the "generateSite" task to the Kotlin sourceset:

kotlin.srcDir(generateSite)

Specifically, Gradle reports a circular dependency between kspKotlinJs and generateSite.

For now, based on our discussion from a few months ago, we are doing this as a workaround:

kotlinCompileTask.configure { source(generateSite) }

But that option will no longer be available to use in K2.

frontend.json

[Ting-Yuan Huang]

Do you have a solution in mind already? As suggested in my previous message, my intuition is to have some way (API) to tell KSP what directories to skip, like KspExtension.excludeSrcDirs.
Another approach is to call KSP2 programmatically and skip KSP's gradle plugin completely, now that you're manipulating sources directly.

[David Herman]

We are planning to look into KSP2. I can tell you more after we have some time to do that.

[David Herman]

I guess the solution I had in mind is what KSP used to have before, with the generatedByKsp sourceset (which I assume was removed for some technical reason?)

[Ting-Yuan Huang]

Do you mean, your plugin added generated sources into generatedByKsp to avoid the circular dependency? If so, would doing the same by generating into $buildDir/generated/ksp/$target/$sourceSetName work for you?

We moved away from generatedByKsp because Kotlin Gradle Plugin is deprecating the API that registers new source sets to KotlinCompilation. After discussing with them, their recommendation is adding generated sources directly to the default source set, hence the current KSP implementation.

[David Herman]

That's an interesting idea but we generally assign our tasks a unique output folder because I think that's Gradle-idiomatic / cache friendly. Let me double check with my friend who is more of a Gradle expert than I am.

[David Herman]

My friend said they think "excludeSrcDirs" as proposed could possibly work, but would possibly have to be carefully designed to appease gradle.

[David Herman]

@Ting-Yuan Huang check out https://github.com/bitspittle/issue-ksp-gradle-cycle

This is a minimal repro case of the problem we're seeing.

I think there should be enough information in the README to make sense of it, but you can of course ping me here if you have any questions.