Hi all, we have a multi-module library where Dokka...

# dokka

[Ian Botsford]

Hi all, we have a multi-module library where Dokka's failing to resolve inter-module links which I filed as dokka#2679. We may be doing something wrong in our configuration...anyone have any experience with getting multi-module projects to work with Dokka?

[Ignat Beresnev]

Hi! I believe @Vadim Mishenev is having a closer look at the issue

[Ian Botsford]

Oh that's great news, thanks! Well I'll be curious to hear about any findings or if any more data from our end would be helpful.

[Ian Botsford]

Later of course, I see it's late for you both!

[Ignat Beresnev]

Hi! JFYI it looks like we now understand what's happening, and there is a workaround for 1.7.10, but it won't work for the upcoming 1.7.20. Vadim is investigating it further to understand why it's happening and how we can fix it

Are you blocked by this in any way? If you are, we can share a temporary workaround

[Ian Botsford]

Yes, our published documentation (example) contains broken references because of the inter-module linking failures. Sharing a temporary workaround would be very helpful.

[Ignat Beresnev]

Will post it to the the issue in case anyone else encounters it

[Vadim Mishenev]

Unfortunately, it is not a trivial workaround:
1. Add kotlin.mpp.enableCompatibilityMetadataVariant=true
2. Build the entire project before a Dokka run
3. Run Dokka

[Ian Botsford]

That doesn't seem too difficult. I'll try it out shortly and let you know what I find. Thanks! 🙌

[Ian Botsford]

It looks like that worked, thank you both for the workaround! 😄

[Ignat Beresnev]

Thanks for reporting it! Somehow we didn't catch it in any other library, although I don't think your setup is very exotic. We'll keep you posted on the proper fix

[Ian Botsford]

Perhaps I spoke too soon!

The workaround lets the code/docs build successfully on my machine but causes builds to fail on the CodeBuild instances we use for CI. We see new warnings about implicit dependencies between Gradle tasks followed by compilation errors for references to types/packages defined in those dependencies:

Untitled.txt

[Ian Botsford]

Is there anything else that might need to be done after adding the kotlin.mpp.enableCompatibilityMetadataVariant=true flag to avoid implicit dependencies between Gradle tasks?

[Ian Botsford]

Sorry, never mind. Upon further investigation it appears one of our dependencies is pulling in the Java Gradle plugin, which creates a jar task whose output conflicts with Kotlin's metadataJar task. We're looking into working around or removing that usage of the Java plugin since it's unnecessary for us.

[Ignat Beresnev]

Hi! We've spoken to folks from the KMP team and we think the problem might be related to Kotlin not having proper support for multiplatform projects with a single target, see KT-52664

That would also explain why it works flawlessly for other multiplatform multi-module projects such as ktor, serialization and others

Does that sound reasonable and align with what you're seeing with other amazon libraries? Is smithy the only library with this problem or have there surfaced more?

[Ian Botsford]

Yes, that sounds plausible. I can understand how our setup might be unusual. Our smithy-kotlin and aws-sdk-kotlin KMP projects only target JVM for now because we intend to release support for JVM first and then fast-follow for other targets. I don't have a lot of other Amazon KMP libraries for comparison—most that I'm aware of are non-KMP projects for the JVM.

Does the KMP team intend to prioritize multiplatform projects with a single target? Or will that be an unsupported (or less supported) configuration?

[Ignat Beresnev]

I believe @Anton Lakotka [JB] from the KMP team wanted to comment on that, but probably forgot 🙂

[Ian Botsford]

We see that Kotlin 1.8.0 is released and we're excited to use it but our upgrade is blocked on this Dokka/KMP issue. @Ignat Beresnev and/or @Anton Lakotka [JB], any update on this issue?

[Ignat Beresnev]

Hi! I'm trying to push this issue internally, but as it's primarily an issue of Kotlin itself, I keep forgetting about it for long periods of time. As far as I understand, it's being considered

If you have the means, perhaps you could add some weight to it as well 🙂 https://youtrack.jetbrains.com/issue/KT-52664/Multiplatform-projects-with-a-single-target

[Ignat Beresnev]

There might be other workarounds, but they won't be pretty - would you settle for that in the meantime?

[Ian Botsford]

I'm definitely interested in at least understanding what the workaround options may be.

[Ian Botsford]

Then we can evaluate if the pain is worth it! 😅

[Anton Lakotka [JB]]

Hey Ian, we can discuss options next week. I'll reach you once I have some ideas.
Do I understand right, that your main priority is to make sure that JVM-specific API is not leaking to commonMain ? And you want to let IDE highlight all such declarations before compilation?

[Ian Botsford]

Yes that and ensuring that we're set up to add additional platforms in the future. We fully intend our library to support Native and JS in the future but we're building/releasing only JVM for now.

[Anton Lakotka [JB]]

Okay, then I already got some ideas on how can we do that. I'll share details next week then.

[Ian Botsford]

Perfect, thanks for your help!

[Ian Botsford]

And as Ignat suggests, I'll go ahead and add a comment to KT-52664 in the off-chance that helps with visibility and/or prioritization! 😁

[Anton Lakotka [JB]]

Hey! Ian,

Can you try doing this:

val isIdeaActive: Boolean = System.getProperty("idea.active") == "true"
    if (isIdeaActive) {
        kotlin.linuxX64()
    } else {
        afterEvaluate {
            val commonMetadata = tasks.named("compileCommonMainKotlinMetadata") {
                enabled = true
            }
        }
    }

It should set up linuxX64 target to trick IDEA that commonMain is truly common.
But on CLI side (when idea is not active), just enable compileCommonMainKotlinMetadata task that will ensure that your commonMain is truly common.

You can also get rid of kotlin.mpp.enableCompatibilityMetadataVariant=true flag in such case.

Let me know if it helps for you.

[Ignat Beresnev]

Hi, @Ian Botsford! Did you have a chance to try it out? If it's something you want to try at all 🙂

[Ian Botsford]

We tried that out but it seemed to have no effect. If I understand correctly, that would alter the KMP configuration in IDEA but we don't run our builds in the IDE...we run them from the command-line.

We upgraded to Kotlin 1.8.0 and stayed on Dokka 1.7.10 and our builds seem to work.

[Matas Lauzadis]

Hi everyone, I work with Ian 👋. I took a deeper look and narrowed down our doc generation issue to this specific PR which was introduced in Dokka 1.7.20. Manually reverting this change lets our doc generation succeed again.

Using the kotlin.mpp.enableGranularSourceSetsMetadata property that's checked in the PR gives a warning because hierarchical structures support is enabled by default since (I believe) Kotlin 1.6.20. Should this logic be checking the kotlin.mpp.hierarchicalStructureSupport property instead?
If the PR is still correct as-is, do you have any recommendations on what we should modify to get this working again?

[Ignat Beresnev]

Hi! Thanks for the investigation, this is of huge help!

The original issue that the PR fixed was that some libraries (ktor in particular) were failing with errors like

The following variants were also considered but didn't match the requested attributes:
          - Variant 'iosArm32MetadataElements-published' capability net.mamoe.yamlkt:yamlkt:0.12.0 declares a library, as well as attribute 'org.jetbrains.kotlin.platform.type' with value 'native':
              - Incompatible because this component declares a usage of 'kotlin-metadata' of a component and the consumer needed a usage of 'kotlin-api' of a component
              - Other compatible attribute:

We consulted the KMP team about what could be done, and the changes introduced in the PR are the result of the discussion. Perhaps, we need to revisit it - I've written it down, we'll have a look

Maybe @Anton Lakotka [JB] has some other ideas about what could be done in the meantime. Alternatively, if you want, I can publish a custom build of Dokka 1.8.10 (once it's released) with the change reverted, but you'll need to add our dev maven repository to use it. This would unblock you and give us more time to investigate and find a proper solution

[Matas Lauzadis]

Thanks for the quick reply! As Ian mentioned, we are not currently blocked because we can just keep Dokka at 1.7.10 until the issue is resolved. As long as it's on your backlog for discussion, I think that's a good solution for now. If publishing the custom build of Dokka 1.8.10 is not too much of a headache, that would be great too,.

[Ignat Beresnev]

Hi once again 🙂

I've reverted the problematic commit in a separate branch, which is based on 1.8.10 - that's literally the only difference with the release version

And I've published a dev version for that branch: 1.8.10-dev-211 . You can find it in the following repository:

maven("<https://maven.pkg.jetbrains.space/kotlin/p/dokka/dev>")

If you want/can try it out - let us know how it goes, it'd be helpful

[Matas Lauzadis]

Thanks! I'll try it out and let you know how it goes

[Ian Botsford]

I realize that we never followed up on this. We've tried the 1.8.10-2679-aws-fix branch you provided and it allows us to successfully generate docs with our existing build configuration.

[Ian Botsford]

What're the next steps then? Can this be PRed and merged into a release so that we can consume it?

[Ian Botsford]

Hi @Ignat Beresnev, any update here?