I’m working on the Dokka Gradle Plugin refactor, a...

# dokka

[Adam S]

I’m working on the Dokka Gradle Plugin refactor, and I’ve got safe cross-project configuration working, except it looks like some of the HTML processing isn’t working.

In the HTML files I can see PathToRootSubstitutionCommand comments, but the ### substitution isn’t triggered

<!--[+]cmd:{"@class":"org.jetbrains.dokka.base.templating.PathToRootSubstitutionCommand","pattern":"###","default":""}-->    <link href="###images/logo-icon.svg" rel="icon" type="image/svg">

I also see in the Dokka logs that some of the ‘extension points’ aren’t working

Unused extension points found: ExtensionPoint: org.jetbrains.dokka.base.DokkaBase/externalClasslikesTranslator, ExtensionPoint: org.jetbrains.dokka.base.DokkaBase/externalDocumentablesProvider, ExtensionPoint: org.jetbrains.dokka.base.DokkaBase/immediateHtmlCommandConsumer

I’ll put the full JSON config that’s being used to trigger the Dokka Generator in the thread - is it missing something?

[Adam S]

this is the config for a project with two subprojects

dokka_conf.json

[Adam S]

.
├── my-project/
│   ├── subproject-hello/
│   │   └── build.gradle.kts
│   └── subproject-goodbye/
│       └── build.gradle.kts
├── setttings.gradle.kts
└── build.gradle.kts

[Adam S]

these are the files generated in the root project (there’s no styles dir…)

[Adam S]

and this is the classpath I’m using to run the Dokka Generator

dokka-core-1.7.20.jar
markdown-jvm-0.3.1.jar
dokka-analysis-1.7.20.jar
kotlin-analysis-intellij-1.7.20.jar
dokka-base-1.7.20.jar
kotlin-analysis-compiler-1.7.20.jar
kotlinx-html-jvm-0.8.0.jar
freemarker-2.3.31.jar
jackson-dataformat-xml-2.12.7.jar
jackson-module-jaxb-annotations-2.12.7.jar
jackson-databind-2.12.7.jar
jackson-annotations-2.12.7.jar
jackson-core-2.12.7.jar
jackson-module-kotlin-2.12.7.jar
kotlin-reflect-1.7.20.jar
jsoup-1.14.3.jar
kotlinx-coroutines-core-jvm-1.6.3.jar
kotlin-stdlib-jdk8-1.7.20.jar
kotlin-stdlib-jdk7-1.7.20.jar
kotlin-stdlib-1.7.20.jar
kotlin-stdlib-common-1.7.20.jar
woodstox-core-6.2.4.jar
stax2-api-4.2.1.jar
annotations-13.0.jar
jakarta.xml.bind-api-2.3.2.jar
jakarta.activation-api-1.2.1.jar

[Adam S]

the result is that the generated index.html doesn’t have any styling

[Ignat Beresnev]

I’m working on the Dokka Gradle Plugin refactor

Hi! Is it for #2839?

I described in more detail how HTML documentation should be generated for multimodule projects here: https://github.com/Kotlin/dokka/issues/2770#issuecomment-1347673833

If you read it, I'm sure it'll clear some things up.

TLDR: You need to run each module separately with delayTemplateSubstitution = true, and then in the last multi-module task apply the templating and all-modules-page plugins, and pass all modules into the DokkaConfiguration#modules option.

(sorry if you've done that already in the json config, can't go over it at the moment)

[Adam S]

Hi! Is it for #2839?

yes that’s right

[Adam S]

I’ll read through that info, thanks

[Adam S]

I don’t think that way the current plugin works is comparable though, the new way is more like how the CLI works.

[Adam S]

(at least I think)

[Ignat Beresnev]

It's extremely difficult to build multimodule documentation with the CLI runner... it basically requires separate runs for each submodule and then another one for the final result

[Adam S]

is it that difficult? I thought that basically each subproject would have a sourceSet. So there would be a sourceSet for

• src/main/kotlin,
• and subproject-alpha/src/main/kotlin,
• and subproject-beta/src/main/kotlin

[Ignat Beresnev]

I wish 🥲 The comment that I linked above explains it well

[Adam S]

yeah boiiiii styles are working

[Adam S]

I had delayTemplateSubstitution set to true for some reason

[Adam S]

I understand there’s a difference between sourceSets and modules

I assumed that a Gradle project with multiple subprojects could effectively be handled by Dokka Generator with a single dokka-conf.json, with one sourceSet per subproject. Dokka Generator would create a standalone site that contains all sourceSets.

And there’s also Dokka Modules, which I assume are standalone instances of Dokka sites.

[Adam S]

how I’ve set it up is that (assume dokka2 plugin is applied to all subprojects)

• each subproject will produce a dokka-configuration.json, that at minimum contains its own sources, e.g. src/main/kotlin
• subprojects can depend on other subprojects

dependencies {
  dokka(project(":some:subproject"))
}

• by default, dokka-configuration.json files from other subprojects will be merged into this project’s dokka-configuration.json (which was already the case in one of current subprojects tasks)
• and then the Dokka Generator only runs once

[Adam S]

however, subprojects can also depend on a Dokka Module,

dependencies {
  dokkaModule(project(":some:subproject"))
}

this will trigger a Dokka Generator in :some:subproject, and produce a Dokka HTML site in ./some/subproject/build/dokka-output - which the consuming project can then receive and merge into its dokka-configuration.json, and pass that along to Dokka Generator (so Dokka Generator will run twice)

[Ignat Beresnev]

With the current architecture of Dokka, each child module needs to be run independently with its own settings, in which delayTemplateSubstitution is set to true. These runs would use the single-module generator, but delay resolving some links (the commands you saw).

In the end, once all child modules have their "partial" documentation generated (with the commands), Dokka is run once again with the all-modules-page plugin (unites all modules into a single website) and the templating plugin (resolves unresolved links/commands relative to the single resulting documentation)

I'm not sure what you're proposing would work, but I don't have much information to tell right now

If you want, I'd be down to have a call where I can explain the inner workings in more detail, and maybe answer some of your questions. I can do it in text, it'll just take longer, and this part isn't documented anywhere 😞

[Adam S]

okay cool, I think I can do that…

when I figured out I can do is print the DokkaConfigurationImpl JSON from the current plugin, and then make sure the new plugin matches it

[Adam S]

I am right in thinking that even in a Gradle build with a single project, the following would work to create a final Dokka HTML Publication?

1. run ./gradlew dokkaGenerateHtml
2. Get Dokka parameters from build.gradle
3. Create the Dokka JSON parameters (used to execute Dokka Generator) ./build/dokka-config/html/dokka-params.json
4. Produce a Dokka Module into ./build/dokka/html-module/ by executing Dokka Generator using dokka-params.json, BUT without all-modules-page and templating
5. Create the final Dokka HTML Publication into ./build/dokka/html by running Dokka Generator again WITH all-modules-page and templating

[Adam S]

so Dokka Generator would run twice for single projects, but the benefit is given multiple projects it’s easy for step 5 to additionally trigger steps 1-4 in other subprojects, and gather the results