hey, I'm having some issues with dokka producting ...

# dokka

[solonovamax]

hey, I'm having some issues with dokka producting very inconsistent behaviour.

on my machine, dokka is producing the correct output. previously, on CI dokka was also producing the correct output.
however for some reason, a recent change has caused dokka to no longer produce the output on CI, but it's still producing the correct output on my machine.

here is an example of the expected output: https://ci.solo-studios.ca/job/solo-studios/job/kt-fuzzy/job/master/13/javadoc/kt-string-similarity/index.html
here is an example of the incorrect output produced by CI: https://ci.solo-studios.ca/job/solo-studios/job/kt-fuzzy/job/master/14/javadoc/kt-string-similarity/index.html

the commit in question which caused it was this: https://github.com/solo-studios/kt-fuzzy/commit/65587a03e1d38012fea9477bd0764a843c70dd54

however, I don't see how that affected it, so I'm just going to assume that it triggered some kind of funky thing to happen with the cache?

I've got absolutely no clue why it's doing this.

[Adam Semenenko]

I checked out the project locally and ran ./gradlew :dokkaGenerate.

The attached screenshot shows the 'incorrect' output, right?

My guess is an input isn't declared correctly, so locally Dokka is picking up a stale file. Could you try running ./gradlew clean :dokkaGenerate and then check?

[Adam Semenenko]

I think the problem is task dependencies aren't configured correctly, so Gradle doesn't run the processDokkaIncludes task.

In buildSrc/src/main/kotlin/kt-fuzzy.dokka.gradle.kts, make this change:

dokka {
    moduleName = nyx.info.name
    moduleVersion = nyx.info.version
    dokkaSourceSets.configureEach {
        // hardcoding the directory means Gradle won't run `processDokkaIncludes` task
        //includes.from(dokkaDirs.includesOutput.asFileTree.files)
        // Instead, use the task reference so Gradle automatically runs `processDokkaIncludes`
        includes.from(files(processDokkaIncludes).asFileTree.files)

[solonovamax]

My guess is an input isn't declared correctly, so locally Dokka is picking up a stale file. Could you try running ./gradlew clean :dokkaGenerate and then check?

afaik Jenkins will clean the directory every single run, however it could be getting cached in the gradle home directory

[solonovamax]

also, even though it doesn't reference processDokkaIncludes, it does later declare the dependency:

tasks {
    withType<DokkaGenerateTask>().configureEach {
        inputs.files(dokkaDirs.stylesOutput, dokkaDirs.assets, dokkaDirs.templates, dokkaDirs.scripts)

        dependsOn(compileDokkaSass, processDokkaIncludes)
    }
}

so it should still work?

[solonovamax]

The attached screenshot shows the 'incorrect' output, right?

and yeah, that's the incorrect output

[solonovamax]

if I run dokkaGeneratePublicationHtml --dry-run, then it shows that the processDokkaIncludes task would be run. further, if I use gradle task tree, it shows that it is definitely a dependency of dokkaGeneratePublicationHtml .
but I can try to use what you said instead to see if it'll fix anything

[solonovamax]

running a clean dokkaGenerate --no-build-cache --no-configuration-cache --rerun-tasks (without the suggested changes), I'm now having a very different funky issue which I thought had gone away, where in the multimodule documentation it's only showing a single module even though it should be showing both of them

[solonovamax]

it seems that if after the clean task, I run dokkaGenerate twice, then it produces the correct outputs. some task dependency here probably isn't getting properly propagated, and I feel like for the multimodule stuff at least that's on dokka's end

[solonovamax]

hmm actually, after running it a second time it's not producing the correct outputs.
if I viewed the html while the task was running, then the correct outputs are present, however once it's finished then it looks like the screenshot I just posted

[solonovamax]

this is what it should look like and is how it looks during the run

[solonovamax]

okay, so this weird behaviour where it's correct during the run but incorrect afterwards only seems to happen if I run dokkaGenerate or dokkaGeneratePublicationHtml, but not if I run :dokkaGenerate or :dokkaGeneratePublicationHtml (which only run the task for the root project and not any of the subprojects), which is... odd.

it also does not happen with dokkaGenerateModuleHtml or :dokkaGenerateModuleHtml.

[solonovamax]

keep in mind, that I'm running all these tasks using --no-build-cache --no-configuration-cache --rerun-tasks

[solonovamax]

okay, something interesting happens if I run clean dokkaGeneratePublicationHtml
for a while there is no html present, which is expected because of the clean task, however a bit before it's done, only the documentation for the kotlin fuzzy module is present.
once the task is fully completed, then it is overwritten by the documentation for the kotlin string similarity module (as shown before).

my theory behind what is happening is the following:
1. when it is run, the kotlin fuzzy module finishes first because it has the least amount of pages & things to document. it gets added on to what was previously completed, if anything exists. so if it was run previously and not `clean`ed, then it will be added on to that. but if there's nothing, it writes a new file.
2. next, when the kotlin string similarity module finishes, it completely overwrites whatever was there, regardless of if it was written in the same run.

[solonovamax]

I still can't reproduce the broken behaviour from Jenkins, though. I'm assuming this somehow has to do with some magic cache gradle has which I'm not disabling...

[Adam Semenenko]

Very strange! I'd suggest removing parts one-by-one to see if anything helps.

I see 'configure on demand' is enabled. I can imagine that's interfering with the configuration. Could you try removing org.gradle.configureondemand=true in gradle.properties and re-running?

[solonovamax]

running with --no-configuration-cache (in addition to the other flags) (which should have the same effect as setting org.gradle.configureondemand=false) produces the same behaviour

[solonovamax]

same happens if I do clean dokkaGeneratePublcationHtml & --no-configuration-cache
it also happens with & without your suggested changes

[Adam Semenenko]

thanks for trying. I've been investigating further, and I can stably reproduce the problem.

1. Run gradle clean
2. Run gradle :dokkaGen - the output is incorrect.
3. Run gradle :dokkaGen again - the output is correct.
When I look at kt-string-similarity/build/tmp/dokkaGenerateModuleHtml/dokka-configuration.json after step 2 I see the includes aren't present, but after step 3 they are.

[Adam Semenenko]

I have an idea, could you try it please? Inside buildSrc/src/main/kotlin/kt-fuzzy.dokka.gradle.kts please make this change:

dokka {
    dokkaSourceSets.configureEach {
        //includes.from(dokkaDirs.includesOutput.asFileTree.files)
        includes.from(dokkaDirs.includesOutput.asFileTree.elements)

asFileTree.files will eagerly evaluate the files, which means no files until after the task has run. asFileTree.elements returns a provider, which is lazily evaluated after the task has run.