Hello :wave: :gradle: :mage: I got a proto monorep...

# gradle

[Dariusz Kuc]

Hello 👋 G 🧙
I got a proto monorepo that contains a number of different service definitions from which I generate stubs using protobuf plugin. While this setup currently works fine, as the size of the repo grows so does the resulting JAR (currently over 10MB(!)). In addition to the single JAR I'd also like to start providing smaller JARs based on their packages.

e.g. I can dynamically create submodules as

// settings.gradle.kts
File(rootDir, "src/main/proto").listFiles()?.filter { directory ->
    directory.isDirectory && java.nio.file.Files.walk(directory.toPath())
        .filter { it.toString().endsWith(".proto") }
        .findAny()
        .isPresent
}?.forEach { service ->
    include(":services:${service.name}-model")
}

And then in the build file

configure(subprojects.filter { it.name != "services" }) {
    val service = this.name
    tasks {
        val serviceJar by registering(Jar::class) {
            from(rootProject.sourceSets["main"].output) {
                include("$service/**")
            }
        }
        val sourcesJar by registering(Jar::class) {
            archiveClassifier.set("sources")
            from(rootProject.sourceSets.main.get().allSource) {
                include("$service/**")
            }
        }
        publishing {
            publications {
                create<MavenPublication>("mavenJava") {
                    artifact(serviceJar)
                    artifact(sourcesJar)
                }
            }
        }
    }
}

Problems arise if I want to verify whether those smaller repackaged JARs are valid, i.e. proto definitions can reference some files from other (common) packages. I'm thinking that I could store those dependencies in build configuration, e.g. just a map of package to list of dependent packages which could then be used to populate POM dependencies.

The problem I'm trying to figure out id how to validate the resulting JARs at build time (whether it specifies all the necessary dependencies) - any ideas how to re-trigger compilation of the specific sources?

[tapchicoma]

probably you could create custom verification task and use finalizedBy

[Dariusz Kuc]

yeah but just wondering what should go into that verification task

[Dariusz Kuc]

hmm I'm thinking if I run the compile tasks only from submodules instead of the root that "should" work

[tapchicoma]

what about each module will have such task?

[tapchicoma]

or verification could not be done in such case?

[Dariusz Kuc]

yeah so i'm thinking if I run compile tasks (Kotlin+Java) to only include package $service from the above then it will fail if I am missing some inter module dependencies

[Dariusz Kuc]

if it fails -> fix the dependency map so it adds missing module as subproject dependency

[Dariusz Kuc]

if all compiles it means each jar has correct dependencies set up

[Dariusz Kuc]

once everything is compiled then i can reassemble the fat jar containing all definitions

[Dariusz Kuc]

basically flip from compiling root and splitting up the compiled output + sources to individual jars ----> TO compile individual packages and then combine all their the outputs + sources to generate single JAR

[tapchicoma]

also a way