Hi everyone! We have a modularized architecture an...

# android

[John Whited]

Hi everyone! We have a modularized architecture and are noticing longer and longer build type applying the plugin for Apollo in modules. Since we have over 10K types in our super graph, we don’t generating all types by default and configure our app using the following to re-use fragments between our modules:

val graphqlModule = subprojects.find {
    it.path.endsWith(":graphql")
}?.let {
    it.afterEvaluate {
        apolloModules.forEach { apolloModule ->
            it.extensions.configure<com.android.build.gradle.BaseExtension> {
                dependencies.apply {
                    this.add("apolloUsedCoordinates", (project(apolloModule.path)))
                }
            }
        }
    }
}

Is anyone else seeing longish startup times (5ish mins to detect up-to-date?). We are seeing a lot of time spent examining kotlin types since the above creates a dependency star that keeps growing (it wasn’t long ago we were around ~2 minutes). Is there a better strategy we should follow for configuring fragments? The schema file isn’t changing here but the same amount of time is being taken up. Any help would be greatly appreciated!

[mbonnin]

Apollo maintainer here 👋 A few comments:
• this code has been quite revamped in v4. If you get a chance to try the alphas, I'm curious if it improves things.
• sounds like your startup time is the bad thing? I'm guessing this is the Gradle configuration time? This is a bit surprising cause most of the work should be done at execution time but maybe resolution is happening eagerly. A build scan would be super helpful to investigate this.
• The "apolloUsedCoordinates" is for schema types.This is orthogonal to fragments. If you only want to reuse fragments, you could generate all the schema types in your schema module (alwaysGenerateTypesMatching.set(listOf(".*"))) and skip "apolloUsedCoordinates" altogether

[mbonnin]

(also we have #apollo-kotlin to discuss this kind of things)

[John Whited]

Thanks @mbonnin. We have over 10K types which adds something like 20+ minutes to the build. I wasn’t sure if this was really orthogonal for the re-use case since there seems to be some duplicate type generation check happening. Is v4 considered production ready to adopt?

[mbonnin]

I personally consider v4 ready for production. It's an incremental release over v3, no huge change. It's alpha for API stability reasons, not overall stability

[mbonnin]

Also right, there is also fragment duplicate checks 👍

[mbonnin]

It's the end of day here but if you can share a build scan or reproducer, I'll look into it tomorrow

[John Whited]

@mbonnin I had a chance to migrate off the v3 plugin to v4. I’m having some issues that maybe you can help assist with though. The first is the __Schema file doesn’t contain “all” so my unit tests currently won’t compile. The second thing is I’m not seeing a minimum set of types being auto-generated. I’m not using manual configuration directive anywhere, etc. I doubled checked and my top level schema project does use “isADepedencyOf” of each project consuming the schema and each project is using “dependOn” back to the same parent schema. Everything builds and works minus tests for the aforementioned reason. Any thoughts on what I might be missing?

[John Whited]

I’m using 4.0.0-alpha.3 btw

[John Whited]

nvm on the __Schema.all, I just noticed it moved locations… but I still see more types than necessary being generated

[mbonnin]

Thanks for the follow up! Does v4 help regarding the initial issue of long build times?

[mbonnin]

Regarding the __Schema.all issue, do you mind ellaborating a bit more? Is the issue that it contains too many or too little types?

[John Whited]

Yes, it’s much faster. ~1min to startup a unit test now

[John Whited]

On __Schema.all, I’m looking in the generated types in the build folder. It’s still seems to be generating all the types and then the lint process is just insanely expensive when that’s enabled in your project.

[John Whited]

Also, I was wondering if you really need to do the isADependencyOf when dependent projects are using dependsOn? Shouldn’t there be enough graph information?

[John Whited]

@mbonnin ^

[mbonnin]

Also, I was wondering if you really need to do the isADependencyOf when dependent projects are using dependsOn? Shouldn’t there be enough graph information?

The problem is Gradle project isolation. Projects can't change each other so the link has to be done both ways.

[mbonnin]

It’s still seems to be generating all the types and then the lint process is just insanely expensive when that’s enabled in your project.

It's hard to tell what's causing those types to be generated without more information. The used typed can be from a wide variety of reasons (dataBuilders, input types, etc....). I'm curious why this makes lints so slow though. Most of the used types are very simple classes, shouldn't be too hard to parse...

[John Whited]

@mbonnin In the case of non-isolated projects, does it have the full graph? We don’t have that feature enabled, but we are also trying to improve build times so perhaps we should.

[John Whited]

Just for context, we are around ~120 modules and at 10k types, lint is crazy expensive

[mbonnin]

In the case of non-isolated projects, does it have the full graph?

Can you ellaborate? What should have the full graph?

[mbonnin]

10k types

Sounds doable if every type is a class with a few lines (which it should be). But maybe there's more to it

[John Whited]

We have a schema module that’s the root (standard as per the docs). We have feature modules which include it and set up the graph information as well. The ugly thing I’m seeing is we have to put in isADependencyOf in the top level schema project or dynamically scan for something to help the root schema project understand those inclusions

[John Whited]

I do something like this:

[John Whited]

val apolloModules = rootProject.subprojects.filter {
            try {
                val projectGradleFile = File(it.buildFile.absolutePath)
                val fileText = projectGradleFile.readText(Charsets.UTF_8)
                !it.path.endsWith(":graphql") &&
                    (
                        fileText.contains(""<junk identifying our apollo gql is being used>"") ||
                            fileText.contains("<more junk identifying our apollo gql is being used>")
                        )
            } catch (e: Exception) {
                false
            }
        }

        apolloModules.forEach {
            it?.let {
                isADependencyOf(it)
            }
        }

[mbonnin]

I'm with you here but I don't see another way?

[John Whited]

Is the file system off limits in project isolated?

[mbonnin]

Modifying other projects is off limits. Something like this is not an option:

// feature/build.gradle.kts
schemaProject.dependencies.add("apolloSomething", project)

[mbonnin]

Without this, we're stuck with duplication

[John Whited]

yeah.. I was just thinking maybe the answer is you have to common path write your dependencies out from each project and collect them in a later task

[mbonnin]

This is the closest to cross project configuration but not quite it either https://github.com/gradle/gradle/issues/22514

[mbonnin]

I was just thinking maybe the answer is you have to common path write your dependencies out from each project and collect them in a later task

Gut feeling there is that ultimately, you'll want a project to read that file and have that trigger a task in a different project and you're back to configuring explicit dependencies

[mbonnin]

Don't get me wrong, I wish there was something... Just haven't found it yet

[John Whited]

yeah, and then there’s all those edge cases for syncing it correctly

[mbonnin]

Back to the perf issue, what lint is taking time, I'm guessing :schema:lintDebugAndroid or something like so? In the schema module containing the 10k types?

[John Whited]

yes

[mbonnin]

Might be that lint/kotlin has complexity that grows in O(numberOfFiles) instead of O(numberOfClasses) or something like so

[John Whited]

Can I DM our .graphql schema to you? Just not trying to broadcast our stuff widely 🙂

[mbonnin]

Sure thing!

[mbonnin]

Real life use cases are always welcome