https://kotlinlang.org logo
Title
m

Malachi de Ælfweald

01/06/2020, 7:08 PM
Hi @marcoferrer - We are auto-generating the classes with the mock interfaces, which are then being used for testing. The mock classes are put into the same source directory as the regular auto-generated classes, thus they get archived together into the library. I am thinking I need to break them up into two separate classifiers, as Google is now reporting
Security Alert: Your app contains embedded private keys or keystore files
due to kroto-plus-test depending on grpc-testing and ending up in the releaseRuntimeClasspath. Any thoughts on how to get the mock classes to exist only in the test classpath?
in the meantime, I’m going to try to get around it by setting transitive to false for the kroto-plus-test dependency.
m

marcoferrer

01/06/2020, 11:04 PM
are you generating the mock classes in the main source set?
I unusually separate the test code generation in to the test source set so it doesnt pollute everything else
m

Malachi de Ælfweald

01/07/2020, 3:40 PM
That’s what I would like to do. I didn’t see any easy way of doing that. I tried doing two separate kroto yml files (one with mock services and one without) but that wasn’t quite right.
m

marcoferrer

01/07/2020, 3:42 PM
I can get an example together but will need about an hour. I’m on the road at the moment.
The fix requires two kroto config and individual configuration of the main and test source set in the protobuf Gradle plugin
m

Malachi de Ælfweald

01/07/2020, 3:46 PM
sounds good. I’ll try again while you’re driving. This is an android project btw. This is what I have at the moment:
protobuf {
    protoc { artifact = Libs.protoc }
    plugins {
        Libs.protocPlugins.forEach {
            id(it.key) { artifact = it.value }
        }
    }
    generateProtoTasks {
        val krotoConfig = file("$projectDir/kroto.yml")
        all().forEach {
            // Allow config changes to affect UP-TO-DATE checks
            it.inputs.files(krotoConfig)
            it.generateDescriptorSet = true
            it.descriptorSetOptions.apply {
                includeImports = true
                includeSourceInfo = true
            }
            it.builtins {
                id("java") {
                    option("annotate_code")
                    outputSubDir = "java"
                }
            }
            it.plugins {
                id("grpc") {
                    outputSubDir = "java"
                }
                id("coroutines") {
                    outputSubDir = "java"
                }
                id("kroto") {
                    option("ConfigPath=$krotoConfig")
                    outputSubDir = "java"
                }
            }
        }
    }
}
If I make a copy of the config; remove the mock services from the main one and change that so it looks like:
id("kroto") {
                    when {
                        it.isTest -> option("ConfigPath=$krotoTestConfig")
                        else -> option("ConfigPath=$krotoConfig")
                    }
                    outputSubDir = "java"
                }
then the
build/generated/source/debug/java
and
build/generated/source/release/java
are populated (without the mock services) and the
build/generated/source/debugUnitTest/java
and
build/generated/source/releaseUnitTest/java
are both empty.
m

marcoferrer

01/07/2020, 4:24 PM
protobuf {
    protoc { artifact = Libs.protoc }
    plugins {
        Libs.protocPlugins.forEach {
            id(it.key) { artifact = it.value }
        }
    }
    generateProtoTasks {
        val krotoConfigMain = file("$projectDir/kroto_main.yml")
        val krotoConfigTest = file("$projectDir/kroto_test.yml")

        // For android projects use `ofNonTest()` 
        // For plain java / kotlin projects use `ofSourceSet("main")`
        ofNonTest().forEach {
            // Allow config changes to affect UP-TO-DATE checks
            it.inputs.files(krotoConfigMain)
            it.generateDescriptorSet = true
            it.descriptorSetOptions.apply {
                includeImports = true
                includeSourceInfo = true
            }
            it.builtins {
                id("java") {
                    option("annotate_code")
                    outputSubDir = "java"
                }
            }
            it.plugins {
                id("grpc") {
                    outputSubDir = "java"
                }
                id("coroutines") {
                    outputSubDir = "java"
                }
                id("kroto") {
                    option("ConfigPath=$krotoConfigMain")
                    outputSubDir = "java"
                }
            }
        }


        // For android projects use `ofTest()` 
        // For plain java / kotlin projects use `ofSourceSet("test")`
        ofTest().forEach {
            // Allow config changes to affect UP-TO-DATE checks
            it.inputs.files(krotoConfigTest)
            it.generateDescriptorSet = true
            it.builtIns {
                remove("java")    
            }
            it.plugins {
                id("kroto") {
                    option("ConfigPath=$krotoConfigTest")
                }
            }
        }
    }
}
Give this a try and let me know
And if you want to avoid setting up two kroto configs you can try the new gradle config plugin.
Its experimental but stable
m

Malachi de Ælfweald

01/07/2020, 4:34 PM
hmm, using the config above - the unit tests paths still don’t generate any files
not even the descriptor files
‘debug’ and ‘release’ generate files. ‘debugUnitTest’ and ‘releaseUnitTest’ are empty.
let me check something
ok I have something
If I put a dummy proto in src/test/proto
then it generates the mock services
for that proto
it’s just not generating anything for the ones it inherits
IE
the ones it reads from debug/release
so it’s treating ‘debugUnitTest’ as just ‘unitTest’
m

marcoferrer

01/07/2020, 4:39 PM
ok so i think i know whats happening
gimme a sec I think I have a fix. you have to make your protobufTest configuration extend from compile. Im not 100% sure what the names are in a android project
android {
  sourceSets {
    test {
      proto {
        srcDir 'src/main/proto'
      }
    }
  }
}
Do you have an other proto dependencies? like
google-common-proto
m

Malachi de Ælfweald

01/07/2020, 4:43 PM
all but one are coming from nexus
m

marcoferrer

01/07/2020, 4:44 PM
Im not sure if the dependencies are being inherited as well but if not you can use this
dependencies {   
    protobufTest "com.fkfkfk.kddkkkg"
}
m

Malachi de Ælfweald

01/07/2020, 4:45 PM
gotcha… trying to get the test source set to accept proto… hold on
is
proto
above supposed to be a ProtobufConfigurator?
m

marcoferrer

01/07/2020, 4:49 PM
Its a new sourceset that gets added by the protobuf plugin https://github.com/google/protobuf-gradle-plugin#customizing-source-directories
m

Malachi de Ælfweald

01/07/2020, 4:50 PM
trying to convert it. that format isn’t valid anymore
sourceSets {
        getByName("test").java.srcDirs("src/main/proto")
    }
You have to use a closure.
Theres the source from when I implemented the kotlin dsl support for gradle protobuf
Wait not 100% sure it neeeeeds to be a closure
lol
m

Malachi de Ælfweald

01/07/2020, 4:56 PM
😉
well, good news is that it mostly generated stuff this time
looks like debug has protos, but not the mock services (good)
and debugUnitTest has both protos and mock services (also good)
trying to determine what the current failures are
but I think this gets me where I needed to go
thank you for all your help
oh - before I forget
a suggestion for you for all that free time you have
m

marcoferrer

01/07/2020, 4:59 PM
No worries! Anytime really. I know what a pain the protobuf plugin dance can be
m

Malachi de Ælfweald

01/07/2020, 5:00 PM
it would be nice to see a dokka configuration that displays the auto-generated Kotlin code nicely (like with suggested dsls)
m

marcoferrer

01/07/2020, 5:00 PM
OoOOOooOoOoOoooOoOoooo
m

Malachi de Ælfweald

01/07/2020, 5:00 PM
devs are digging through code to figure out how to use it
m

marcoferrer

01/07/2020, 5:00 PM
Thats a great idea
Can you file an issue in github? You dont have to go into detail but I would like to keep track of it there while I did into it.
m

Malachi de Ælfweald

01/07/2020, 5:01 PM
thanks again for your help. I’ve got some tests to tweak and dependencies to swap around now 🙂
sure
m

marcoferrer

01/07/2020, 5:02 PM
Awesome glad its all working for you now. Thanks again!
m

Malachi de Ælfweald

01/07/2020, 5:03 PM