https://kotlinlang.org logo
Join Slack
Powered by
When I use tags in IntelliJ by specifying VM optio...
# kotest
k
When I use tags in IntelliJ by specifying VM options: 
-Dkotest.tags="UnitTest"
the report with test results includes test classes that were not tagged. They are listed at the end of report as single line per class, i.e. no specific tests are reported. I tagged test classes, so AFAIU the specs should not even be instantiated (and they aren't when I use tags in Maven). Is this UX bug?
this is how this looks like in the UI, first three lines are tagged tests (allows to expand individual test cases) and the rest are tests tagged with different tag.
Instantiating these extra test classes is very problematic for us because we are trying to separate unit from integration tests and our integration tests are expensive/slow to start.
r
I tag my tests by calling a tag method on them - which means the tag is a property of the instantiated object, so the class has to be instantiated for kotest to know what tags it has. I move my expensive initialisation out of init and into before functions.
(Or make the vals that capture the expensive setup lazy)
s
It's because it has to instantiate the spec to look at the top level tests
Adam do you have an example code of how you add the tags ?
k
Copy code
import io.kotest.core.annotation.Tags

@Tags("UnitTest")
class ReactiveLoggerTest : FunSpec() {

    init {
        test("example usage: Scoped log") {
...
s
Ok, so the reason for the behavior you are seeing is that you may have top level tests that are annotated, so when it doesn't bother with @Tags on specs when using include, only exclude
Because you may do something like this:
Copy code
class ATest : FunSpec() {
    init {
        test("example usage: Scoped log").config(tags = setOf(UnitTest)) {
    }
}
I think what we need is some other annotation like @MustAbsolutelyBeIncludedOrTheWholeSpecIsIgnored
k
I think I misread the docs at https://kotest.io/tags/ I looked at 
kotest.tags=!Linux
and concluded that if the tag doesn't match the class-level tag it won't be instantiated.
sorry about that
s
The docs could easily be misleading
Copy code
If you wish to avoid creating the spec class at all then you can annotate a spec using @Tags(tag1, ...). Any tags added using this annotation apply to all tests in the class, however this will not stop a class from being instantiated unless explicitly excluded.
I definitely think better annotations are needed
k
Also I should read the docs more carefully 😊
s
🙂
k
will try adding !IntegrationTest to my tagging expression
cheers for helping me out!
s
We'll improve it for 4.4 because its not intuitive currently
k
This is not intuitive for me because I came from Java world where every test is a separate method, so you can annotate them and inspect via reflection before instantiating the test. If you would like to keep it simple just say the spec is ALWAYS instantiated because we have to learn all its tags. WARN users to not put any expensive eager operations in init.
s
I actually just re-wrote that ticket (if you read it a couple of mintues ago 😂 )
I don't think we can tell users that a spec will always be instantiated, as many users use tags to avoid this. I think it's better to make @Tags behave like most people expect it to - that is a spec level tag is more definitive than it currently is
k
if I tag my class with A but a test with B my -
Dkotest.tags=B
will not pick up the test. Maybe we simply need two types of tags.
s
I think that's more intutitive though, don't you? If you do
Copy code
@Tagged(A)
class Foo()
And you do kotest.tags=B shouldn't that class be ignored?
I don't know, what you're describing is the current behavior. It seems to raise questions frequently though.
Perhaps @TagForTestsInThisClass and @TagForSpec is what we need (better names of course)
k
this OR always instantiate spec
s
Many users rely on the behavior to not always instantiate spec, that can't change as it would break many existing systems.
We can certainly improve things to be clearer with more explicit options though
(as we're discussing here)
k
I see your point with backward compatibility (or at least a path for existing users). In that case for me @TagForSpec and -Dkotest.spec.tag would work.
s
Right, please feel free to opine on that ticket, the more involvement from users the better
k
👍
4 Views
Open in Slack
PreviousNext
Powered by