https://kotlinlang.org logo
Join Slack
Powered by
Hi, I'm new with kotlin, trying to decide whether ...
# test
m
Hi, I'm new with kotlin, trying to decide whether I should use Junit or kotlinTest for unit testing. I'm not sure if this is the right forum to ask but, any help? 🙂
a
Junit is a good choice by default. It works with Java not only with Kotlin. Now is a good time to start using junit5. I might be wrong but I recall kotlinTest is using junit runner under the hood anyways.
m
Yes, KotlinTest uses the JUnit Jupiter platform. JUnit5 split the testing framework from the testing platform (runner). So many frameworks now use JUnit Jupiter runner, and you can choose to use the JUnit5 testing framework. As to which, it depends on your experience, and how fully Kotlin you want to go. Do you want/care about property tests? What IDE and build tool do you use? JUnit5 is a bit better supported across various tools, but KotlinTest is reasonable as they’ve created an IntelliJ plugin. As far as features, JUnit5 and AssertJ is a very powerful combination. KotlinTest is also very powerful, and adds built-in property based testing.
If you’re already familiar with JUnit5, and learning Kotlin, I’d suggest sticking to the test framework you already know, and focus on learning Kotlin. A bit less to learn all at once.
m
thank you for the response! I'm using android studio and gradle. I have a little bit of experience with junit but very little.
m
Are you working with others or alone? Do you collaborate with others in user groups or anything like that? Just thinking from a support perspective if/when you run into problems. I’ve used JUnit5 extensively with Kotlin, and only done a little with KotlinTest. Overall, they’re more alike then they are different. KotlinTest is nice in that it’s more code, and less annotations. It also supports different testing styles, so if you have done JS, and like the format of it’s test tools, KotlinTest supports that. JUnit5 supports nesting of tests, and using the back-tick, you can use descriptive names for test methods, so they end up being similar. I’m not in the Android dev world, so I don’t have any sense of which way, if any, the community is going.
m
I work in a team 🙂
thanks a lot!
s
JUnit is fine but isn't as powerful as KotlinTest. If you're just starting a project you can use KotlinTest in the same basic manner as JUnut but then expand as your project expands.
m
@sam I’m curious. What is the main feature from KotlinTest that JUnit5 doesn’t have? Property based testing is the only one I’m really aware of. If you’re talking JUnit4, then I’d totally agree.
s
Property based testing yes. Table driven testing. A much more flexible way of structuring tests. Native support for coroutines. The best assertion library (subjective of course). Helpers like eventually and inspectors.
m
You’re obviously thinking of JUnit4 for some of this as JUnit5 has an extremely powerful Data driven testing approach now, and Nested Classes for flexible structuring. I always use AssertJ, which is better than KotlinTest’s assertions generally. But don’t get me wrong. I do agree that KotlinTest is the most compelling Kotlin focused testing framework out there, and if you need to test some Kotlin specific features, it can’t be beat. And for a new project, I’d lean slightly to KotlinTest unless you’re already quite familiar with JUnit5. I did experience some issues with KotlinTest, but not insurmountable. IntelliJ plugin doesn’t seem to support running a single test from a class. Using the 
f
and 
x
worked most of the time, but I’d occasionally get complaints about duplicate classes from compiler, and only option seemed to be a clean.
s
I agree with you that KT is the best testing framework on Kotlin and that JUnit is still very good. You're going to be happy with either choice really aren't you. I'm not thinking of JUnit4, but I didn't know 5 had data driven testing. Is that @Dynamic tests? The nested tests is good but it's still quite ugly compared to what KT can do, although it's hardly a major problem of course. I am intrigued by you saying that AssertJ is better than KT's assertions because I believe that KT has more assertions, they're more easily discovered due to being extension functions so the compiler can help you, and support some kotlin only things like Pair. Can you tell me why you think AssertJ is better?
m
For JUnit5 : https://junit.org/junit5/docs/current/user-guide/#writing-tests-parameterized-tests I’m not 100% sure where one would use Dynamic Tests, but it does look interested. For AssertJ, I’ve always felt that it is larger than KT’s list of assertions. For AssertJ, once you have the 
assertThat
, the IDE gives great help. KT has some helpers that AssertJ doesn’t have because of it’s Kotlin focus (Arrow being one), but AssertJ seems to me that it has deeper, and more powerful, assertions. Collections, extracting properties from collections and comparing based on that. Comparing POJOs recursively, ignoring certain fields, and other controls. Comparing files with very helpful messages. JDK8 Date asserters, although I think KT has this too. Soft Assertions. i.e. checking a number of assertions and reporting on all of them rather than first one that fails. AssertJ is a very deep library and keeps growing as it has a large community around it. And I do like KotlinTest’s extension functions. That’s why for my project, I started creating an extension for AssertJ so that one could write 
actualString shouldBe "Expected Value"
and such. Obviously takes some time, but often one only uses a small subset of assertions, and one can always fall back on direct 
assertThat
calls.
For parameterized testing, I tended to use a TestData data class and an ExpectedData data class as input with an ArgumentsSource class. Provided the strongest typing, but if the test data was simple enough, the others are very useful.
s
This is great stuff thanks. Let me have a read through it all and come back to you.
I personally find that @parameterizedtest syntax quite ugly but there's no argument - JUnit definitely has data/table driven testing. So I stand corrected there.
One set of assertions that KT doesn't have is around reflection - which is what the property extraction stuff is all about right - for when you don't know the type but you want to say field "x" should be equal to value "y". For completion they should be added to kotlintest. https://github.com/kotlintest/kotlintest/issues/822
I think the date time support is the same. Kotlintest has soft assertions too 
assertSoftly { }
. File comparisons too. Overall, yeah assertJ is very nice, a bit better in places and a bit weaker in others.
So as I said earlier, you can't really go wrong with either. I'd (obviously) go for KotlinTest, I think the syntax is nicer. But JUnit is without doubt still more popular even on Kotlin projects, so I guess there's more work to be done to make a compelling case to switch.
💯 1
The selling points of KotlinTest are definitely the better syntax and options for test layout, native coroutine support, and I very much like it's batteries included.
The selling points of JUnit are it's ubiquity and thus integration with every tool out there.
m
Agree on the syntax, but given it’s target is Java, annotations are all the rage 😉 I was glad those tools are as powerful as they are as I needed/wanted to minimize the learning curve for the teams I was working with, and try to keep it to Kotlin that they were learning as much as possible. And I’d add property based testing to your list for KT, although personally, I haven’t taken advantage of that much in my code.
It’s so ubiquitous that KotlinTest uses the platform as it’s runner. Gives it good tool support for very minimal cost. As I said, very impressed with what the JUnit team did with Jupiter. Lots of great decisions there.
s
Jupiter is the actual implemntation
Platform is the "platform"
So KT uses junit5-platform, and jupiter is the testing framework you're using.
Now no arguments, platform is great, but this discussion I think was around jupiter vs kt
m
I think Jupiter is what they call the project. Platform is effectively the Runner and API is the test framework. But I’d have to review the docs again to confirm.
And yes, the discussion was around the testing framework.
s
JUnit 5 = JUnit Platform + JUnit Jupiter
from their docs
Also the dependencies are named that way - for example KT depends on platform, but not jupiter.
m
Yep. Just saw that too 😉 So Jupiter vs KT just as you said.
s
Interestingly, there is work underway to move away from JUnit platform for KT
m
Any ideas what they’re hoping to achieve, or what’s lacking in Junit Platform?
s
Gradle makes tons of assumptions about how junit works - such as all tests being methods in a class.
So in order to get non-jupiter style tests to work properly, you have to fudge it
The open issues on gradles github are 18-24 months old, with little progress.
Also for support for things like KotlinJS it will need to work in a different way than than gradle's junit plugin allows
m
Yes, of course. MPP changes a lot of things. For File comparsions/assertions, I was also referring to content comparisons. It’s a rare case, but I was working on some generators, knew the expected output, and wanted to ensure the generator was doing the expected thing. https://joel-costigliola.github.io/assertj/core-8/api/index.html They also have one where it will read the file, and you can use the usual String comparisons, but as they warn, it can’t be too large a file because it gets loaded to memory. I added a link to the more powerful recursive comparison of objects that AssertJ has to your ticket on KT.
s
Do you have an example of the "nice file messages" that you mentioned earlier. That sounded interesting.
m
I can generate one by breaking one of my tests… Just give me a couple minutes
s
haha
m
@sam In your first response, you said: "but then expand as your project expands." did you mean that as my project expands I can add Junit and use both KT and Junit? is it possible to use both of them?
and if it is, would it be wise or I better stick to one framework?
s
I meant as your project became more complicated you could use more complicated features. I think you'd be happy with either.
m
You’ll be happy with either, you can combine them, but I’d recommend picking one and sticking with it as you don’t have much experience with either. Given you’re coding in Kotlin on Android, I suspect the extra features of KotlinTest will be valuable to you, so would lean to recommending that. And for mocking, I’d recommend Mockk. It’s becoming more well known. Pivotal has added Mockk into their Kotlin focused tutorials. So I think it’s here for the long haul.
d
In my experience JUnit 5 on Android is not yet well supported. That means that you are going to be swimming against the tide using anything other than JUnit 4 there.
👍🏻 1
m
thank you all it was very helpful 🙂
5 Views
Open in Slack
PreviousNext
Powered by