I added a massive hack to make it work There is zero chance

Question

I added a massive hack to make it work There is zero chance gradle fix the bugs They simply don t care

Sebastian Schuberth · Answer

I m curious could you point us at the Gradle issues so we could upvote potentially fix them

sam · Answer

There s a bunch of them They ve been around since 2016 etc

sam · Answer

If I have time later I ll try to dish them out

LeoColman · Answer

We even used to have a `gradle bug` label

LeoColman · Answer

To track issues that were out of our control

christophsturm · Answer

do you mean running a single test for example ` tests com mytest TestClass context test`

sam · Answer

yep

christophsturm · Answer

Isn t the root cause of this problem that gradles testing api is so tied to classes and functions

sam · Answer

yes

sam · Answer

Emil and I are looking at writing our own gradle plugin so we can bypass gradle s junit support completely

sam · Answer

Then gradle s bugs can just be avoided

christophsturm · Answer

I have been wanting to add this to failgood for quite some time it seems to me that gradle first sends class selectors for all the classes in a package but then it filters the nodes and does not send all of them back which is bad in my case because failgood already runs the tests when it discovers them

sam · Answer

You just need to get onboard Kotest and help us wink

christophsturm · Answer

if you want my help in throwing out all config switches and 90 of the features then ok

sam · Answer

lol

christophsturm · Answer

does massive hack mean detecting if the suite runs via gradle and then acting different than when it runs in idea

sam · Answer

It means using reflection to delve inside the objects

christophsturm · Answer

phew even more massive

christophsturm · Answer

but to me it now seems that something that i had to add to get jumping to line numbers work in idea breaks single test execution in gradle

christophsturm · Answer

idea wants a file source and gradle wants a class source sadly gradle does not support composite sources so I cannot add both

christophsturm · Answer

that would be an easy thing to fix in gradle probably

sam · Answer

gradle need to re write their test support to properly support all the features of modern test frameworks

christophsturm · Answer

maybe parts of kotest could be the blueprint for that

sam · Answer

even with all of Kotest s bloat and slowness

christophsturm · Answer

but most issues that i discovered could be easily fixed while sticking to junit engine

christophsturm · Answer

from my point of view just fixing bugs without introducing new abstractions could improve gradle for third party test runners immensly

sam · Answer

yes probably

sam · Answer

You could easily tweak Kotest to make it like you want You don t have to use all the features you don t like

christophsturm · Answer

my problem with all the features and config switches is that it makes it harder to eliminate all bugs I probably can t help make kotest bugfree but I think i can make failgood mostly bugfree my software engineering skill is cutting features slightly smiling face

christophsturm · Answer

I want to work on the best test runner for people who want to invest the time to make their test suite really good not help them run their broken test suite

sam · Answer

You ll end up adding more and more features and then you just hit parity with junit and someone else comes along and says I can build a really small test runner and it repeats again slightly smiling face

christophsturm · Answer

for example I know there are some usecases for a beforeEach callback but I m not adding one because it makes people write code that declared dependencies as `var` and init them in the beforeEach callback instead of making a `val`thats initialized at decaration time

sam · Answer

beforeEach is useful for many things Let the users decide if they want to use it or not

christophsturm · Answer

most of the more exotic dsl features of kotest are not needed if you always use instancePerLeaf and I think everybody should always use it

christophsturm · Answer

gt Let the users decide if they want to use it or not thats a great point of view but I think theres an alternative for an opinionated test runner

sam · Answer

Perhaps

christophsturm · Answer

everybody who agrees with my point of view can just use failgood and everybody else can use kotest currently seems to imply everybody except me using kotest slightly smiling face

sam · Answer

lol

christophsturm · Answer

I started it for fun and its still fun and went a bit out of hand

sam · Answer

Well maybe if we succeed in replacing all the gradle junit stuff you can leverage it for your own test runner

christophsturm · Answer

maybe I should just try to fix some gradle test runner bugs I filed so many IDEA bugs related to junit engine that failgood now works pretty well in idea

christophsturm · Answer

maybe we should just write a new build tool

sam · Answer

kbuild

christophsturm · Answer

hmm I think there is just no way to get single test execution working with gradle and failgood this works ``` gradlew failgood test tests failgood internal ContextExecutorTest ``` but when i add a test name to it ``` gradlew failgood test tests failgood internal ContextExecutorTest filtering ``` gradle still only sends a class selector for ContextExecutorTest and then wants to do the filtering after discovery which is too late because at that point failgood is already running all the tests inside ContextExecutorTest

sam · Answer

It should send a ClassMethod post discovery filter

sam · Answer

and if you run tests before post filters then you re out of luck

christophsturm · Answer

hmm yes exactly

christophsturm · Answer

i could read the postDiscoveryFilters via reflection slightly smiling face

sam · Answer

yep

christophsturm · Answer

is that what you use reflection for

sam · Answer

to delve inside the filter and pull out the regex

christophsturm · Answer

I m now going to add issues in the gradle issue tracker for things that I find <https github com gradle gradle issues 20941>

sam · Answer

sam · Answer

sam · Answer

christophsturm · Answer

hehe i had to start somewhere

christophsturm · Answer

also I wanted a really actionable change as a first step

christophsturm · Answer

maybe I m too optimistic but theres no way that my ticket can turn out in so much bike shedding as the other tickets

sam · Answer

You d hope

sam · Answer

Fingers crossed it gets accepted

christophsturm · Answer

I don t even know if idea supports Composite Sources

christophsturm · Answer

did they ever add an integration test that uses kotest like marcphilip said in the ticket

sam · Answer

I m not sure didn t hear anything about it

christophsturm · Answer

I think my change alone is useless there are so many assumptions that gradle does in its test support that are probably workarounds to junit bugs or junit custom runner bugs The correct way would probably really be to add an integration test that uses kotest and one that uses failgood and then fix all issues

Sebastian Schuberth · Answer

gt kbuild I was thinking of kradle grin