What is the appropriate way to write tests for an application that builds a command line tool (using...

# announcements

[Justin]

What is the appropriate way to write tests for an application that builds a command line tool (using kotlinx.cli)?

The app itself looks pretty typical, e.g.:

fun main(args: Array<String>) {
   // Do stuff that might exit with status code 0 or 1
}

My (apparently wrong) instinct was to write tests like:

@Test fun testExitsWithoutError() {
   val result = main(args)
   assertEquals(0, result)
}

But main can only return void. How do I write tests that can pass args into main and expect a certain result?

[nanodeath]

I'd move as much as physically possible out of your main method into other classes and test those instead

[nanodeath]

even another class that returns an exit code, and then main just invokes it and checks the return value

[Justin]

Yea that's the direction I was headed. So I guess there's no way to directly test the main fx then?

[nanodeath]

I wouldn't say you "can't" -- you might be able to mockk System.exitProcess or something, but I think it's better design to just create modular stateless classes (when possible) and test those individually, with maybe a couple "integration tests" that do actually invoke main

[James Richardson]

I would just run the program and make assertions that it does what it is supposed to.

[Justin]

That's what I was hoping to do @James Richardson but I wasn't sure how to translate those assertions into runnable kotlin tests

[James Richardson]

You can only make assertions about observable effects. I dont know what the program does, but you can run it by invoking it as a process, and check that it did what it should. If it creates a file, check that the file is there and has the correct content. If it exits with a status, check the status.

[James Richardson]

You can use a processbuilder to invoke the program, as "java <classpath> Main <args>" if you want, or just call the main function, if it doesn't call System.exit. or you can set up a security manager to stop it calling system.exit maybe

[Justin]

@James Richardson

If it exits with a status, check the status.

This is exactly what I am trying to do. I guess my question is, how do I capture the exit status code from a kotlin @Test ?

[nanodeath]

you'd have to shell out with ProcessBuilder and call waitFor (or exitStatus)

[Justin]

ohh you (both) are saying to actually build the CLI and then invoke it as a process from the test

[Justin]

that makes sense

[James Richardson]

yeah - run the program...

[nanodeath]

my opinion is still to move most of the stuff in main() into classes and then unit test those, but it would make sense to have at least one integration test that just runs the thing using ProcessBuilder

[James Richardson]

absolutely...

[James Richardson]

something like:

import com.natpryce.hamkrest.assertion.assertThat
import com.natpryce.hamkrest.equalTo

class MainclassTest {

    @org.junit.Test
    fun `process exits with code 0`() {
        val status = ProcessBuilder(
                System.getProperty("java.home") + "/bin/java",
                "-classpath",
                System.getProperty("java.class.path"),
                MainclassTest::class.qualifiedName + "Kt"
                ).start().waitFor()
        assertThat(status, equalTo(0))
    }
}

fun main() {
    println("Hello")
}

[James Richardson]

you would want to capture the output and display it if the process failed.

[James Richardson]

or assert on it as well, if it was important to you

[James Richardson]

and waitFor with a timeout that made sense for how long the program should take

[Justin]

That was incredibly helpful @James Richardson.

Nothing better than an answer that leaves no more questions to be asked. Thank you!

[James Richardson]

👍 you're welcome!