Hello, good morning you guys, I have a small question about testing: What kind of test do you often ...

# test

[Sam]

Hello, good morning you guys, I have a small question about testing:
What kind of test do you often implement in Startup ( Release many features in a short time) ?
Many thanks.

Small tests are unit tests that validate your app's behavior one class at a time.
Medium tests are integration tests that validate either interactions between levels of the stack within a module, or interactions between related modules.
Large tests are end-to-end tests that validate user journeys spanning multiple modules of your app.

[tddmonkey]

This is a really good question, but:
• forget that “one class at a time” thing.
• being in a startup != release many features in a small time and vice versa

[tddmonkey]

It feels like there’s perhaps a deeper question here?

[Sam]

being in a startup != release many features in a small time and vice versa

1. This is my dream, but not true in my area. Haha.
Lots of feature to put in a sprint.

2. OK, let’s me clear about this:
2.1 How much time should automated testing take compared to the development time of the feature?


2.2 What kind of testing should be provided with little time to write?

2.3 What kind of testing will startup companies usually write?

[aandreyev]

1. unit test coverage at least 50%
build fails on CI when unit test coverage decreases
2. integration tests for all services that expose REST API
3. e2e at least for basic scenarios

[tddmonkey]

why 50%? And why doesn’t CI fail for integration/e2e tests?

[aandreyev]

50% is sane number between 100% and 0%

In reality for JVM projects I have never seen more than 80%. It is virtually impossible to reach due to tons of autogenerated code, that has to be excluded from coverage metrics manually (and of course no one usually bothers to do that).

Code coverage significantly below 50% usually means you:
• have a plan to increase test coverage in the nearest future
• have a plan to remove unused code
• have a plan to turn your startup into f**k up

[aandreyev]

why doesn’t CI fail for integration/e2e tests?

[aandreyev]

it does only on test failure

[aandreyev]

in case of poor coverage CI also fails on unit test absense

[christophsturm]

maybe this helps:
https://medium.com/@kentbeck_7670/the-product-development-triathlon-6464e2763c46
https://medium.com/@kentbeck_7670/fast-slow-in-3x-explore-expand-extract-6d4c94a7539

[Mike]

I suspect the answer to the question depends on how big the SW will ultimate get, how long-lived the SW is expected to be, how high a quality you need to make it. These aren’t one-sized fits all answers. Automated tests add cost, and if I am throwing away the code in a few weeks, it isn’t clear that it is worth writing the tests. However, if the code will live and be modified for decades, then tests will be a significant productivity enhancer over the life of the product.

[tddmonkey]

I think it’s even more subtle than that - even if it’s only going to live a few weeks - will writing tests help me deliver it quicker than not?

[Mike]

Sometimes it will. For developers who have made a mental transition to test first, it might take longer asking them to write without tests.