Hello everyone, I am facing one issue related to t...

# android

[Kavita]

Hello everyone,
I am facing one issue related to test cases.
My Android project has more than 250 tests. Most of them include API mocking & some of them are unit tests.

In Android Studio when I run the individual test class, all tests in that class succeed. But when I run the entire test package(altogether), some of the tests fail randomly.

The project has the unit & espresso tests with MockWebSerever & Awaitility to handle API mocking & async operation.
The .gradle file also has Android Orchestrator & clearPackageData declared. All the tests are independent.
Still, the issue persists.

Can someone help me to fix this issue?
Please let me know if you need more details about tests & Android version.

[John O'Reilly]

Are you using Robolectric and, if so, what version?

[John O'Reilly]

Might be shot in the dark but just had same issue earlier and was related to incompatibility between version of Robolectric we had and Java 11 (still no idea why it was working, as in your case, with individual tests but not when run as part of test class/package)

[Kavita]

Hi @John O'Reilly,
No I am not using Robolectric. I am using espresso testing.

[Kavita]

Since I have updated the studio with SDK 8+, started getting this issue.

[Kavita]

My gradle & plugin version:

[John O'Reilly]

What version of Java are you using?

[Kavita]

Java 8

[faogustavo]

Which mocking library are you using? Maybe some mocks are persisting from one test to another and this is interfering in the results

[Kavita]

Hi @faogustavo ,
I am using mockito & mockwebserver for API mocking:

androidTestImplementation 'org.mockito:mockito-android:2.18.3'
implementation 'com.squareup.okhttp3:mockwebserver:3.8.1'

[Kavita]

@faogustavo
Actually every time the same test case doesn't fail. There is no pattern. Sometimes 4/250 test or 8/250 tests are failing.

[faogustavo]

the tests that fail are junit or espresso?

[Kavita]

espresso tests only

[emmano]

I am assuming Awaitility tells Espresso when your app is Idle? Does it use an IdlingResource?

[emmano]

It seems that Awaitility uses timeouts

[emmano]

What library are you using in code for your async operations? RxJava? Coroutines?

[Kavita]

Hi @emmano, Actually I have used Awaitility along with MockWebServer for API test cases, to deal with asynchronous API operations. I am simply using Retrofit but in that flow from ViewModel to Repository I have to deal with different callbacks.
That is why I had used Awaitility.

[Kavita]

I am assuming Awaitility tells Espresso when your app is Idle? Does it use an IdlingResource?

Yeah @emmano Awaitility is used for the same purpose, when u have to deal with asynchronous functionality and want to add some waiting time. As I wanted to avoid usage of SystemClock.sleep(..) in my test cases. It doesn't use IdlingResource.

[emmano]

For your async operations I would try to make sure your tests use a Thread pool that is associated with AsyncTask. This is just to test if the issue has to do with Espresso not knowing when it should do assertions. The AsyncTask Thread pool is automatically monitored by Espresso.

[Kavita]

Sure @emmano, I need to check this. Thanks for the suggestion 👍