https://kotlinlang.org logo
#test
Title
# test
m

Marco Garofalo

03/04/2024, 4:48 PM
Hi, let me borrow your brain for a second here, I am trying to prove the following code is not thread-safe by writing a test for it, make it fail (consistently) and fix the issue, but I am really struggling to make it fail LOL. Here’s the code:
Copy code
package com.example.gameplay

class RotatingSecrets(private val secrets: List<String>) : Secrets {
    private var position = 0

    override fun next(): String {
        return secrets[position++ % secrets.size]
    }
}
This is what I have tried so far:
Copy code
package com.example.gameplay

import io.kotest.matchers.equals.shouldBeEqual
import kotlinx.coroutines.CoroutineStart.LAZY
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.test.runTest
import org.junit.jupiter.api.Test

class RotatingSecretsTests {

    @Test
    fun `is thread safe`() = runTest {
        val concurrentAccesses = 100
        val secrets = (1..concurrentAccesses).map { "secret-$it" }
        val rotatingSecrets = RotatingSecrets(secrets)

        val jobs = secrets.map {
            async (start = LAZY) {
                rotatingSecrets.next()
            }
        }

        val seenSecrets = jobs.awaitAll()
        seenSecrets.sorted() shouldBeEqual secrets.sorted()
    }
}
I have also tried increasing the concurrency up to 10_000, but no luck. Any clever idea?
p

Peter Farlow

03/04/2024, 5:15 PM
Why did you sort seenSecrets and secrets before comparing them? What happens if you compare them unsorted?
m

Marco Garofalo

03/04/2024, 5:17 PM
I am sorting them just because I am interested in the content, not the order, so if for some reason the coroutines result end up being all over the place I want the test to pass.
p

Peter Farlow

03/04/2024, 5:19 PM
Can define exactly what ‘thread-safe’ means in the context of the use case for your RotatingSecrets class?
m

Marco Garofalo

03/04/2024, 5:22 PM
Well that class uses an increment operation, which is by definition not thread-safe, since is like 2 instructions in one, so depending when the threads access the data they could end up reading stale data.
Copy code
var x = 0
val y = x++

=>

val y = x
x = x + 1
That said, I have the suspicion that Kotlin might compile that class in someway that prevents the race condition, since is just a very simple code.
p

Peter Farlow

03/04/2024, 5:24 PM
I understand that, I’m just trying to understand how you think that will apply to your code. What would an incorrect output look like?
you said you aren’t interested in order, so I don’t know what you want the failure case to look like
I don’t see how the content could change in this class
m

Marco Garofalo

03/04/2024, 5:25 PM
Well I am expecting gaps in the result, i.e. “secret-33” repeated twice, meaning that it will miss “secret-100"
p

Peter Farlow

03/04/2024, 5:26 PM
got it. So did you try using a multi-threaded coroutine dispatcher?
m

Marco Garofalo

03/04/2024, 5:26 PM
As with any non thread-safe operation, I am expecting the position variable to have every now and then the wrong value, creating gaps in the result
Yeah right now I am using this:
Copy code
@Test
    fun `is thread safe`() = runTest {
        val concurrentAccesses = 100
        val secrets = (1..concurrentAccesses).map { "secret-$it" }
        val rotatingSecrets = RotatingSecrets(secrets)
        val threads = newFixedThreadPoolContext(10, "rotating-secrets-test")

        val jobs = secrets.map {
            async (start = LAZY, context = threads) {
                delay((100..200L).random())
                println("[${Thread.currentThread().name}] running")
                rotatingSecrets.next()
            }
        }

        val seenSecrets = jobs.awaitAll()
        seenSecrets.sorted() shouldBeEqual secrets.sorted()
    }
I have also added some arbitrary delay, but still works fine 🤣
e

ephemient

03/04/2024, 6:15 PM
runTest
is not helping you here at all - its goal is to make tests fast and repeatable. might as well dump it
very nice 1
m

Marco Garofalo

03/04/2024, 7:29 PM
Thanks @ephemient that worked like a charm!
Copy code
class RotatingSecretsTests {

    private val rotatingSecrets = RotatingSecrets(listOf("first", "second"))

    @Operation
    fun nextSecret() = rotatingSecrets.next()

    @Test
    fun `is thread safe`() = StressOptions()
        .threads(2)
        .iterations(10)
        .check(this::class)
}
2 Views