Hello, So, with a fully concurrent service, I am ...

# coroutines

[Hugo Costa]

Hello,

So, with a fully concurrent service, I am now basically ddos'ing the services I connect with, more specifically, AWS. Some of my operations require a call to the database, which on large sets, are all done at once. After checking a few options, I chose to go with Resilience4j as it offers a Kotlin implementation, meaning my code is now something like

private val executeStatementRateLimiter = RateLimiter.of(
            "ExecuteStatement",
            RateLimiterConfig.custom()
                .limitForPeriod(25)
                .limitRefreshPeriod(1.seconds.toJavaDuration())
                .timeoutDuration(MAX_WAIT_TIME.toJavaDuration())
                .build(),
        )

private suspend fun executeQuery(
        query: String,
    ): String {
        return try {
            executeStatementRateLimiter.executeSuspendFunction {
                redshiftClient.executeStatement(
                    ExecuteStatementRequest {
                        clusterIdentifier = ID
                        database = DB
                        sql = query
                    },
                ).id ?: error("Received null statement id from RedshiftDataClient")
            }
        } catch (e: RedshiftDataException) {
            log.error(e)
            error("There was an error querying the cluster: $e")
        }
    }

the executeSuspendFunction being a suspended function on its own that only executes when it manages to get a token. I have tried reducing the limit below the API ones, and even knowing I am the only one using the Redshift API at that time (I own the cluster), I'm still hitting the limits and getting throttled/my requests eventually even get rejected.

My code is all using the same instance of a Redshift accessor, meaning it should all be hitting the same rate limiter and thus this shouldn't happen. Could there be any issue with the concurrency? Locally, within my unit tests, I can observe that by decreasing the limit, the unit tests take longer to run (as expected - since a smaller amount can run). Currently, I'm solving the issue my artificially limiting the amount of concurrent requests that can happen. Is there anything else you'd advise me to try?