Can I use the ktor client on a background thread (...

# kotlin-native

[Brendan Weinstein]

Can I use the ktor client on a background thread (eg Dispatchers.Default)? I am testing with coroutines 1.3.3-native-mt.

launch {
              val response = withContext(Dispatchers.Default) {
                val client = HttpClient { }
                val emptyBody = object  : OutgoingContent.NoContent() {
                  override val contentLength: Long = 0

                  override fun toString(): String = "EmptyContent"
                }
                client.get<String>(body = emptyBody, port = 0, block = {
                  url {
                    takeFrom("<https://api.basebeta.com>")
                    encodedPath = "/rankings"
                  }
                })
              }
              output.send(Action.RankingsLoadedAction(
                list = emptyList(),
                diffResult = KDiffUtil.calculateDiff(
                  MItemDiffHelper(
                    oldList = emptyList(),
                    newList = emptyList()
                  )
                ),
                fromNetwork = true
              ))
            }

This leads to the below error

kotlin.native.concurrent.InvalidMutabilityException: mutation attempt of frozen <http://io.ktor.utils.io|io.ktor.utils.io>.core.ByteReadPacket@1bb0698

If I follow the stacktrace further down I see references to

io.ktor.client.features.HttpCallValidator.Companion

I've read that we should avoid companion objects being used by multiple threads. My understanding is that companion objects are implemented as singletons in native (static inners on jvm?). My hunch is that the inners of Ktor reference a singleton/companion object initialized on the main thread and that leads to recursive freezing throughout the ktor client object graph.

Is there a strategy to avoid this that still allows me to make requests on a background thread? I've seen code examples that use ktor on the main thread. For my aim I'd like to make a request via ktor on a background thread.

[Aleksey Chugaev]

https://kotlinlang.slack.com/archives/C0A974TJ9/p1577623588061000?thread_ts=1577606563.060500&cid=C0A974TJ9

[Aleksey Chugaev]

My guess is - not yet.
I have tried ktor 1.3.0-rc2 with mt coroutines and still had an exception, something about trying to mutate frozen HttpPipeline. Not sure if the issue is in Ktor or my code.

[basher]

You should file these as GH issues for ktor, so they can be fixed. These kinds of issues are hard to otherwise know about

[Aleksey Chugaev]

I will, just finishing a sample project to reproduce it (to rule out my own code).

[Brendan Weinstein]

https://github.com/ktorio/ktor/issues/1537

[basher]

Oh I see. Yeah with native mt this makes sense. Launching there freezes the lambda and all the state it captures

[basher]

Either the launch or withContext is likely freezing that thing that's then being mutated and causing the exception

[Aleksey Chugaev]

I have submitted an issue which I had https://github.com/ktorio/ktor/issues/1538

[kpgalligan]

There are some other issues too. We’ve found workarounds, but I have to collect thoughts and send them along to either the ktor or mt coroutines discussion areas.

[kpgalligan]

If you use the mt coroutines, you have to be careful mixing background ops and “same thread” ktor calls. The mt coroutines doesn’t freeze results if you’re staying on the same thread. However, once you do one operation that crosses a thread boundary, all same-thread calls after that point will freeze results and break ktor. TL;DR, We wound up creating multiple scopes to mix sqldelight and ktor.

[Brendan Weinstein]

My assumption was that if an object is not referenced by two different threads, objects should not be frozen.

withContext or launch both provide the ability for a coroutine to change thread contexts, so the way that the transfer of immutable objects is enforced is to freeze any object that gets referenced by both the launch and withContext block, even if the code in each lambda is run on the same thread. Is this correct?

fun main() {
  GlobalScope.launch(context = <http://Dispatchers.IO|Dispatchers.IO>, block = {
    for (i in 0 until 1_000) {
      delay(200)
      println(Thread.currentThread().name)
    }
  })

  GlobalScope.launch(context = <http://Dispatchers.IO|Dispatchers.IO>, block = {
    for (i in 0 until 1_000_000) {
      Thread.sleep(10)
      assert("DefaultDispatcher-worker-1".equals(Thread.currentThread().name))
    }
  })

  while(true) {

  }
}

I was gonna say that I found enforcing this way unintuitive because my understanding of coroutines was that a single coroutine can be executed on multiple threads. I thought this because you can specify the context to be a dispatcher backed by a threadpool. And then it'd make sense to freeze an object by adding a field keeping track of the last thread it was accessed by. When that field changes, freeze the object.

Execution of a coroutine can happen on multiple threads, but if I am taking the right lesson from the code sample above, this only happens when a suspending function, async, withContext, or launch call is made within the coroutine.

And then it kind of makes sense that you'd freeze an object when it is referenced by a lambda parameter for any two of those functions.