Hello ! :wave: Has anyone been working with the new async sqlite web worker for the browser? I am tr...

# squarelibraries

[Carlos]

Hello ! 👋 Has anyone been working with the new async sqlite web worker for the browser?
I am trying to implement an OPFS sqlite database in the browser inspired on this https://github.com/dellisd/sqldelight-sqlite-wasm/blob/main/src/jsMain/resources/sqlite.worker.js
When I run any transaction like:

val players = listOf<Player>()
database.playerQueries.transaction {
  players.forEach { player ->
    database.playerQueries.insert(
      player_number = player.number,
      full_name = player.fullName
    )
  }
}

The sqlite.worker.js will receive a postMessage call for each insert, but I'm only able to receive around 5-10 inserts per second when running a transaction. I have a use case to insert around 100,000 entries of 0.5kB max each, which would take hours at this speed.
Has anyone experienced something similar or any ideas what might be going wrong? I wonder if postMessage calls are expensive on the browser.

[Derek Ellis]

There's an open feature request (and an old draft PR) to introduce batching support to SQLDelight which would help a lot in a case like this (sending one big batched set of execute statements instead of thousands of individual ones)

[Derek Ellis]

For now though, in my own project I worked around this by customizing the web worker script to do the batch insert directly in the worker

[Derek Ellis]

https://github.com/dellisd/reroute/blob/trunk/web/src/jsMain/resources/worker.js#L63-L86

[Carlos]

Hi Derek ! Thank you for the work done so far

Nice, I was thinking I can customize the web worker to do your own batching too or maybe also web worker pooling. Thank you for your example ! will check it out

[Carlos]

I considered whether the postMessage async call could be accelerated, but finding ways to enhance its speed appears to be more complex.