If anybody was looking for an async postgres libra...

# server

[dave08]

If anybody was looking for an async postgres library, I just found this: https://github.com/vietj/reactive-pg-client (J. Viet is the head of the Vert.x team), I just didn't try it yet, but it looks promising... if it's based on the Vert.x version it looks stable (I have production code on the Vert.x version). It even has Kotlin docs https://www.julienviet.com/reactive-pg-client/guide/kotlin/index.html

[orangy]

It has lots of dependencies on vert.x as far as I can see, which is perfectly understandable, but isn’t it kinda make it too complex to use it outside of vert.x?

[dave08]

I know the Vert.x version needs to have a Vertx instance passed to it, this lib looks more independent (although you're right, I just looked at the pom file and found all those vertx dependencies...).

What are your concerns about being more complex?

[cedric]

I don’t really understand the point of specialized async access to db (whether the one you just listed or the new JDK effort). It’s trivial to take the current sync API and make it async with a lot of options and variety (Rx, coroutines, etc…)

[orangy]

It is not. First you need to have async connection pool that is not blocking a thread. Then you need to send request to SQL server and suspend (not block) until it responds. Then, ideally, you would receive rows for the result set in an async way, like in a channel or so, and do not block thread and not fill in your RAM with all the received raws before returning it to client.

[orangy]

Most existing pseudo-async solutions just use secondary thread pool for sync JDBC connections.

[dave08]

And all that is covered by the Vert.x client...

[dave08]

I'd suppose this one too... I would just be very interested to know if I could use Ktor with it? Could there be some pitfalls involved?

[fred.deschenes]

well you can't use any ORM on top of it since it doesn't use JDBC

[cedric]

I see

[fred.deschenes]

and from what I remember it boots a vertx server which isn't exactly lightweight

[cedric]

So the real work would be at the JDBC driver level, right?

[dave08]

This perticular library? @fred.deschenes

[cedric]

(and database itself too obviously)

[cedric]

not so much at the app level?

[fred.deschenes]

well any "async" SQL library (this one included)

[dave08]

I mean booting full vertx? I don't think it would deploy verticles behind the scenes or anything..

[fred.deschenes]

honestly until the new async JDBC (whatever they called it, can't remember) I don't think it's worth it trying to work with those libraries (although they are a good way to figure out what to do and not to do in the real async implementations that will come eventually)

[dave08]

@cedric Node.js works like this too, the "event loop" of the app keeps on receiving requests and running queries, and gets responses and sends them in callbacks, effectively not blocking any threads and being able to process many more requests w/ less resources...

[dave08]

@fred.deschenes It all depends what you're trying to accomplish..

[orangy]

We are preparing to do it for #squash as well. @Deactivated User got a prototype for MySQL, and we have a branch with suspending function, but no connection pool yet.

[cedric]

Ok… squash?

[dave08]

Is there some kind of idea of a release date @orangy?

[cedric]

Ah https://github.com/orangy/squash

[Deactivated User]

My fork (with the suspending stuff):
https://github.com/soywiz/squash

And the MySQL CIO client (along other clients like redis):
https://github.com/soywiz/ktor-cio-clients

Still WIP, but somehow works. Also those are temporal repos until merged. But I put in the case you are curious about it.

[dave08]

Very interesting... 😉

[dave08]

That could put it ahead of #exposed ...

[orangy]

suspending stuff minus other related work is in branch for squash already, but I’m not yet sure how to promote it to master. It will require after user to wrap in runBlocking or something similar, unless they use #ktor already of course 🙂

[orangy]

As for release date, I can’t say anything. This is side project for me, I can’t put a lot of effort there. I think when @Deactivated User can migrate all his korio stuff to coroutine libraries we are building for IO & network (channels, etc) we can consider making it default for squash and change it to be the async lib.

[Deactivated User]

Nice. I think it is almost done modulo bugs and remaining cleanups (did some additional work last friday night by reducing GC and added some protocol-level redis tests).

Of course it will require stress tests and some time of heavy use to determine that everything is working fine. But at least it is a start 🙂 and brings some of the benefits of Node.JS clients.

I will check later the status of those repos, clean up the things that are already migrated to coroutines library, and see if I can publish cio clients to bintray in the meantime so the squash fork including the mysql pure async client can work.

We have to see where to put those cio clients once they are done. They depend on ktor-network for raw sockets, that only depends on kotlinx.coroutines. So they should be pretty stand alone.

[orangy]

We will be shuffling stuff between kotlinx.coroutines, kotlinx.io, ktor to extract useful and self-contained libs for building other stuff in the Kotlin ecosystem.

[dave08]

Sounds great!

[gregopet]

I tried using the impossibl Postgres driver (https://github.com/impossibl/pgjdbc-ng)

[gregopet]

Supposedly it is built on much better principles than the standard PG driver

[gregopet]

It turned out however that the standard PG driver is so well fine-tuned that it still beats Impossibl at performance

[gregopet]

On top of that, the standard driver's bugs are well understood by any software you're using to access the database (jOOQ in my case), while alt drivers are not

[gregopet]

At the time, the jOOQ author advised me to just wait and stick with the standard driver for the time being

[orangy]

Wait for sql2

Yes, that’s what I see all the time when I ask about async sql