Does connection itself implies something? We have connection to HTTP server, and still can process its request(s) asynchronously.

It does on all database servers I've used. The database server executes all connections concurrently. It doesn't internally queue work based on server resources available. If it did the performance difference between large and small connection pools wouldn't make much difference - it would just move the with queue from the client to the server.

Connections are very heavyweight in general. e.g. in Oracle each connection spawns a new process on the server (https://docs.oracle.com/cd/E11882_01/server.112/e40540/memory.htm#CNCPT1226). Or in Postgres, prepared statement caching only happens per connection.

I'm not saying this is a great design. However, until the database server has lightweight connections and an async protocol I'm not sure we gain much by trying to force one on top (gain over the thread pool design).

^^^ +1. You spend A LOT of resources for each connection on your typical DB server. I do remember that we used to have very up-scale Oracle instance which was used by a cluster of around 20 app servers each keeping 100 connections (for a total a couple of thouthand) and that was the max Oracle server was able to handle (memory-wise). So you just configure your conn pool for 100 connections and you can just as well have a thread pool of 100 threads, too. No need to have them fewer or more.

A traditional DB (like Oracle) has quite a traditional design inside. Each connection == a separate thread plus of lots of internal data structures dedicated to that specific thread. Those threads do share data page caches, etc usually though locking. They don't usually known anything else about each other. Each one will read DB meta from data page caches, fetch the data, etc

It is like almost "share nothing" as each thread/connection is separate with an exception for a shared data page cache. Sometimes queries can be parallelized but that is an "advanced" feature.

Yep. I've had similar experiences and have had a hard time convincing people to reduce pools to increase performance. 😝

So, if you are tasked with writing a mixed app where there is some access to a classic DB and some micro-services to use, then you have hard time sizing threads and pools. What I'd do with kotlin coroutines in a such a case is to have a DB thread pool with a size equal to the number of DB connections and a separate "work pool" (~= # of CPU). I would dispatch all my coroutines to work pool and make sure they suspend while waiting for my microserices (use async APIs!) and use

run(dbContext) { ... }

for my DB queries and let them block my DB pool, but have them free my work pool for other stuff needing execution.

Sounds workable to me. The only outstanding issue for me is transactions, but I guess you could make multiple calls in the

run

block and take care not interleave blocking calls to other resources.

An added bonus is that you could ditch the connection pool altogether and just use

ThreadLocal

connections. Although given the performance of HikariCP it is probably not worth the effort.

Hi, guys. What libraries are you using for db migration? I’ve tried doing some Python/Django and it seems to me that db migration is a “must”. If there isn’t, I’m gonna use Python/Django just to maintain the db schema in my Java/Kotlin backend project. 😂

@dathoang.se https://flywaydb.org works well for us

Thank you K

All hail Kotlin!

liquibase not bad either

My 50cents about async and db. Connection pools and executing queries in separate connections can help only if you are not aware of transactionality. In apps which mostly reads from database and don't aware on showing not latest available data - it's ok. But when you rely on using transactions - you have to work with single db-connection and call every query in it/synchronously. AFAIK DBMS developers trying to fix it, but there is no standard (such as jdbc) and every vendor implements it in "own unique way". I.e. mysql team create X Plugin, which allows to fetch results concurrently within one connection (https://dzone.com/articles/asynchronous-query-execution-with-mysql-57-x-plugi). I think something similar may exists in other mainstream DBMS.

I think the question was not about serializing database access (which is application choice of isolation it needs), not about transactions, but about not waiting for DB results to come and instead get them into continuation like async I/O and such.

On Reactive Spring side we have decided to just support NoSql async drivers for now because we don't want to build a Reactive API on top of a thread pool. That's why Spring Data Reactive supports "only" MongoDB, Cassandra and Redis for now. JDBC is blocking by design and async JDBC won't be there until Java 10, but maybe native driver will evolve to something we could leverage. Fyi Reactor Core 3.1 will introduce support for Reactive Context which could be used to support transactions at driver or application level.

Nicely written article on Kotlin improvements in compatibility w/ Java EE: https://www.zeljko.link/2017/04/kotlin-java-ee-plugins.html?1

I'm trying to imagine a universe where a company uses JEE and Kotlin 😉

it's not hard. i'm building REST based systems in kotlin right now.

I was more thinking companies green lighting Kotlin wouldn't be using JEE servers - more the Spring Boot/Dropwizard end of the spectrum. Although I've been out of the JEE universe for a while so maybe all the profiles have slimmed it down

Based on the recent discussions around bridging JDBC with coroutines, I've done a small POC here: https://github.com/andrewoma/coroutine-jdbc

It seems to "work" but its my first play with coroutines so there may be gaping holes

The Spring blog has had a few Kotlin interop articles... If you have a legacy code base, it's hard to drop everything and switch to microservices so being polyglot is the next best thing..

Round 14 of TechEmpower's Web Framework benchmark (https://goo.gl/pTgx1U) was just released. Hexagon (https://goo.gl/6JKwqj) a Kotlin Web framework I develop did good... check it out 😉

Yeah, congrats @jaguililla 🙂 We submitted #ktor as well, but for some weird reason all stats are just zero.

But it’s our first round, so pretty sure we will catch up 😉

@jmfayard has left the channel