Hi. Do I understand correctly that `selectBatched`...
# exposed
d
Hi. Do I understand correctly that
selectBatched
will split the query across multiple SQL queries? If that's the case, is there a way to use a single SQL query, but retrieve its results in batches? Or is this achievable transparently using the
Iterable.chunked
called on the returned
Query
?
a
Could you describe what your scenario is, please?
chunked
would work, but it will read the entire query result set into memory
d
I am computing MD5 hash of the entire table by first letting the SQL Server compute MD5 for each row using
HASHBYTES
function and then in my kotlin code concatenate those hashes to compute the resulting final hash. I want to conservatively limit the memory usage in case of tables with lots of entries by computing intermediate hashes for each chunk of 10000 results and then compute the resulting hash from that. I found that
Query
has
fetchSize
parameter, but I don't know if that's what I want. This is code I have for now
Copy code
const val TABLE_HASH_BATCH_SIZE = 10_000

suspend inline fun <reified T> T.tableHash(): ByteArray where T : Table, T : Hashable {
    val dbHashes = slice(hash).selectAll().orderBy(hash).fetchSize(TABLE_HASH_BATCH_SIZE)
    val count = dbHashes.count()
    val digester = Digest(hashBytesAlgo.algoName)
    if (count == 0L) return ByteArray(digester.build().size) // bytearray of zeroes with correct size
    return if (count <= TABLE_HASH_BATCH_SIZE) {
        dbHashes.map { it[hash].bytes }.forEach(digester::plusAssign)
        digester.build()
    } else {
        val digesterOuter = Digest(hashBytesAlgo.algoName)
        for (chunk in dbHashes.chunked(TABLE_HASH_BATCH_SIZE)) {
            chunk.map { it[hash].bytes }.forEach(digester::plusAssign)
            digesterOuter.plusAssign(digester.build())
            digester.reset()
        }
        digesterOuter.build()
    }
}