Hi all :wave: We noticed that `default.realm` rea...

# realm

[Paolo Rotolo]

Hi all 👋
We noticed that default.realm reached 17 GB in one of our apps, even if we don't store a lot of data (there are very frequent writes, but then entities are deleted when in sync with server). We noticed a similar problem on github, so we upgraded realm version to latest one.
Now we're tracking the numberOfActiveVersions on each write, and we see that the number increases each time a new write is performed (1 write for second). It should be related to this issue: https://github.com/realm/realm-kotlin/issues/1081

We're still performing more tests with latest realm 1.6.1 but, will this issue of always growing realm versions affect database size on disk?
According to github, https://github.com/realm/realm-core/pull/5440 (Fsa/new version management) was merged in latest release, is there a way we can mitigate active versions growing with this new version management?
Thanks in advance.

[chrmelchior]

Yes, a growing number of active versions will affect the file size. Basically Realm needs to track all changes between the “oldest” and “newest” version, which is what the active version number represents.

That space is then reclaimed when the Realm is closed and can reused by future writes. But you won’t see the filesize go down unless you compact the Realm.

Before 1.6.0, we needed to track all intermediate versions as well, i.e. if you had version 1 and version 10 active it would need to track everything in between. But with 1.6.0 we have optimised the space needed for that by quite a lot.

We do cleanup of versions based on the GC in Kotlin, so depending on when that triggers you might accumulate quite a lot of versions. That don’t need to be a problem by itself , since since each version shouldn’t add much, but it should cleanup at periodic intervals.

If you enable debug logs you should see the version number increase for each write and then suddenly you will see a lot of Closing intermediate version in the log.

The easiest way to get into this situation is if you query for an object on startup and then store it in a static variable. Then you will pin the database to that version preventing it from releasing all the intermediate versions. If you have objects that are long-lived you can use copyFromRealm . Then the data is copied into memory and the database is free to progress from there.

I hope that helps?

[Paolo Rotolo]

Thanks for the accurate answer, I still can’t figure out why version number increases each time I perform a write even if I’m detaching all the objects queried using copyFromRealm.
The only cases where I don’t use copyFromRealm are Flows, but I guess this is equivalent, right?

realm.query<Data>("id = $id")
  .asFlow()
  .map { it.list.firstOrNull()?.asMyObject }

[Paolo Rotolo]

I was also experimenting using copyToRealm after a write, but still the number increases

realm.write {
    copyToRealm(data, updatePolicy = UpdatePolicy.ALL )
}.copyFromRealm()

[chrmelchior]

The number will increase even if you use copyFromRealm because the Realm itself is moved to that version.

Anecdotically I have often seen 50+ versions before the GC kicks in

[Paolo Rotolo]

I've reached 3000 versions in one hour, because our usecase is a background service that needs to write almost every second 😕

[chrmelchior]

Note, we are tracking various improvements to it right now. We have some work in progress for https://github.com/realm/realm-kotlin/issues/1234 and we also have https://github.com/realm/realm-kotlin/issues/709 for more manual control (but we would like to be in a situation where that isn’t needed in everything but the hottest path)

[chrmelchior]

Hmm, that sounds like something is being pinned somewhere then

[chrmelchior]

Unfortunately tracking that down isn’t the easiest thing 😕

[Paolo Rotolo]

is it fine if realm itself is a singleton injected with DI?

[chrmelchior]

Doing stuff like:

realm.query<Data>("id = $id")
  .asFlow()
  .map { it.list.firstOrNull()?.asMyObject }

Should be fine, it will send a version down the flow and then that version will be removed by the GC once the map operation is done with it

[chrmelchior]

Are you perhaps storing RealmObjects in Compose?

[chrmelchior]

Yes, Realm Kotlin is designed to work as a singleton and all the API’s should be threadsafe

[Paolo Rotolo]

no, I'm not using compose. I'm going to check more carefully if all the objects are detached, but I'm pretty sure we're copying and mapping entities from realm to other objects as soon as the query is performed.
Anyway thanks a lot for the tips, I'll at least check if with latest realm version the file size doesn't grow so fast. I'll keep an eye on github if you need help testing future improvements in real life scenarios.

[Paolo Rotolo]

@chrmelchior Quick question, what happens if I collect a flow like this and use the collected object without mapping?

realm.query<Data>("id = $id")
  .asFlow()
  .map { it.list.firstOrNull() }

Will it cause version pinning, so I should be doing this instead?

realm.query<Data>("id = $id")
  .asFlow()
  .map { it.list.copyFromRealm().firstOrNull() }

[chrmelchior]

Yes, the first code could cause version pinning depending on what you do with it

[chrmelchior]

The 2nd version will not