https://kotlinlang.org logo
#realm
Title
# realm
p

Paolo Rotolo

03/10/2023, 2:47 PM
Hi all 👋 We noticed that
default.realm
reached 17 GB in one of our apps, even if we don't store a lot of data (there are very frequent writes, but then entities are deleted when in sync with server). We noticed a similar problem on github, so we upgraded realm version to latest one. Now we're tracking the
numberOfActiveVersions
on each write, and we see that the number increases each time a new write is performed (1 write for second). It should be related to this issue: https://github.com/realm/realm-kotlin/issues/1081 We're still performing more tests with latest realm
1.6.1
but, will this issue of always growing realm versions affect database size on disk? According to github, https://github.com/realm/realm-core/pull/5440 (Fsa/new version management) was merged in latest release, is there a way we can mitigate active versions growing with this new version management? Thanks in advance.
c

chrmelchior

03/12/2023, 3:43 PM
Yes, a growing number of active versions will affect the file size. Basically Realm needs to track all changes between the “oldest” and “newest” version, which is what the active version number represents. That space is then reclaimed when the Realm is closed and can reused by future writes. But you won’t see the filesize go down unless you compact the Realm. Before 1.6.0, we needed to track all intermediate versions as well, i.e. if you had version 1 and version 10 active it would need to track everything in between. But with 1.6.0 we have optimised the space needed for that by quite a lot. We do cleanup of versions based on the GC in Kotlin, so depending on when that triggers you might accumulate quite a lot of versions. That don’t need to be a problem by itself , since since each version shouldn’t add much, but it should cleanup at periodic intervals. If you enable debug logs you should see the version number increase for each write and then suddenly you will see a lot of
Closing intermediate version
in the log. The easiest way to get into this situation is if you query for an object on startup and then store it in a static variable. Then you will pin the database to that version preventing it from releasing all the intermediate versions. If you have objects that are long-lived you can use
copyFromRealm
. Then the data is copied into memory and the database is free to progress from there. I hope that helps?
p

Paolo Rotolo

03/12/2023, 4:03 PM
Thanks for the accurate answer, I still can’t figure out why version number increases each time I perform a write even if I’m detaching all the objects queried using copyFromRealm. The only cases where I don’t use copyFromRealm are Flows, but I guess this is equivalent, right?
Copy code
realm.query<Data>("id = $id")
  .asFlow()
  .map { it.list.firstOrNull()?.asMyObject }
I was also experimenting using
copyToRealm
after a write, but still the number increases
Copy code
realm.write {
    copyToRealm(data, updatePolicy = UpdatePolicy.ALL )
}.copyFromRealm()
c

chrmelchior

03/12/2023, 4:10 PM
The number will increase even if you use
copyFromRealm
because the Realm itself is moved to that version. Anecdotically I have often seen 50+ versions before the GC kicks in
p

Paolo Rotolo

03/12/2023, 4:12 PM
I've reached 3000 versions in one hour, because our usecase is a background service that needs to write almost every second 😕
c

chrmelchior

03/12/2023, 4:12 PM
Note, we are tracking various improvements to it right now. We have some work in progress for https://github.com/realm/realm-kotlin/issues/1234 and we also have https://github.com/realm/realm-kotlin/issues/709 for more manual control (but we would like to be in a situation where that isn’t needed in everything but the hottest path)
Hmm, that sounds like something is being pinned somewhere then
Unfortunately tracking that down isn’t the easiest thing 😕
p

Paolo Rotolo

03/12/2023, 4:14 PM
is it fine if realm itself is a singleton injected with DI?
c

chrmelchior

03/12/2023, 4:14 PM
Doing stuff like:
Copy code
realm.query<Data>("id = $id")
  .asFlow()
  .map { it.list.firstOrNull()?.asMyObject }
Should be fine, it will send a version down the flow and then that version will be removed by the GC once the
map
operation is done with it
Are you perhaps storing RealmObjects in Compose?
Yes, Realm Kotlin is designed to work as a singleton and all the API’s should be threadsafe
p

Paolo Rotolo

03/12/2023, 4:20 PM
no, I'm not using compose. I'm going to check more carefully if all the objects are detached, but I'm pretty sure we're copying and mapping entities from realm to other objects as soon as the query is performed. Anyway thanks a lot for the tips, I'll at least check if with latest realm version the file size doesn't grow so fast. I'll keep an eye on github if you need help testing future improvements in real life scenarios.
@chrmelchior Quick question, what happens if I collect a flow like this and use the collected object without mapping?
Copy code
realm.query<Data>("id = $id")
  .asFlow()
  .map { it.list.firstOrNull() }
Will it cause version pinning, so I should be doing this instead?
Copy code
realm.query<Data>("id = $id")
  .asFlow()
  .map { it.list.copyFromRealm().firstOrNull() }
c

chrmelchior

03/13/2023, 11:02 AM
Yes, the first code could cause version pinning depending on what you do with it
The 2nd version will not
54 Views