Hi there, I’m using SQLDelight + SQLCipher in a KM...

# touchlab-tools

[Damiano Giusti]

Hi there, I’m using SQLDelight + SQLCipher in a KMP module and I’m switching away from the cocoapods integration. I’m integrating sqlcipher using cinterops and the sqlcipher amalgamation. This KMP module is not using sqldelight but is using sqliter since it’s a module shared by other KMP libraries to handle the database migration from plain to encrypted.
I wanted to add some tests to the migration and I started to face some issues.

I started from the assumption that I should avoid to link sqlite because otherwise the host iOS app will use the system embedded sqlite library instead of the provided sqlcipher implementation. So I don’t specify any -lsqlite3 flag neither in the .def file nor in the build.gradle file. Unfortunately, when I run the tests on the iOS simulator I get a bunch of errors that likely indicate that I didn’t link sqlite3

e: /Applications/Xcode-15.4.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld invocation reported errors
The /Applications/Xcode-15.4.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld command returned non-zero exit code: 1.
output:
Undefined symbols for architecture arm64:
  "_sqlite3_bind_blob", referenced from:
      _co_touchlab_sqliter_sqlite3_sqlite3_bind_blob_wrapper69 in test.kexe.o
  "_sqlite3_bind_double", referenced from:
      _co_touchlab_sqliter_sqlite3_sqlite3_bind_double_wrapper71 in test.kexe.o

....

On the other hand, if I link sqlite3 in the def file I’m able to run the test but I get the following error

error rawExecSql: ATTACH DATABASE '/Users/damianogiusti/Library/Developer/CoreSimulator/Devices/3511AD2C-AAC6-4392-B854-166C3737D57A/data/Library/Application Support/databases/encryptedDatabase.db' AS encryptedDb KEY 'mostSecurePasswordEver', file is not a database | error code SQLITE_NOTADB

the database file exists on disk but if I open it with an editor before running the ATTACH command looks like it is not encrypted.
Also, running PRAGMA cipher_version returns nothing. I think that sqlcipher is not recognized at runtime because I linked sqlite3, thus the system embedded sqlite.
I had a look also at this github issue but it didn’t help: https://github.com/sqldelight/sqldelight/issues/1442

Anyone can help? thank you! 🙏

[kpgalligan]

Unfortunately, when I run the tests on the iOS simulator I get a bunch of errors that likely indicate that I didn’t link sqlite3

Those errors are definitely liker errors saying it can't find a sqlite implementation (system or sqlcipher).

On the other hand, if I link sqlite3 in the def file I’m able to run the test but I get the following error

the database file exists on disk but if I open it with an editor before running the ATTACH command looks like it is not encrypted.

sqlcipher on iOS is tricky. There are several things you need to do, IIRC. Obviously you need the sqlcipher version of sqlite. Then, avoid linking the system build. There is also some flag in Xcode that needs to be set (not the linker flag). The flag might only be needed if you build sqlcipher in Xcode, vs however you're including sqlcipher.

Unfortunately, when I run the tests on the iOS simulator I get a bunch of errors that likely indicate that I didn’t link sqlite3

Are you running xctests in Xcode, or are these the Kotlin tests that get run from the Kotlin build? Kotlin tests do run on a simulator, but you don't really "see" it happen, so when somebody mentions simulator sometimes they mean they're running xctests from Xcode and using the Kotlin, which is a set of potentially different issues.

[kpgalligan]

On a recent client build, our Kotlin tests were run with clear sqlite. The sqlcipher build itself was done in Xcode when building the app. It just built the custom sqlite.c file as part of its build. However, it's easy to change the config unintentionally and suddenly the app would be using the system sqlite without the dev knowing it. So, I added code to check if the db was actually encrypted before letting the regular db init code run. Sqlcipher on Android is simpler to manage for a number of reasons. Anyway...

[kpgalligan]

For tests, unless you're testing that files are actually encrypted, which I don't think would be very useful, I'd just link to the system sqlite. When building the app in Xcode, I'd link sqlcipher there.

[kpgalligan]

The sqlite link for tests would be gradle config that adds the linker flag to the test compilation only.

[Damiano Giusti]

Are you running xctests in Xcode, or are these the Kotlin tests that get run from the Kotlin build?

I’m running kotlin tests under the commonTest directory targeting iosSimulatorArm64…

So basically you’re suggesting to use plain sql for my kmp module, but link sqlcipher directly on the host Xcode project, right?

My goal was to embed sqlcipher in the umbrella kmp xcframework, that is the one we import on the xcode project. I thought that if I linked sqlcipher inside it then I could have used directly the xcframework without additional configurations on xcode side

[kpgalligan]

This all gets potentially more complicated if you're building a dynamic framework. I almost always build a static framework, unless I can't for some reason. My mental model of how you'd do this with a static framework is more recent, so I can discuss that with more confidence than dynamic. With a dynamic framework, I would think it would include the linker config pointing at the system lib, or you would need cinterop to the sqlcipher build, but compiling a dynamic framework requires satisfying all the linking when the framework is built, rather than a static framework, which doesn't enforce that.

[kpgalligan]

So basically you’re suggesting to use plain sql for my kmp module, but link sqlcipher directly on the host Xcode project, right?

I'd replacing "suggesting" with "saying the thing you did once". It's an option, but it depends on what you're doing.

My goal was to embed sqlcipher in the umbrella kmp xcframework, that is the one we import on the xcode project. I thought that if I linked sqlcipher inside it then I could have used directly the xcframework without additional configurations on xcode side

cinterop doesn't embed anything itself. It just generates kotlin definitions for native calls. You could need something else that compiles the sqlcipher version of sqlite.c into platform-compatible binary library code, then tell the Kotlin compiler that you want that binary embedded. Conceptually simple, in practice, complicated.
An example of code that does this is zipline. It builds some c code, and adds it to the library publication. It's not exactly the same thing, as the output is a klib and not a framework.: https://github.com/cashapp/zipline/blob/trunk/zipline/build.gradle.kts
I don't off-hand know of an easy way to just tell the Kotlin compiler to add some c code and compile it. There might be a much easier approach that I'm forgetting, but I don't think so. KMP and linking native dependencies is an ongoing issue for a few of our libraries.
Anyway, to replicate what zipline is doing, you could have a KMP module who's only purpose is to compile the sqlcipher c code, then include that in your project to build the umbrella. cinterop actually wouldn't accomplish anything here. You need to have the c bindings available to the linker.

[kpgalligan]

zipline uses a library we publish to do the job of building and including the output in a Kotlin library: https://github.com/touchlab/cklib. however, I explicitly say we don't support it, because anything involved with building C/ObjC and linking is usually a huge pain and most of the issues have nothing to do with cklib itself.

[Damiano Giusti]

I apologize for the delay… Thanks for your clarifications! The tests we do to verify that classes interact with the db properly are passing (e.g checking that some data passed to a datasource get stored). For those kind of tests we could use the plain sql database because we are testing the behavior of our code.
But if I want to test the migration from a plaintext database to an encrypted one in KMP there may not be a way to go, at this point.

However, we exported the umbrella xcframework to iOS which specifies some sqlite flags on the project file, and the operations with the database seem working correctly, so I succeeded removing cocoapods in favor of using cinterop directly

[MJegorovas]

Hey @Damiano Giusti, at the moment I am trying to do the very same - add sqlcipher without cocoapods. Would you be able to share your cinterops and related config(if any)?