https://kotlinlang.org logo
Join Slack
Powered by
Hey all, hows things! Working on a multiplatform p...
# multiplatform
a
Hey all, hows things! Working on a multiplatform project, and we need to encrypt our db. Using sql-cipher on android and its working fine but for the ios side its proving a bit tricker. We don't want to use cocoapods because of the overhead, so we were hoping to build the lib and link it. I'm having some issues with this and was wondering if anybody ran into something similar? SQLCipher readme says you can run:
Copy code
./configure --enable-tempstore=yes CFLAGS="-DSQLITE_HAS_CODEC" \
	LDFLAGS="/opt/local/lib/libcrypto.a"
However this gives me an error saying "C compiler cannot create executables" Also I saw on the forums somebody said to try: 
./configure --enable-tempstore=yes CFLAGS="-DSQLITE_HAS_CODEC -I/usr/local/opt/openssl/include/" LDFLAGS="/usr/local/opt/openssl/lib/libcrypto.a"
This looks like it builds but at the end says "error: Library crypto not found. Install openssl!" but I have already installed openssl via homebrew. Basically I need to get the .a file build and bring it into the ios part of the multiplatform project and then bring in the .h file and create the .def file but its proving trickier than I thought. I know this isn't exactly related to multiplatform specifically but I thought maybe somebody encountered something similar and could help. Thanks!
r
I haven't worked through it personally but here's a walkthrough based on how we've done sqlcipher on some of our apps https://touchlab.co/sqlcipher-kotlin-multiplatform-mobile/
a
Thanks for the reply @russhwolf Unfortunately it seems like that implementation is using cocoapods, which we don't want to use. So it involves having to compile sqlcipher to a static library (.a file type) which I can't seem to get to work from their documentation
v
To figure out the required clang flags for ios lib compilation, you can for example (maybe there're simpler options but that's the only one I came up with) create an empty Xcode ios project, add cocoapods support and add 
SQLCipher
pod to it, then build it and inspect build logs. Also, there's SQCipher podspec with library-specific compile flags and 
./configure
flags they're using https://github.com/CocoaPods/Specs/blob/master/Specs/1/2/7/SQLCipher/4.5.0/SQLCipher.podspec.json With this information, I could build 
libSQLCipher.a
like that:
Copy code
./configure --enable-tempstore=yes --with-crypto-lib=commoncrypto CFLAGS="-DSQLITE_HAS_CODEC -DSQLITE_TEMP_STORE=2 -DSQLITE_SOUNDEX -DSQLITE_THREADSAFE -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_STAT3 -DSQLITE_ENABLE_STAT4 -DSQLITE_ENABLE_COLUMN_METADATA -DSQLITE_ENABLE_MEMORY_MANAGEMENT -DSQLITE_ENABLE_LOAD_EXTENSION -DSQLITE_ENABLE_UNLOCK_NOTIFY -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_FTS4_UNICODE61 -DSQLITE_ENABLE_JSON1 -DSQLITE_ENABLE_FTS5 -DHAVE_USLEEP=1 -DSQLITE_MAX_VARIABLE_NUMBER=99999"

make sqlite3.c

`xcrun --sdk iphoneos --find clang` -x c -target arm64-apple-ios8.0 -isysroot `xcrun --sdk iphoneos --show-sdk-path` -fmodules -DSQLITE_HAS_CODEC=1 -fstrict-aliasing -fembed-bitcode-marker -DSQLITE_HAS_CODEC -DSQLITE_TEMP_STORE=2 -DSQLITE_SOUNDEX -DSQLITE_THREADSAFE -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_STAT3 -DSQLITE_ENABLE_STAT4 -DSQLITE_ENABLE_COLUMN_METADATA -DSQLITE_ENABLE_MEMORY_MANAGEMENT -DSQLITE_ENABLE_LOAD_EXTENSION -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_FTS4_UNICODE61 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_UNLOCK_NOTIFY -DSQLITE_ENABLE_JSON1 -DSQLITE_ENABLE_FTS5 -DSQLCIPHER_CRYPTO_CC -DHAVE_USLEEP=1 -DSQLITE_MAX_VARIABLE_NUMBER=99999 -DNDEBUG -DSQLITE_HAS_CODEC -DSQLITE_TEMP_STORE=2 -DSQLITE_SOUNDEX -DSQLITE_THREADSAFE -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_STAT3 -DSQLITE_ENABLE_STAT4 -DSQLITE_ENABLE_COLUMN_METADATA -DSQLITE_ENABLE_MEMORY_MANAGEMENT -DSQLITE_ENABLE_LOAD_EXTENSION -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_FTS4_UNICODE61 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_UNLOCK_NOTIFY -DSQLITE_ENABLE_JSON1 -DSQLITE_ENABLE_FTS5 -DSQLCIPHER_CRYPTO_CC -DHAVE_USLEEP=1 -DSQLITE_MAX_VARIABLE_NUMBER=99999 -fno-objc-arc -fmodule-name=SQLCipher -Oz -fno-common -c sqlite3.c -o sqlite3.o

`xcrun --sdk iphoneos --find libtool` -static -arch_only arm64 -syslibroot `xcrun --sdk iphoneos --show-sdk-path` -o libSQLCipher.a sqlite3.o
I haven't tested the resulting lib, hope this helps in some way.
a
@Vadim Thanks for the reply! I copied and pasted your commands there from ./configure until the make command and it seemed to finish without issue, it said "make: `sqlite3.c' is up to date." Where can I find the .a file? Do I need to run that 'xcrun command you have there? Should I be able to find the .a file in the sqlcipher folder then? And I can drag that into my multiplatform project and hook it up for cinterop use using a .def file?
I did it and I see the libSQLCipher.a file, nice one I can't believe that worked! Any idea what is different about what you did vs their instructions? You seem to have many more parameters, are these just from viewing the podspec .configure flags? Wonder why they don't mention all of this
v
In one of GitHub threads, they’re saying that there exists a commercial version of library with all iOS frameworks/libs included, maybe this is the reason, idk. Yeah, 1st command just generates an amalgamation, “merged” sqlite.c file. Next 2 commands actually build object file and then, library. As I suggested to you, I've built Xcode project with sqlcipher pod and inspected build commands. Most of them came from podspec, the rest are ios-specific flags telling compiler to build ios arm64 library
a
Ah I see, thanks! So for the project we need to add it as a cinterop in our multiplatform project. This involves adding it to the cinterop folder (The .h and .a file) and then adding some stuff to the gradle file, I'm basically copying some of the existing stuff we have. However for the iOS side of things we need to build an XC Framework, would this change some of those commands do you know? Not super familiar with iOS but I know the XCFramework is much newer to build, I don't know if this would affect the ios specific build flags
v
Right, you need to link libsqlcipher to your framework somehow. Either cinterop would take care of this or you can use 
linkerOpts(...)
within 
framework {}
gradle block
a
Nice, we have a compilations["main"].cinterops in our build.gradle where we are pulling them in. Just a question though, would the 'xcrun' and onwards commands change based on xcframeworks or architecture?
Right now the project is building however the ios app is crashing on launch when i include the changes to build.gradle. I have a feeling making its something to do with the --sdk or -arch_only arm64 or sometghing
v
You have to change sdk from 
iphoneos
to `watchos`/`iphonesimulator`/etc and arch from 
arm64
to `armv7`/`x86_64`/etc if you need to build for older devices/watchos/simulator/etc. What kind of crashes are you experiencing, do you have logs?
a
If we just build on iPad for example, iphoneos should be fine right? arm64 is for the more modern ipads right? I guess that should be fine also.
In terms of crash logs, I will have a look now but I'm not super familiar with iOS and how it handles crash logs, I will try post something more concrete
Weird it ran fine a few minutes ago, I actually opened the app and did a test and checked and saw the database was there and was asking me for the passphrase to decrypt it
Tried again and it crashes
Hmm so I think it might be related to a change I made where we retrieve the DatabaseConfiguration. I added a key = "test" param which I saw in the kampkit example Seems like it crashes when i do that
Function doesn't have or inherit @Throws annotation and thus exception isn't propagated from Kotlin to Objective-C/Swift as NSError.
 
It is considered unexpected and unhandled instead. Program will be terminated.
 
Uncaught Kotlin exception: kotlin.Exception: android/database/sqlite/SQLiteException - unknown error (code 0): Queries can be performed using SQLiteDatabase query or rawQuery methods only.
   
at 0  testLib           0x00000001011acd98 kfun:kotlin.Throwable#<init>(kotlin.String?){} + 96
   
at 1  testLib           0x00000001011a4d58 kfun:kotlin.Exception#<init>(kotlin.String?){} + 92
   
at 2  testLib           0x00000001012aff80 ThrowSql_SQLiteException + 384
   
at 3  testLib           0x000000010179f0f8 _ZN7android23throw_sqlite3_exceptionEiPKcS1_ + 960
   
at 4  testLib           0x000000010179f46c _ZN7android31throw_sqlite3_exception_errcodeEiPKc + 0
   
at 5  testLib           0x000000010179fec8 SQLiter_SQLiteStatement_nativeExecute + 72
   
at 6  testLib           0x00000001012b6788 kfun:co.touchlab.sqliter.NativeStatement#execute(){} + 208
   
at 7  testLib           0x00000001012a6260 kfun:co.touchlab.sqliter.setCipherKey$lambda-0#internal + 116
   
at 8  testLib           0x00000001012a6634 kfun:co.touchlab.sqliter.$setCipherKey$lambda-0$FUNCTION_REFERENCE$313.invoke#internal + 76
   
at 9  testLib           0x00000001012a66b4 kfun:co.touchlab.sqliter.$setCipherKey$lambda-0$FUNCTION_REFERENCE$313.$<bridge-UNNN>invoke(-1:0){}#internal + 96
   
at 10 testLib           0x00000001012a5764 kfun:co.touchlab.sqliter#withStatement@co.touchlab.sqliter.DatabaseConnection(kotlin.String;kotlin.Function1<co.touchlab.sqliter.Statement,0:0>){0§<kotlin.Any?>}0:0 + 412
   
at 11 testLib           0x00000001012a4f30 kfun:co.touchlab.sqliter#setCipherKey@co.touchlab.sqliter.DatabaseConnection(kotlin.String){} + 428
   
at 12 testLib           0x00000001012b549c kfun:co.touchlab.sqliter.NativeDatabaseManager.createConnection#internal + 1936
   
at 13 testLib           0x00000001012b4998 kfun:co.touchlab.sqliter.NativeDatabaseManager#createMultiThreadedConnection(){}co.touchlab.sqliter.DatabaseConnection + 160
   
at 14 testLib           0x00000001012b8b50 kfun:com.squareup.sqldelight.drivers.native.NativeSqliteDriver.<init>$lambda-0#internal + 232
   
at 15 testLib           0x00000001012b9248 kfun:com.squareup.sqldelight.drivers.native.NativeSqliteDriver.$<init>$lambda-0$FUNCTION_REFERENCE$295.invoke#internal + 144
   
at 16 testLib           0x00000001012bd7cc kfun:com.squareup.sqldelight.drivers.native.SinglePool#<init>(kotlin.Function0<1:0>){} + 592
   
at 17 testLib           0x00000001012b7ccc kfun:com.squareup.sqldelight.drivers.native.NativeSqliteDriver#<init>(co.touchlab.sqliter.DatabaseManager){} + 444
   
at 18 testLib           0x00000001012b7f08 kfun:com.squareup.sqldelight.drivers.native.NativeSqliteDriver#<init>(co.touchlab.sqliter.DatabaseConfiguration){} + 176
   
at 19 testLib           0x000000010168bc10 kfun:databaseRetriever.DatabaseRetriever#getDatabase(){}com.test.testLib.testLibDatabase + 484
   
at 20 testLib           0x0000000101597240 kfun:dao.factory.RealDaoFactory#<init>(databaseRetriever.IDatabaseRetriever){} + 400
   
at 21 testLib           0x000000010169cfa4 kfun:gateway.testLib#getGateway(synchronization.configuration.GatewayConfiguration;genericdevices.Delgate;genericdevices.PrinterDelegate;genericdevices.BarcodeScannerDelegate){}gateway.Gateway + 912
   
at 22 testLib           0x0000000101734764 objc2kotlin.2474 + 296
   
at 23 testLibDemo           0x0000000100d1a9e4 $s15testLibDemo35DeviceIntegrationTestViewControllerC11viewDidLoadyyFTo + 32
   
at 24 UIKitCore              0x00000001ad03ce50 CC6E5AC7-8248-35F6-8B42-2E25C93DCF0A + 4623952
   
at 25 UIKitCore              0x00000001ad041408 CC6E5AC7-8248-35F6-8B42-2E25C93DCF0A + 4641800
   
at 26 UIKitCore              0x00000001acf88a94 CC6E5AC7-8248-35F6-8B42-2E25C93DCF0A + 3885716
   
at 27 UIKitCore              0x00000001acf88d9c CC6E5AC7-8248-35F6-8B42-2E25C93DCF0A + 3886492
   
at 28 UIKitCore              0x00000001acf89c60 CC6E5AC7-8248-35F6-8B42-2E25C93DCF0A + 3890272
   
at 29 UIKitCore              0x00000001acf8afe0 CC6E5AC7-8248-35F6-8B42-2E25C93DCF0A + 3895264
Only happens when i add key = "passphrase" to my DatabaseConfig
Copy code
private fun getConfiguration(schema: SqlDriver.Schema, name: String, inMemory: Boolean): DatabaseConfiguration {
    return DatabaseConfiguration(
        name = name,
        version = schema.version,
        create = { connection ->
            wrapConnection(connection) { schema.create(it) }
        },
        upgrade = { connection, oldVersion, newVersion ->
            wrapConnection(connection) { schema.migrate(it, oldVersion, newVersion) }
        },
        inMemory = inMemory,
        key = "testPassphrase"
    )
}
without the key there it seems to work
v
I haven't worked with sqlite libs at all so i can't help here :( It seems sqlite lib doesn't like the way it's being queried by sqldelight… or not
a
I'm hoping its just an issue with the declaration and now how sqlcipher is being created 😄
Appreciate the help you provided anyways, got the thing build which is the main thing
v
Yeah I was concerned it'd be some linker crash on start, glad it wasn't :)
a
Hey @Vadim sorry for the direct ping but thought you might know since you helped get it working originally, but it seems none of the SQLCIPHER methods in the header file are included in the static library. Android doesn't seem to have access to it. Header file has a section:
Copy code
/* BEGIN SQLCIPHER */
#ifdef SQLITE_HAS_CODEC
But from the commands we use it looks like its included. Any ideas on potential fixes?
v
How have you built/included libsqlcipher for android? Maybe you forgot some android-specific flags…
a
So essentially all I did was run the commands you mentioned above. Then followed the implementation instructions for libraries here: https://kotlinlang.org/docs/kmm-add-dependencies.html#workaround-to-enable-ide-support-for-the-shared-ios-source-set So basically brought in the .h and .a files and created a .def file, then update the build.gradle file. We actually have the Android implementation working. The issue is the ios implementation requires the .a file and runs with the interop. So in the ios implementation file, I can use methods from sqlciphers .h file such as sqlite3_open(). However I can't use sqlite3_key() as it is defined under the section marked
Copy code
/* BEGIN SQLCIPHER */
#ifdef SQLITE_HAS_CODEC
 // sqlite3_key(), sqlite3_key_v2(), sqlite3_rekey(), sqlite3_rekey_v2(), sqlite3_activate_see() methods are defined here
#endif
/* END SQLCIPHER */
v
Yeah this flag is included in compilation scripts I've came up with earlier so it's weird that you cannot use these functions…
a
Yeah, and i just verified by building a new KMM project and using cocoapods to import sqlcipher, it works and I can view the methods
so i'm assuming its something to do with the generation of the libSQLCipher.a that I drag into the IDE to setup. If you have any ideas let me know. I'm really not familiar with building things from terminal so I'm sort of lost with what options I have now. Already tried contacting them on their forums but now reply unfortunately
v
Can you try a silly hack and just remove #ifdef SQLITE_HAS_CODEC /#endif lines from the .h file (from the part you've pasted here)? Does this make functions visible?
a
Haha I already tried that yesterday thinking it would work but nope, still seems like it isn't visible which I don't understand. You know the xcrun commands? Where did you get all those from? I know you said previously you got the commands from the cocoapods json, but just wondering where the xcrun stuff came from, maybe issue could be around that since that seems to actually build the .a file.
@Vadim Just to update you on this, adding
Copy code
compilerOpts("-DSQLITE_HAS_CODEC=1")
to the .def file seems to of fixed it! Thanks for the help along the way. I will have to write an article on this after I'm finished incase anybody gets stuck with it
v
Hm, interesting. Maybe you don't even need 
=1
but I'm not sure all xcrun flags are “standard” for clang cross compiling for ios. Xcode generates them according to project config. What I've done was created a new Xcode app project, added sqlcipher cocoapod to it and then compiled. After that I had a log with all the build commands so I just extracted these flags from it because I don't remember all of them :) there were couple of other flags like commands to silence some compiler warnings, I just omitted them.
a
Thanks for the explanation. I actually did the same as you after you said what to do, just so I can see how it works myself, and saw all the parameters there, so thats good to know going forward. I know this is a long shot but do you have any advice on converting Objective C / C to Kotlin? SQLCipher has this example on their website:
Copy code
var rc: Int32
    var db: OpaquePointer? = nil
    var stmt: OpaquePointer? = nil
    let password: String = "correct horse battery staple"
    rc = sqlite3_open(":memory:", &db)
    if (rc != SQLITE_OK) {
        let errmsg = String(cString: sqlite3_errmsg(db))
        NSLog("Error opening database: \(errmsg)")
        return
    }
    rc = sqlite3_key(db, password, Int32(password.utf8CString.count))
Which doesn't really translate to Kotlin too well. I have tried this:
Copy code
var rc: Int
val db: CValuesRef<CPointerVar<cnames.structs.sqlite3>>? = null
val db2: CValuesRef<cnames.structs.sqlite3>? = null
rc = sqlite3_open(databasePath.encodeToByteArray(), db)
val isOk = rc == SQLITE_OK
NSLog(isOk.toString())
rc = sqlite3_key(db2, null, 0)
val isOk2 = rc == SQLITE_OK
NSLog(isOk2.toString())
I have 2 dbs here because the first db only works in the sqlite3_open and the 2nd one works in the sqlite3_key call. It doesn't make sense to me because the example they show just has 1 db object. None of this works anyway as it crashes on ios when I try run it. Something strange going on for sure, I had to add null to the sqlite3_key because it wouldn't accept a string even though the example has one. Sorry for all the questions but thought you might have an idea 😄
v
That's a good question :) I'm also still learning Native, I've tried working with C pointers from within Kotlin and quickly gave up. now using C string only as they converts nicely, also my interaction with C is really trivial right now so I can get by with strings as the most complex structure.
a
How does that work?
I keep getting crashes on rc = sqlite3_open(databasePath, db) I just see Thread 1: EXC_BAD_ACCESS (code=1, address=0x0) in xcode. Sort of hard to know what to do
the database path is /var/mobile/Containers/Data/Application/800A5D27-85C4-4DA4-B1AC-3F6A630F45DE/Documents/testDatabase.db
Not sure if its an issue with resolving that, but all the logic I have is in a try/catch but it doesn't seem to be getting caught, just showing white screen on the app and the debugger just keeps going back to the line with rc = sqlite3_open(databasePath, db)
v
I mean I have little experience with C strings and no experience with more complicated pointers
a
Ah I see 🙂 thanks for the help anyways, will keep on playing with it and see if can make any progress
41 Views
Open in Slack
PreviousNext
Powered by