https://kotlinlang.org logo
Join Slack
Powered by
If you’re considering writing your first Kotlin li...
# kotlin-native
s
If you’re considering writing your first Kotlin library and thinking about making a logging library… just don’t. 😬 What the ecosystem really needs is a pure Kotlin SigV4 library (that also works for Kotlin/Native). We now have the kotlin-native-aws-lambda-runtime to write our AWS lambdas in Kotlin/Native, but to interact with DynamoDB we seem to need to use something called "_AWS Signature Version 4"_ which isn't available to Kotlin/Native as far as I can see.
h
s
Yes. It's used for example in https://github.com/minio/minio-java - would be nice if that lib had a pure Kotlin variant. 😄
Your link seems to refer to an older version. AFAIK v4 is the current one: https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_sigv.html
c
I'll create a logging library if I want to! But more seriously, there's a big need for these types of small algorithm implementations. Compression algorithms are missing too.
s
I'll create a logging library if I want to!
How many more do you think we need? 😬😉
Compression algorithms are missing too.
Yes, pure Kotlin zlib for example. 🙂 But at least that got a bit easier lately. Also, someone should port https://github.com/Menecats/polybool-java to pure Kotlin.
👍 1
c
How many more do you think we need? 😬😉
No idea, but I'm writing my second one 😅
😆 2
The Polybool port sounds a like a nice short project 🤔
👍 1
s
Yes, I have this on my backlog for quite a while. I don't think I'll get to that in the next weeks.
Also it's math... so I won't be able to solve potential bugs in the original. 😅
I like image metadata. It's pretty easy once you understood the general pattern and involves only basic math.
But with my ONI Seed Browser I have 1.2 million samples in my test data set; so I could tell if a port does the same thing as the original. 😄
j
Also, whoever does it, please do a better job of it than Amazon did. Their Java implementation for signing S3 URLs is literally 100x slower than an implementation that I wrote. I wrote performance tests confirming it. ... actually, I might be able to port that one to KMP. There's a KMP library supporting HmacSHA256 and plain SHA256 right? I believe dev.whyoleg.cryptography has it.
🙏 1
👍 1
s
Sounds like you are the right one for that job. 😄
j
@Stefan Oltmann Do you have plans to make KMP drivers for other AWS services? It's clear they generate the SDKs from some kind of spec, so I'm not sure it's that out there to generate true KMP sdks for most of their services.
s
That's something @Viacheslav Ivanovichev, the maker of that native Kotlin Native Runtime for AWS Lambda, would know. I'm just a user of his lib.
j
s
I don't trust 4 year old issues that much, but hope is there.
Okay... https://github.com/orgs/awslabs/projects/175 ... It's the only item in the "We're working on it"... That indeed looks good.
j
Merged code 3 days ago related to it. Crossing my fingers that their implementation is faster
👍 1
s
Ah, great. Progress! 🙂
j
If they don't have it in the next few weeks, hit me up and I'll port what I have. I don't think you'd want to be dependent on something non-standard though when the standard one is coming so soon.
👍 1
s
SigV4 is still useful outside of AWS, tho. For example I wish to have a MinIO kotlin client.
But yeah, maybe what they do can be used universally. I have no idea how flexible "AWS CRT Kotlin" can be used. We'll see.
Amazon cares more about Kotlin than Cloudflare. That's nice.
a
Also, someone should port https://github.com/Menecats/polybool-java to pure Kotlin.
I'm interested in this!
🙂 1
s
Ok, I'll try it.
At least I have lots of test data, so I can verify if a port does the same.
The unit tests ensuring nothing changed are written so far.
Junie still can't convert Java projects to Kotlin. It's a manual process of using the "Convert to Kotlin" function and cleaning up.
My polybool-kotlin is on it's way. Just need to get it to Maven Central.
K 3
c
Do you plan to 1.0 it directly? When migrating a lib from Java to Kotlin, I tend to change a few things, e.g. use value classes etc
s
No, like with Ashamoo Kim it'll be 0.1.0
Here's a preview: https://github.com/stefanoltmann/polybool-kotlin-dev Once I pushed it to Maven Central I will reset the project with a proper history - just a "Initial commit". 😄 I always like to hide the first 100 embarassing dirty commits where I try to get CI working. 🫣
Once released it'll still need a lot of cleanups like the one you mentioned. For a 0.1 I'm happy if it works and does the same for my 10k sample data. 😄 For 1.0 it should be really pretty and idiomatic Kotlin 🙂
👍 1
c
Depending on how much time I have, I may be interested in helping out in the "make it idiomatic" part
🙂 1
👍 2
s
You're very welcome. Also everybody who understands the math is welcome to fix potential errors. I don't really understand the math behind all that - that's why I have this big regression test. 😄
All I know is that for my ONI seed browser polybool-java union has some decent results and in the 10 months I used that lib I didn't see any errors. So I feel comfortable with basing on that.
If the other operations aside of union are broken I wouldn't know... I hope it's correct. 🙂
It's published. 🙂 🎉 Feel free to fork and do your magic. 😄 https://github.com/StefanOltmann/polybool-kotlin
🎉 1
a
I don't really understand the math behind all that - that's why I have this big regression test. 😄
fwiw I think the original JS lib didn't have any automated tests, and the TS lib only has a few https://github.com/velipso/polybool/blob/v2.0.11/src/polybool.test.ts, so it would be better! I guess the main test is the interactive example - but it's not automated, which is not ideal for KMP (no one's going to manual test 10+ targets...)
If it helps: I started porting the TS lib to Kotlin and I though property-based-testing would help. So, I started writing a test that uses GraalVM to run the TS lib against the Kotlin port. The plan was eventually to randomly generate data, and verify the result is the same for the TS lib and Kotlin port. https://github.com/aSemy/polybool-kt/commit/f3446820a6d4aceed345cf4ba17fd61d12f9a692
👀 2
s
I guess the main test is the interactive example - but it's not automated, which is not ideal for KMP (no one's going to manual test 10+ targets...)
Yes, polybool-java doesn't have had any tests. Just an example. Right now I run the tests only for the JVM target because I do resource loading. I'll improve that like I did with Ashampoo Kim in the past and run the regression test for all targets.
The plan was eventually to randomly generate data, and verify the result is the same for the TS lib and Kotlin port.
That's also a regression test. Unfortunately we lack real unit tests for the stuff right now. I'm not sure how good purely random data will work for tests. That's why I resort for my 10k test sample to real world data. I hope that 10k is a big enough sample to catch even edge cases, but of course with bad luck I may still miss some. Also my test data isn't good for intersect unfortunately. I need to improve there, too. I tried letting ChatGPT generate some, but that didn't work out.
I'm interested in this!
Now I understand what you meant: You're interested in doing the port. 😅 I understood you're interested in someone doing the port and that's because I took action. 🫣 Otherwise I wouldn't have sit down for three long evening doing the port. 😬 How do you do the port? Automated with ML or by hand? Your port may be even better than mine, because you port directly from the source. I ported from another port that's a few years old - so maybe there are unfixed bugs or newly introduced bugs that I now inherited. Hit me up when polybool-kt is available on Maven Central. I like to run tests to see if the results are the same.
Something that irritates me: You imported the whole history of the original project, but it doesn't have the fork marker. Intentional?
a
I understood you're interested in someone doing the port and that's because I took action. 🫣
Yeah, but starting ports and not finishing them is my hobby. I'm probably never going to finish this one!
😆 2
👌 1
s
All fine. I hope this misunderstanding doesn't demotivate you to go on. 😄
fist bump 1
a
How do you do the port? Automated with ML or by hand?
by hand, TS is really similar to Kotlin. Multi-caret find/replace is the most useful tool. I kinda gave up on porting when I encountered a bug, there's something wrong with 
SegmentChainer
- the Kotlin port ends up with an extra region, and debugging it was really hard
👀 1
s
Yeah, I see that. I can't TypeScript / JavaScript. I'm only really good at Java, that's why I take Java projects as a source for my Kotlin ports.
a
Something that irritates me: You imported the whole history of the original project, but it doesn't have the fork marker.
Intentional?
yeah, the "forked from $original" marker on GitHub bugs me! Plus I was going to do what you do: make lots of messy commits, and then clean everything up for the v0.0.1 release.
💯 1
s
So the guy porting from JavaScript to Java did the really hard work. I just converted it and cleaned up syntax, variable names, etc. Given the math behind that you can say I took the easy part. 😄
> v0.0.1 release I start with v0.1.0 , because 0.0.x is a bugfix release for v0.0.0 - and that's an illegal version number in most systems. Sorry for nitpicking. 😄🫣
true 1
@joseph_ivie I don't understand how https://github.com/awslabs/aws-crt-kotlin can help me with SigV4 That's not like a library that I can readily use, or is it?
I found https://gist.github.com/luca992/969af9735c71a284128ba91623741efa ... Is this similar to what you do know in Kotlin/Native ?
Didn't try if that thing in the Gist actually works.
Ok, if I got that right this crt thing is a low-level dependency of https://github.com/awslabs/aws-sdk-kotlin ... which will be Kotlin/Native eventually...
I don't want to deploy a JVM lambda on AWS again. Their startup times are just awful.
j
If you want to go ahead and make SigV4 it's really easy honestly Translate this: https://github.com/lightningkite/service-abstractions/blob/main/files-s3/src/main/kotlin/com/lightningkite/services/files/s3/S3FileObject.kt#L156 into multiplatform. To do more than S3 you'll need to tweak some steps, but it's the right idea.
s
That looks similar to the gist
j
Just looked at said gist - yeah, that's the same thing. We wrote it indepdently - if that existed at the time of writing of the original (in the lightning-server repo) then I didn't know about it
s
Strange that there isn't a basic minimal S3 kotlin multiplatform already if it's not that much.
j
agreed, it's not rocket science
s
People create new logging libraries every week, but S3 clients don't exist. lol.
j
If you put something together let me know - I'll probably make service-abstractions depend on it so that module can go full multiplatform.
s
I'll see if that Gist works. But if I make something it won't be a full blown S3 client, just basic stuff, I guess. As you said maybe an official AWS S3 kotlin multiplatform client comes out...
Maybe that's why the Gist exists... people just take that what they need, but as soon as they start a lib other expect to support the full range of S3 commands.
l
@CLOVIS We also need a character conversion library. I just implemented wrappers around the JVM conversion stuff, and delegating to ICU on Linux, and iconv-lite on JS. It's very annoying, and someone should make a kotlin-multiplatform version (not me, because that would mean implementing support for windows and iphone, and I don't want to do that).
👀 1
h
@Edoardo Luppi
l
Woah, I should I known about this before I spent all that time last night.
It doesn't state which charsets it support though, although I'm sure it supports the important ones.
h
Yeah, I don’t know that, that’s why I pinged Edoardo
e
Hey. That lib is designed primarily to support EBCDIC charsets (https://github.com/lppedd/kotlinx-charset?tab=readme-ov-file#EBCDIC). Although I can add other charsets if necessary. If you have a 
map
file, or a 
ucm
file for the charset you want to support, it's not a lot of work.
h
Regarding AWS S3, why did AWS not publish a OpenAPI specification? Then it would be quite easy to create a library for all the operations. You would still need to compute the signature though.
➕ 2
☝️ 1
s
I found that OpenAPI / Swagger is not that popular as it should be.
At least I expected / hoped for much broader and better support in all the tooling we have available.
Open in Slack
PreviousNext
Powered by