https://kotlinlang.org logo
Join Slack
Powered by
Say I have a KMP library where the JVM target requ...
# gradle
s
Say I have a KMP library where the JVM target requires JNI and therefore needs to publish the native shared object. There's up to six native binaries in total: Windows, macOS, Linux, each for x64 and ARM. What's the best way to publish this library to Maven Central with the native binaries easily available for the user of the library? • In a plain JVM project I might use feature variants, but I think KMP doesn't support that. • I could publish each native binary in a jar with a special classifier, so for example the jar is 
library-x.y.z-natives_os_arch.jar
. The user of the library would depend on the library (without classifier) plus one or more natives , depending on which platforms they support. • I could publish all the native binaries outside of Maven Central, and document for the user how to add them to their app. But this seems hard to use, essentially introducing some nonstandard dependency management. Or, I could publish a small gradle plugin that configures this for the user. • I could publish my library as a single fat jar with all six binaries included. But that feels like it unnecessarily inflates the binary size. And building this one jar in CI would get complicated, with build steps on multiple different runners all coalescing into one jar. How is this typically done in the JVM and the KMP world? For example, I assume Compose Multiplatform itself includes some native libraries (skia?). But as a user I don't have to do anything special to depend on it, perhaps because the Compose gradle plugin deals with it for me? Are there some good examples out there that don't come with an associated plugin? The second approach (classifiers) seems like the most viable to me. But I'm unclear on: • What would the gradle config look like for such a library? • What would the configuration look like for the user of the library? • Would they be able to use a version catalog with these special artifacts? My goal with this KMP library is to support Kotlin/JVM and Android with JNI, and Kotlin/Native with cinterop. But for now, I'm focusing on the (non-android) JVM build only.
e
Android publishing has all the native libraries in the .aar, and I think the easiest way to handle it on JVM would be to also include all native libraries in a single JAR
s
This is like, 20+ megabytes per platform x 6 platforms. On Android, at least the app, when downloading from the play store, doesn't include all the irrelevant natives. But yeah, I guess a JVM app could do the same I guess? Fat library, then distribute builds with unnecessary libs stripped out?
e
yeah if an app is built as an AAB then the Play Store will filter the architecture (and other bits) in the APK it delivers to the clients
for JVM apps, I feel like it's actually annoying that Compose Desktop uses separate Skiko artifacts; it makes it harder to build a multiplatform release, since you have to merge them back together
(also their 
compose.desktop.currentOs
dependency annoyingly conflates the building and target platform…)
s
with that strategy, I'm curious how to manage my library's gradle assemble + publish in CI. The natives will get built across 3-6 different CI runners, all somehow getting copied into a single jar's resources.
Right now, a gradle assemble task knows its dependencies and builds the native binary for at least the platform I'm on With the inputs to a jar distributed across builds on other machines, I guess I can't rely on gradle's task graph. So I need to wire it all up manually, while still somehow preserving a good local dev experience with gradle task dependencies
e
can it not be cross-compiled?
even if you use feature variants, they're all the same artifact name so you can't just publish separately from each host
s
I can prob cross compile across x64 and ARM. I'm not so sure I can cross compile a complex C++ library for like, macos, windows, and linux all on one machine
e
I think it would be possible to use JVM feature variants in a JVM module, which your KMP module depends on, if that's how you want to do it
well. not necessarily the best solution, but Skiko publishing simply downloads prebuilt assets
s
> I think it would be possible to use JVM feature variants in a JVM module, which your KMP module depends on, if that's how you want to do it That's an interesting approach. My KMP module would then need a bunch of 
actual typealias
on the JVM target to still export a common api with its native target. But it's probably viable. But it results in an extra maven artifact
> well. not necessarily the best solution, but Skiko publishing simply downloads prebuilt assets Could set up my CI to build all six natives, upload them as artifacts somewhere, then have the publish job download them and include them into the jar. But the local dev flow will be different (building locally) so gradle needs to be configured for both paths (probably switch on some property set in the publish job I guess?)
e
My KMP module would then need a bunch of 
actual typealias
on the JVM target to still export a common api with its native target
sorry, maybe I didn't explain it well. I was thinking you could make a JVM project, whose only job is to wrap the `.so`/`.dll` artifacts into variant JARs. then depend on that from the JVM target of your KMP module, where all the normal Kotlin code would reside
s
Ah I see, so the JVM project with feature variants only includes the natives, not the classes that consume them
and then my CI would publish the whole dependency tree from one platform, then from all other runners additionally publish that native jar
e
I think publishing the native JAR as feature variants from different machines will still collide with each other, but give it a shot
s
ah damn, why is that? I'm actually not familiar with how feature variants get published
e
they don't have different Maven coordinates, they're different classifiers under the same coordinates
s
Ah, so classifiers for the same coordinates can't be published independently? I guess that makes sense
Looks like skiko does separate modules, similar to Kotlin Native klibs. So there's a JVM module plus a runtime module per platform
e
well, in principle they could, but you'd also be trying to write the POM and Gradle Module Metadata from each publication, which overlaps
yeah Skiko does, and that makes a universal binary impossible (without custom tooling) https://youtrack.jetbrains.com/issue/CMP-5125
👀 1
(although TBF the packaging would also need work to support that for CMP, so it's not the only blocker)
s
but I think you're right that it's not worth managing the complexity of overwriting the metadata
yeah Skiko does, and that makes a universal binary impossible (without custom tooling)
That's interesting. For macOS I think I can build a universal dylib at least for my library. It'll double the size but that's probably fine, at least it's not 6x.
Unsure if windows/linux support something like that, though unsure if they even need to
(I just remembered that in my case there's also double the variants for linux/windows, because we have opengl and vulkan variants available there, but just metal on macos) so probably, the list of natives for the jvm target 😬: • macos_universal_metal • linux_amd64_opengl • linux_amd64_vulkan • linux_aarch64_opengl • linux_aarch64_vulkan • windows_amd64_opengl • windows_amd64_vulkan • windows_aarch64_opengl • windows_aarch64_vulkan (this is what I'm working on if you're curious)
e
oh that's why the binary size is so large… I was thinking, it's unusual that including multiple native libraries would be that big of a deal
s
Lol yeah, if it was just kilobytes it'd be whatever A single platform is roughly 20MB right now. Though the MapLibre Android .aar, pretty comparable to my JVM builds, is also ~20MB and I assume that includes multiple architectures. So maybe there's some things I can do to reduce my binary size somewhat
v
I have no experience with that, but it indeed sounds to me like feature variants should be the way to go. If your CI for example is GHA, you could upload the artifacts to GHA and in the publish job download those artifacts to publish them together, or if you don't want them as single artifacts on GHA you could use the cache to share the artifacts between jobs. If you want to publish them as "standard" feature variants, you indeed need to publish them together if you want to publish to Maven Central. If publish to some other repository, that might or might not support publishing artifacts partially, but even then it might not be the best idea, besides that you anyway have to have one publishing task that is aware of all variants as they are listed in the Gradle Module Metadata. Actually you can also publish feature variants in a slightly non-standard way like KMP projects do, where the variants are published under individual coordinates like foo-jvm and foo-android and the Gradle Module Metadata has two feature variants that use the 
url
or 
download-url
or how the artifact is called to point to the artifacts of that other coordinates. This also improves consumability for non-Gradle consumers as those could depend on the specific variant by coordinates, while Gradle users can just use the "main" coordinates and use for example 
OperatingSystem
and 
MachineArchitecture
attribues to select the variant(s) they need.
e
afaik the classifier approach is somewhat standard in Maven (insofar as anything is standard), e.g. https://github.com/fmarot/nativedependencies-maven
8 Views
Open in Slack
PreviousNext
Powered by