I have a KMP project that has a 'common', 'common-...

# javascript

[yawkat]

I have a KMP project that has a 'common', 'common-wasm' and a 'web-app' module. the 'common-wasm' module produces a kotlin/wasm library. is it possible to depend on that library from the kotlin/js 'web-app' executable? i added a normal dependency, but it shouts at me due to the platformType mismatch, and i dont think thats what i want anyway since it would be an IR dependency.

the branch i'm testing is here: https://github.com/yawkat/kcd2dicesim/tree/special-wasm

[Edoardo Luppi]

No, there is no way to do that, you cannot depend directly on a wasmJs project from a js one. What does the web-app do with the WASM code?

[Edoardo Luppi]

But even if you could, you most likely want two different outputs: one for the web application, and one for the WASM component.

[yawkat]

the problem i'm trying to solve is that i have an existing kvision app (kotlin-js) that does some heavy platform-agnostic computation (common module) and i'd like to offload that computation to wasm

[yawkat]

it seems to work fine within kotlin/wasm but the interaction of the two i have no idea how to do

[yawkat]

(im open to different project structures, this is just my latest attempt)

[Edoardo Luppi]

Manually creating the external bindings in the js project, for the `@JsExport`ed functions in the wasmJs project is what you're looking for I believe.
You'll then have to do some Gradle magic to depend on the common-wasm module, and include its output in the final JS bundle.

[yawkat]

the gradle magic is what im afraid of 😬 is the general idea to produce a node module in common-wasm and then add an npm dependency in web-app on that output, and then fix up the gradle task dependencies?

[yawkat]

i see a bunch of webpack and npm and yarn but frankly im a jvm guy and i dont really know what the right module format is

[Edoardo Luppi]

The key point here is you should treat the WASM module as a standalone project.
However it's difficult to give you a good answer without creating a sample project.

[Edoardo Luppi]

I have to admit linking tasks between sub-projects is a rare occurrence.
Gradle explicitly discourages doing it, and instead points you to this.
Maybe @Vampire can give you an hint or two

[Edoardo Luppi]

> is the general idea to produce a node module in common-wasm and then add an npm dependency in web-app on that output, and then fix up the gradle task dependencies?
Forgot to answer to this part. Nope, there is no need to do that.
What you want to do is simply have A (js) depend on the output of project B (wasm), and include that B output into A resources, so that it's included in the final bundle.

A will then have external declarations to load the exported WASM functions through the JS wrapper.

[yawkat]

it does actually seem to work with that approach with @JsModule("kcd2dicesim-common-wasm-wasm-js")

[yawkat]

at least webpack builds it correctly. it doesnt import it properly yet

[yawkat]

but progress 🙂

[yawkat]

it seems the wasm code produces a module export, but the javascript external functions try to use a non-module function. it's forcing me to use

@file:JsModule("kcd2dicesim-common-wasm-wasm-js")
@file:JsNonModule

but i would really only like the former. would it be better to fix this on the js side by fixing the import or on the wasm side by making the export non-module?

[yawkat]

the error looks like this:

Uncaught TypeError: wasmCalculateEv is not a function
    think webpack-internal:///./kotlin/kcd2dicesim-web-app.js:684
    lambda_5 webpack-internal:///./kotlin/kcd2dicesim-web-app.js:430
    lambda webpack-internal:///./kotlin/kvision.js:12638
    invokeHandler webpack-internal:///../../node_modules/snabbdom/build/modules/eventlisteners.js:8
    handleEvent webpack-internal:///../../node_modules/snabbdom/build/modules/eventlisteners.js:22
    handler webpack-internal:///../../node_modules/snabbdom/build/modules/eventlisteners.js:27

and looking at the js, wasmCalculateEv is just called bare, not on a module like i think it should be?

[Edoardo Luppi]

It's definitely a bit cumbersome. Which is also why generally you don't see this kind of things yet.

I'm not sure if my idea is going to work, but what if you encapsulate the WASM module loading inside of the WASM project itself?
Add the js() capability, and make the js tasks depend on the wasmJs ones, than move the outputted WASM binary into js resources.
At this point you can depend directly from web-app to common-wasm using js source sets.

but the javascript external functions try to use a non-module function. it's forcing me to use

Did you set useEsModules() in the web-app project?

[yawkat]

well useEsModules seems to have broken a bunch of other things 🙂

[yawkat]

i did actually try what you describe before. it was all a single module and i had copy tasks copy the .bundle.js back and forth. but it was a huge mess

[Edoardo Luppi]

Keep in mind ESM is the only variant that is supported for K/WASM JS exports

[yawkat]

i managed to get the single-module build working once, transiently, and then never again. too brittle

[Edoardo Luppi]

By single-module you mean you had the web-app and wasm projects combined into a single Kotlin project?

[yawkat]

yes. i made the js tasks depend on the wasm tasks through gradle voodoo

[yawkat]

it seems kvision has some problems with ESM

[yawkat]

so maybe not the path forward

[Edoardo Luppi]

Ahhhhhh nooo, that's a big mess and won't ever work nicely.
My idea was to keep the two things separated in different projects. To clarify:

- web-app
    js()
- wasm-module
    wasmJs()
    js()      <-- This simply wraps what is outputted by the wasmJs tasks.
                  You basically do the wrapping work here, instead of doing it in web-app

You then have the following in your web-app Gradle script:

jsMain {
  dependencies {
    implementation(project(":wasm-module"))
  }
}

[yawkat]

ah i see what you mean

[Edoardo Luppi]

Now, this is all hypothetical, never tried it. In said wasm-module you could even play with expect-actual to mandate functions' implementations between js and wasmJs source sets, ensuring signatures stay the same.

[yawkat]

hm i can see how it might be preferable for organization but i dont see how it helps with the loading. wasm-module:js would still be IR that is consumed by the web-app:js compiler after all

[yawkat]

add the wasm-module:wasm output dir as a resource dir to the :js build, and then just add wasm.bundle.js to the html?

[Edoardo Luppi]

wasm-module:js would still be IR that is consumed by the web-app:js compiler after all

Yes, but the js functions' implementations would load and call the WASM binary - through the JS wrapper - that you now have in your js resources, thanks to having a dependency to the wasmJs tasks

[Edoardo Luppi]

The problem here is, are resources retained in klibs? And will your web-app correctly see those wasm-module resources?

[yawkat]

basically implement a wrapper manually in js that loads the wasm?

[yawkat]

in js, not in kotlin/js

[Edoardo Luppi]

Not manually, that's what is built by depending on the wasmJs compilation tasks

[Edoardo Luppi]

This is getting a bit complex, so maybe we're not understanding each others, but when I say depending on I mean the wasm-module js tasks depending on the wasm-module wasm tasks

[Edoardo Luppi]

Would be nice to have a POC of this stuff, you're not the first to ask about it

[yawkat]

i get that part. and i understand that wasmJs builds a js wrapper that loads the wasm. im just not sure how to access those js exports from my other js

[yawkat]

(which is probably because i have 0 experience with js module loading)

[yawkat]

i see a loader like this:

import { instantiate } from './kcd2dicesim-common-wasm-wasm-js.uninstantiated.mjs';


const exports = (await instantiate({

})).exports;

export const {
    wasmCalculateEv,
    wasmGetMoveKeep,
    wasmGetMoveShouldContinue,
    memory,
    _initialize,
    startUnitTests
} = exports;

[yawkat]

yes i think i can figure out the gradle pieces

[Edoardo Luppi]

To clarify my idea, in the wasm-module js source set you'll simply use @JsModule("wasm-wrapper.js"), there is no need to use useEsModules() or useCommonJs() because you're simply building a multiplatform library.
In the web-app side, on the other hand, you'll be somewhat forced to use useEsModules(), as that's how the JS wrapper exports those functions.

[yawkat]

it seems useEsModules irreparably breaks kvision, or at least sufficiently badly that fixing it is beyond the scope of what i want to do 😄

[yawkat]

hm well useCommonJs doesnt seem to die immediately at least

[Edoardo Luppi]

I think at this point you're left with trial and error. We're in a realm that's not been explored well yet ahaha
I wouldn't know what to tell at this point to be honest : (

[Edoardo Luppi]

Generally speaking in the end CommonJS doesn't matter btw, web-app Webpack's task would output a single bundle (a single .js file) suitable for the browser environment.
In this case tho, with your JS resources trick I'm not sure how that would work.

[yawkat]

yep i got it to output a single bundle earlier but the importing didnt work

[yawkat]

anyway thanks for the help. it looks pretty close now, just need to figure out the modularity mess. but i will do that tomorrow. the gradle stuff is a bit untangled now, maybe i can recruit melix to fix it up at some point 😄

[Edoardo Luppi]

No problem. Maybe tomorrow you'll get better help!

[Vampire]

Not going to read this whole conversation as it does not seem to be of too much interest for me from a quick look, but I was mentioned. So, can I help with something, or is that expired already? :-D

[Edoardo Luppi]

@Vampire the issue was with Gradle basically, and what's the best way to have task X from subproject A, depend on task Y from subproject B.
The idea was to have the JS compile task depend on the corresponding WASM compile task of another subproject, since you cannot have the JS project depend directly on the WASM one.

[yawkat]

i was able to get the import to work with a shim like this:

import {
    wasmCalculateEv,
    wasmGetMoveKeep,
    wasmGetMoveShouldContinue
} from "./kcd2dicesim-common-wasm-wasm-js.mjs";

window.wasmCalculateEv = wasmCalculateEv;
window.wasmGetMoveKeep = wasmGetMoveKeep;
window.wasmGetMoveShouldContinue = wasmGetMoveShouldContinue;

ugly but it does the job.

[yawkat]

oh and the gradle setup for completeness:

val commonWasm = project.parent!!.project("common-wasm")

kotlin {
  ...
  sourceSets["jsMain"].resources.srcDir(commonWasm.layout.buildDirectory.dir("dist/wasmJs/productionLibrary").get().asFile)
}
tasks.named("jsProcessResources").configure {
  dependsOn(commonWasm.tasks.named("wasmJsNodeProductionLibraryDistribution"))
}

[yawkat]

unfortunately the perf gain is not what i hoped, still >10x worse than JVM. so more debugging to do 🙂

[Edoardo Luppi]

Yeah you're not going to get massive perf gains, but it's still better than JS.
I also run some benchmarks here last year.

[yawkat]

the reason i was hoping for big perf gains is that im working with bitwise ops on 64 bit ints, so i figured the js emulation was bad. but i guess that was not the root cause, so i need to do some proper profiling

[Vampire]

what's the best way to have task X from subproject A, depend on task Y from subproject B.

A direct dependency?
Never.
Well if it really is just a "run that task before this task, *but I don't need any output of it*", then you could do it with dependsOn(":path:to:the:task").
But never ever ever ever ever try to do it like @yawkat did it now by reaching into the project model of other projects.
That is highly discouraged and unsafe, especially when combined with then configuring some output directories as dependencies manually like @yawkat did.
All this is highly fragile, unsafe, and discouraged.

If you need things from another project in the same build, either create a feature variant with that stuff or if it is only build-internal with only an outgoing configuration that is requested explicitly.
If that is properly set up, you depend on that other project and either request the according feature variant or the according configuration and all necessary tasks run automatically when necessary.

See this for more in-depth information: https://docs.gradle.org/current/userguide/cross_project_publications.html

[Edoardo Luppi]

Thanks! To clarify a doubt, does KGP use feature variants for the various platforms it supports?

Edit: looking at the example on the Gradle docs

val instrumentedJars by configurations.creating ...

It looks familiar to what is used in KGP, so I'd say the answer might be yes.
I also recall seeing attributes mentioned when discussing multiple js() targets, to distinguish them

[Vampire]

I think so, yes

[Vampire]

(Btw. totally off-topic, so please don't answer here, but I still hope you one day continue on idea-debugger-enhancer and add Kotlin support and actually make it work with latest IJ versions 😉)