Hello, is there a way to run something like „yarn ...

# javascript

[Florent Martin]

Hello, is there a way to run something like „yarn cache clean” via gradle in Kotlin MPP?

Background: we built a web-multiplatform application which use backend-, core- and frontend-packages.
Because we needed another package written in typescript, we also now have a typescript-package used as a library.
This typescript-package do not use kotlin but use our core-js-distribution as a local dependency and is also used by the kotlinJS-package as a local-dependency (frontend-package dependant on typescript-package dependant on core-package, all used as npm-local-dependencies).

We struggled but we actually managed to make it work. Our actual problem is the yarn-cache which won’t always update even if the typescript-package-version changed.
Is there a way (with something like „kotlinClearYarnCache”) to force a „yarn cache clean” in our Gradle-Tasks? We currently have to remove everything for that („gradle clean”)
Or is it eventually possible to tell grade to first build only our core-js package without also building the frontend-js?

[Artem Kobzar]

@Ilya Goncharov [JB] ^^

[Ilya Goncharov [JB]]

We have no such commands, but you can create them by your own with Exec task type of Gradle.

I am not sure that I understand your setup.
How do you declare NPM dependency in Gradle?
You can use something like with File

implementation(npm(projectDir.resolve("your-package")))

[Florent Martin]

Our setup is a bit complex.

We have a Core-Package (common) which is used by our Kotlin-Backend and our KotlinJS-Frontend and now also by our new typescript-package.

The typescript package is independant and has its distribution built by gradle (using node-gradle-plugin). Before it’s distributed, our Core-kotlin-js-distribution is built (“corejsNodeProductionLibraryDistribution”) in Order to be used as a local npm-module in the typescript library.

This typescript-library is then well implemented in our kotlin-js-frontend via `

implementation(npm(File(pathToLocalNpmModule.absolutePath)))

` in Order to use it (via a kotlin-wrapper) in kotlinJS-Frontend.

We figured out that new changes in the typescript-library won’t be taken by the kotlinJs-Frontend because the frontend seems to use the yarn-cache even if we change the typescript-library-version (package.json) and run the gradle-task “kotlinUpgradeYarnLock”.

The only way to be sure that the kotlinJs-Frontend is up-to-date is to run “gradle clean” (or eventually remove the “node_modules” folder in the kmp-build).

That’s why we think that we need to clear the yarn-cache before building our kotlinJs-Frontend

[Ilya Goncharov [JB]]

Hm, as I know, yarn does not use cache for local dependencies (it does not exist even in yarn.lock file).
After task kotlinNpmInstall task was executed, your library in node_modules should be updated.
kotlinNpmInstall always is up-to-date, so it should aware about changes in local dependencies.

I tried it in some sample, and just created folder with package.json and foo.js
And use

implementation(npm(projectDir.resolve("foo")))

And every change can be seen after task kotlinNpmInstall

[Florent Martin]

Thanks for you reply.

I thought that it is the opposite and that kotlinNpmInstall should not be up-to-date in Order to be run. Btw, I never succeeded in forcing kotlinNpmInstall not to be up-to-date , this is something I actually thought would do the trick. Is there a way?

I changed our implementation for “projectDir.resolve(“foo”)” and it looks better.

[Ilya Goncharov [JB]]

Hm, you always have kotlinNpmInstall with up-to-date state?

[Florent Martin]

Yes.
I thought it’s because running task`"corejsNodeProductionLibraryDistribution"` before already re-install the modules.

[Florent Martin]

@Ilya Goncharov [JB] Do you have any idea why our kotlinNpmInstall is most of the time up-to-date?
Do you need more infos about the setup?
Is there a way to force the kotlinNpmInstall before running frontendProcessResources?

[Florent Martin]

How can I find out where all these tasks have been defined?
Thanks for your time!

[Florent Martin]

PS: I just tested it again, running :kotlinNpmInstall --rerun-tasks before frontendBrowserDevelopmentRun or frontendBrowserProductionRun but this do not update my local typescript-library in node_modules
only a `gradle clean`will do it

[Ilya Goncharov [JB]]

Do you need more infos about the setup?

I think, yes. If it is possible I’d like to take a look on minimal reproducer (like, for example easy JS library - package.json and one js file), and main Kotlin module with build.gradle.kts where you have NPM dependency on this easy JS package

[Ilya Goncharov [JB]]

Oh, nevermind, looks like I can see the problem now. Will try to investigate

[Ilya Goncharov [JB]]

So you are right, actually Yarn indeed caches local dependencies as well.
It is possible to declare this package as a workspace of Yarn. In build/js/package.json we declare workspaces as all Gradle modules, so we could add your NPM package to workspace list.

But I can recommend you to try NPM package manager instead of Yarn. NPM processes file dependency as a symlink while Yarn just copies dependency to node_modules
To try it you can switch property in gradle.properties

kotlin.js.yarn=false

[Florent Martin]

I did a test with this setup and it works as expected, no need to clean anymore. It’s also faster 🚀.
I see that my kotlin-js-store/package-lock,json now have an absolute path for my library `

"packages/myproject-frontend-frontend": {
  "dependencies": {
    "library": "file:/Users/myname/myproject/typescript/library",

`
If I deploy this, I always get an Execution failed for task kotlinStorePackageLock.
Lock file was changed. Run the kotlinUpgradePackageLock task to actualize lock file error
Do you think there is a way to fix that?
Is this the reason why you actually use yarn as dependency manager?
Thank you

[Edoardo Luppi]

@Florent Martin it's probably caused by

implementation(npm(File(pathToLocalNpmModule.absolutePath)))

which resolves to an absolute path. You need a relative one.

[Edoardo Luppi]

Is this the reason why you actually use yarn as dependency manager?

AFAIK the use of Yarn Classic is just historical. npm has been getting better lately and that's why support was implemented.

[Edoardo Luppi]

To avoid issues in very specialized environments, especially in complex monorepos, it would be nice to have a plugin system for package managers so that we're able to customize package resolution, but IIRC @Ilya Goncharov [JB] had mentioned possible extensibility in the future.

[Ilya Goncharov [JB]]

Is this the reason why you actually use yarn as dependency manager?

NPM still have some issues around workspaces usage and stability of package-lock.json ю

it would be nice to have a plugin system for package managers so that we’re able to customize package resolution

So actually it has proto extensibility, because both Yarn and NPM are now supported through common interfaces, and not hard coded into KGP

[Florent Martin]

Unfortunately npm() is expecting in this case a
java.io.File not a string in
implementation(npm(pathToLocalNpmModule)) which I now use.
Do you have any code-example where I can just give the absolute path?

[Florent Martin]

@Ilya Goncharov [JB] You said that “It is possible to declare this package as a workspace of Yarn.” Could you also tell me how to do that? (or send a link if an example exists)

[Ilya Goncharov [JB]]

No ways to do it smoothly via DSL.
Only manual change of file
There is a task rootPackageJson which produces build/js/package.json where all workspaces are declared
So in doLast block you can change content of the task

tasks.named("rootPackageJson").configure {
    doLast {
     // change content of the package.json
    }
}

[Florent Martin]

Thank you, that would be a kind of hack though.

Using npm as dependency-manager,
I still have the problem that implementation(npm(pathToLocalNpmModule)) use an absolute path, which is not ok for my deployment-process.
Do you evtl. know a way to define a relative File path there?

[Ilya Goncharov [JB]]

I think, Kotlin Gradle plugin is responsible on taking absolute path in this scheme (https://github.com/JetBrains/kotlin/blob/71770ee51423d1f34cee14aa5e04bb83c053f3b0/[…]lin/org/jetbrains/kotlin/gradle/targets/js/npm/NpmDependency.kt)

[Edoardo Luppi]

Can that responsibility be moved out to the consumer?

[Edoardo Luppi]

It seems logically more sound to me

[Ilya Goncharov [JB]]

The problem here is that user declares path relative to build.gradle(.kts) file, but package.json is hosted in another place (build/js/<package>/package.json). So anyway we need to align this path, but I think relative path sounds more logical

[Edoardo Luppi]

Are you going to open an issue yourself in YT?

[Ilya Goncharov [JB]]

Feel free to report it, I can of course, but it is better if someone else to report it 🙂

[Edoardo Luppi]

Will do it today

[Florent Martin]

Thanks for your help and interest, give me eventually an update here if you update the directoryNpmDependency

[Edoardo Luppi]

Created https://youtrack.jetbrains.com/issue/KT-74372

[Florent Martin]

@Ilya Goncharov [JB] we actually never managed to catch kotlin-gradle-tasks using:

tasks.named("rootPackageJson").configure {
    doLast {
     // change content of the package.json
    }
}

We always get Task with name 'rootPackageJson' not found in root project .

[Ilya Goncharov [JB]]

It is allowed in scope of NodeJsRootPlugin

rootProject.plugins.withType<org.jetbrains.kotlin.gradle.targets.js.nodejs.NodeJsRootPlugin> {
    rootProject.tasks.named("rootPackageJson").configure { 
        
    }
}

[Florent Martin]

@Edoardo Luppi Can you estimate how long it might take to find a solution & implement it for the issue above?
Do you think I could first use gradle to overwrite the package-lock.json in a task in Order to commit it or will it always be automatically overwritten?
Something like:

rootProject.tasks.named("kotlinStorePackageLock").configure { 
        doLast { 
overwriteMyPackageLockWithRelative
}
    }

[Edoardo Luppi]

That's a question for @Ilya Goncharov [JB]
For a workaround, imo I would never touch the lock file, I would only act strictly on package.json

[Florent Martin]

It would be a workaround until it’s implemented in kotlin.

[Edoardo Luppi]

Yup but as it was suggested by Ilya, you can change the Kotlin-generated package.json file, and not package-lock.json

rootProject.plugins.withType<org.jetbrains.kotlin.gradle.targets.js.nodejs.NodeJsRootPlugin> {
    rootProject.tasks.named("rootPackageJson").configure { 
        doLast { ... }
    }
}

[Florent Martin]

I now succeeded and could overwrite the package.json, so that the package-lock.json also get the right path (relative).
I am using react in my local library and I get a react Error on production TypeError: Cannot read properties of null (reading 'useState') but not during development.
In my custom library, I added react as a peerDependency. I am using vite and it is alsi set as external in my rollupOptions:

rollupOptions: {
            external: [
                'react',
                'react-dom',
                'react/jsx-runtime',

and is also in resolve.dedupe:

resolve: {
    dedupe: ['react', 'react-dom'],

but I keep having the error.
Did you already had this kind of problems @Ilya Goncharov [JB]?

[Florent Martin]

I got back to the yarn-setup.
Adding my package in build/js/package.json workspaces as "../../mylibrary", it doesnt seem to change anything so that I still have to run gradle clean in Order to see the library-changes in kotlinJS.
@Ilya Goncharov [JB] Am I missing something?

[Ilya Goncharov [JB]]

Hm, workspace should produce symlink, so the folder in node_modules should be linked with your library

[Florent Martin]

The folder is linked to the library, but somehow changes are not taken until I restart the frontend.
At least I can see now that I don’t have to run a gradle clean anymore,

[Florent Martin]

I can see that my module is well linked in build/js/node_modules but it is also present in build/js/packages/myfrontend/node_modules/myModule and the frontend use this instead of the symlink.
@Ilya Goncharov [JB] Any idea what could cause that?

[Florent Martin]

I guess I finally understand what was wrong, I was adding the Library/module twice, via implementation(npm(pathToLocalNpmModule)) and as you advised via workspaces. Adding it only via workspaces does the trick

[Edoardo Luppi]

Would be nice to have a small reproducer to point people to if this comes up again

[Florent Martin]

Yes I will try to upload the setup we have in a small repo.

[Florent Martin]

@Edoardo Luppi @Ilya Goncharov [JB]
Here is a repo where you can see how we currently use a Typescript-module in kotlin/JS: https://gitlab.com/florent.martin1/kotlin-kmp-typescript-library-import.git

Do you know if there are plans to move @JsExport to something more stable? We just realized that Experimental means “try it only in toy projects”

[Ilya Goncharov [JB]]

cc @Artem Kobzar

[Artem Kobzar]

@Florent Martin, there is a plan (and even an issue);
We are discussing ways to make it more stable and the Kotlin->JS interop more usable, but I don't have any ETA.
The only thing I can say is that right now, it's on focus, and as soon as we agree on the design of solutions for exporting a wider range of declarations, we will notify everybody about it.

[Edoardo Luppi]

We just realized that Experimental means “try it only in toy projects”

My 2c: basically everyone likes the current JS export model, and everyone I know has been using it by opting in via Gradle for convenience.
There has been some criticism about having to annotate every public declaration in certain cases, but that can be probably solved via KGP's DSL in the future.
I think you shouldn't worry too much about using it.

[Florent Martin]

Thanks for your replies, common-elements as JsExport is definitely very useful.