Is there an easy way to output a valid npm module ...

# javascript

[Joel]

Is there an easy way to output a valid npm module from the JS IR compiler?

js(IR) {
    moduleName = project.name
    nodejs()
    binaries.executable()
}

The only relevant documentation I can find are here and here.

There also appears to be a bug when the module name includes a group resulting in an invalid package main path:

[Robert Jaros]

You could check https://github.com/mpetuska/npm-publish

[Joel]

@Robert Jaros good find, thank you!

[Joel]

Looking through that repo, there's an option binaries.library() which produces a folder called productionLibrary. This might be the ticket.

[Joel]

The transitive dependencies are not included. I get the sense that outputting an NPM library for usage in an existing project is not yet possible. 😞

[Robert Jaros]

@Big Chungus

[Joel]

For reference here is the personal project that I am bootstrapping. The idea is to allow the server authors to maintain the JS/TS client as an npm package that's plug-and-play for app developers.

[Big Chungus]

@Joel with my plugin above, you should get your library properly assembled after running gradle assemble

You can find the assemblies in build/publications/npm

[Big Chungus]

You can also run gradle pack to get installable npm tarballs

[Joel]

Thanks @Big Chungus. Your plugin is creating the library but there's an issue with yours and the built-in plugin when using transitive dependencies. ktor-client-websockets depends on a few npm packages such as ws. The IR compiler correctly compiles ktor-client-websockets into the output, but those transitive npm dependencies are not enumerated in package.json using either plugin.

[Big Chungus]

They're not, because they're bundled into js output file via webpack

[Big Chungus]

The plugin only includes dependencies declared as dependency(npm("xxx","1")) into package.json. All kotlin.js dependencies are bundled into js output instead (since they're not on npmjs)

[Joel]

ktor-client-websockets relies on ktor-client-core which does have npm dependencies. See here.

[Big Chungus]

And as such, transitive dependencies of kotlin.js dependencies also get bundled. But you should be able to install packed tarball into your node package regardless

[Joel]

I'm using the nodejs target, let me see if it works using browser which enable webpack

[Big Chungus]

Try installing packed tarball first and see if it works

[Joel]

ok

[Joel]

@Big Chungus no dice 😞

[Joel]

Error: Cannot find module 'ws'

[Joel]

Shall I try browser targer?

[Big Chungus]

Ah, damn it. Give it a go with browser then.

[Big Chungus]

If it still doesn't work, I'd appreciate you raising an issue on that plugin repo for me to look into. Add as much details as you can from your trials.

[Big Chungus]

A small reproduce would also be a massive boost

[Joel]

Will do. I'll link to my project.

[Joel]

https://github.com/mpetuska/npm-publish/issues/5

[Joel]

Note that this is also broken in the plugin provided by JetBrains. If you solve this you should probably notify them and put it on your resume. 🙂

[Big Chungus]

I've offered them a while a go to move my plugin features to kotlin plugins. Unfortunately they're stretched too thin for now to maintain that. I'll sort it out on npm publish over the weekend

[Big Chungus]

Anyways, thanks for pointing this out!

[Joel]

That'd be great. I feel like being able to cleanly publish npm packages will greatly improve the chances of kotlin.js and MMP being adopted, since you can drop these into existing projects.

[Big Chungus]

That's the idea behind the plugin. I'm trying to keep the experience on par with maven-publish.

[Big Chungus]

Btw, the assembly should work fine with legacy mode for you ;)

[Joel]

Need that sweet TypeScript definiton 😞

[Big Chungus]

Hah, yeah, can't argue with that 😀

[Joel]

It feels like there are a bunch of bugs with packaging and IR compiler. The main and types paths are also wrong.

[Big Chungus]

Well type declarations are still experimental. Also JB is now focusing at ir compiler stability over kotlin.js usability as a way of writing js libraries

[Joel]

Makes a lot of sense

[Big Chungus]

@Joel, there you go npm-publish@v1.1.3
Let me know if you still face any issues (you shouldn't)!

[Joel]

@Big Chungus amazing!!! I’ll check this out today and report back. Can’t wait and thank you for fixing so quickly.

[Joel]

@Big Chungus integrated here and working flawlessly. Is there a good way to DRY up the configuration for multi-module projects?

[Big Chungus]

You can do that from root project via allprojects {}

[Big Chungus]

Alternatively use gradle conventional plugins

[Big Chungus]

https://docs.gradle.org/current/samples/sample_convention_plugins.html

[Big Chungus]

Here's an example of allprojects way https://www.github.com/mpetuska/template-kmp-library/tree/master/build.gradle.kts

[Joel]

Very cool, makes sense. I try to limit plugin application to the applicable modules, so my root is unaware of the npmPublishing extension. I figured there may be an incantation like extensionWithType<NpmPublishingExtension> but no cigar.

[Big Chungus]

In that case use conventional plugins

[Joel]

That's a heck of a config

[Big Chungus]

Yeah, multiplatform libs targeting ALL kotlin targets are not easy 😀

[Joel]

Think I'll stay off of native for now 😄

[Joel]

Lots of other good nuggets in here

[Big Chungus]

And you should. I only linked to that to show what you can do with allprojecrs builder

[Joel]

I know a couple libraries that could really benefit from explicit API mode

[Joel]

I must've missed that one in the release notes

[Big Chungus]

It came with 1.4.0 there's also explicitApiWarning() for easier migration

[Joel]

@Big Chungus another one for you 😞

[Big Chungus]

Clean your gradle global and local caches, I have encountered simmilar errors with my own convention plugins before

[Big Chungus]

Also what gradle version are you on?

[Joel]

6.8.3

[Joel]

OK well that problem is gone, now replaced by Unresolved reference: npmPublishing

[Big Chungus]

Can you reopen that issue and drop new console output?

[Big Chungus]

Also post/link to your buildSrc/build.gradle.kts

[Joel]

Done. The code is linked in the original issue.

[Big Chungus]

Did bumping kotlin version fix your issue?

[Joel]

Trying now

[Joel]

It's not happy about something

[Joel]

@Big Chungus I backed out your plugin entirely and added this single line to buildSrc/build.gradle.kts :

implementation("lt.petuska:npm-publish:1.1.3")

[Joel]

FAILURE: Build failed with an exception.

* Where:
Build file '/Users/joel/Code/bank/build.gradle.kts' line: 4

* What went wrong:
Error resolving plugin [id: 'org.jetbrains.kotlin.jvm', version: '1.4.32', apply: false]
> Plugin request for plugin already on the classpath must not include a version

[Big Chungus]

Ah, that's an error with your own setup. Remove kotlin(jvm) and kotlin(multiplatform) plugins from your root build.gradle.kts (they're applied for you somewhere else already

[Joel]

Right, but it builds fine when I remove npm-publish plugin, meaning you're defining my kotlin version and I'm not sure I like that 🙂

[Joel]

Not that I don't trust you, but I want to control my kotlin version

[Big Chungus]

it shouldn't, really.

[Big Chungus]

Just double-checking in code

[Joel]

sec I'll push this branch

[Joel]

Here's the branch, if you back out this line it builds fine

[Big Chungus]

I know what's up

[Big Chungus]

Since you're adding a plugin via convention plugin, it hard chooses transitive dependency versions (in this case kotlin). You can still override it by adding implementation("org.jetbrains.kotlin:kotlin-gradle-plugin:_") to your buildSrc/build.gradle.kts

I guess it's just the way convention plugin dependencies are resolved

[Big Chungus]

While it doesn't implicitly apply those transitive plugins, it adds them to the classpath via buildSrc and as such locks the version from further modifications outside of buildSrc

[Joel]

oh boy

[Big Chungus]

which is kinda nice when you think about it, it allows you to manage ALL versions in buildSrc

[Big Chungus]

As opposed to splitting between Versions object and buildSrc. Add refreshVersions plugin and it'll be even easier

[Joel]

After all that it's still unable to find npmPublishing

[Big Chungus]

push your latest and greatest. I'll have a look

[Joel]

I've never see the cache get corrupted so quickly than with this convention mechanism

[Joel]

Nothing is working right

[Joel]

https://github.com/jnfeinstein/spring-bank/compare/bug/something

[Big Chungus]

can you temporary add me to that repo so i could push back?

[Joel]

Done

[Joel]

joel@computer bank % ./gradlew clean
Watching the file system is not supported on this operating system.
> Task :buildSrc:extractPrecompiledScriptPluginPlugins UP-TO-DATE
> Task :buildSrc:generateExternalPluginSpecBuilders UP-TO-DATE
> Task :buildSrc:compilePluginsBlocks UP-TO-DATE
> Task :buildSrc:generatePrecompiledScriptPluginAccessors UP-TO-DATE
> Task :buildSrc:configurePrecompiledScriptDependenciesResolver
> Task :buildSrc:generateScriptPluginAdapters UP-TO-DATE

> Task :buildSrc:compileKotlin FAILED
The `kotlin-dsl` plugin applied to project ':buildSrc' enables experimental Kotlin compiler features. For more information see <https://docs.gradle.org/6.8.3/userguide/kotlin_dsl.html#sec:kotlin-dsl_plugin>
e: /Users/joel/Code/bank/buildSrc/src/main/kotlin/springbank.npm-conventions.gradle.kts: (5, 1): Unresolved reference: npmPublishing
e: /Users/joel/Code/bank/buildSrc/src/main/kotlin/springbank.npm-conventions.gradle.kts: (6, 5): Unresolved reference: organization
e: /Users/joel/Code/bank/buildSrc/src/main/kotlin/springbank.npm-conventions.gradle.kts: (8, 5): Unresolved reference: publications
e: /Users/joel/Code/bank/buildSrc/src/main/kotlin/springbank.npm-conventions.gradle.kts: (9, 9): Unresolved reference. None of the following candidates is applicable because of receiver type mismatch:
public inline operator fun <K, V> Map<out TypeVariable(K), TypeVariable(V)>.get(key: TypeVariable(K)): TypeVariable(V)? defined in kotlin.collections
public operator fun MatchGroupCollection.get(name: String): MatchGroup? defined in kotlin.text
public operator fun <T : Any> NamedDomainObjectCollection<TypeVariable(T)>.get(name: String): TypeVariable(T) defined in org.gradle.kotlin.dsl
public operator fun ExtensionContainer.get(name: String): Any defined in org.gradle.kotlin.dsl
public inline fun <S : Any, T : SoftwareComponent> BinaryCollection<TypeVariable(T)>.get(type: KClass<TypeVariable(S)>, spec: Spec<in TypeVariable(S)>): BinaryProvider<TypeVariable(S)> defined in org.gradle.kotlin.dsl
e: /Users/joel/Code/bank/buildSrc/src/main/kotlin/springbank.npm-conventions.gradle.kts: (9, 13): No get method providing array access
e: /Users/joel/Code/bank/buildSrc/src/main/kotlin/springbank.npm-conventions.gradle.kts: (10, 13): Unresolved reference: repository
e: /Users/joel/Code/bank/buildSrc/src/main/kotlin/springbank.npm-conventions.gradle.kts: (11, 17): Unresolved reference: url
e: /Users/joel/Code/bank/buildSrc/src/main/kotlin/springbank.npm-conventions.gradle.kts: (17, 9): Unresolved reference: repository
e: /Users/joel/Code/bank/buildSrc/src/main/kotlin/springbank.npm-conventions.gradle.kts: (18, 13): Unresolved reference: registry
e: /Users/joel/Code/bank/buildSrc/src/main/kotlin/springbank.npm-conventions.gradle.kts: (19, 13): Unresolved reference: authToken

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':buildSrc:compileKotlin'.
> Compilation error. See log for more details

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at <https://help.gradle.org>

BUILD FAILED in 729ms

[Big Chungus]

There you go https://github.com/jnfeinstein/spring-bank/pull/1

[Big Chungus]

Apparently there's no project.properties during conventional plugins resolution... Had to refactor into using it the ugly way 😄

[Joel]

Ah so there was an issue in the plug-in?

[Joel]

Did the linter change the spacing on your PR? I thought it used four spaces.

[Big Chungus]

Might've been my ide, apologies. Rerun it on that branch to fix.

[Big Chungus]

And the issue was actually gradle bug, will report it tomorrow. My plugin simply surfaced that bug 😀

[Joel]

Less than ideal. I think we’re certainly at the bleeding edge right now.

[Joel]

Thanks again for handling this. It’s a very cool way of doing business.

[Big Chungus]

The bug is irrelevant to the npm-publish plugin now as I've implemented a more verbose workaround. Also it only relates to convention plugins.

[Big Chungus]

In any case, happy to fix the issues with the plugin. Thanks for such detailed reports!