Hello <@UL1A5BA2X>, I have been battling with `npm...

# javascript

[andylamax]

Hello @Big Chungus, I have been battling with npm-publish with no avail. The plugin works well on my machine to publish on ithub packages. But fails on github ci. Have you tested it on CI? Is there an example?

[Big Chungus]

That's odd, are you able to share ci run?

[Big Chungus]

What error are you getting? How is your CI configured?

[andylamax]

npmPublishing {
    repositories {
        repository("github") {
            registry = uri("<https://npm.pkg.github.com>")
            authToken = System.getenv("GH_TOKEN")
        }
    }

    publications {
        val js by getting {
            organization = "picortex"
            version = "${project.version}-rc.7"
            moduleName = "pi-monitor-client-sdk"
            readme = file("README.md")
//            files { from(".npmrc") }
            packageJson {
                repository {
                    type = "git"
                    url = "<https://github.com/picortex/monitor-client.git>"
                }
            }
        }
    }
}

[Big Chungus]

I suspect it might be failing due to a fact that gh actions use jdk 8 by default (plugin is compiled against jdk 11)

[andylamax]

That how it is configured. I am getting this error

> Task :pi-monitor-client-sdk:publishJsNpmPublicationToGithub FAILED
npm notice 
npm notice 📦  @picortex/pi-monitor-client-sdk@0.0.15-rc.7
npm notice === Tarball Contents === 
npm notice 547.3kB pi-monitor-client-sdk.js  
npm notice 256B    package.json              
npm notice 355B    README.md                 
npm notice 9.9kB   pi-monitor-client-sdk.d.ts
npm notice === Tarball Details === 
npm notice name:          @picortex/pi-monitor-client-sdk         
npm notice version:       0.0.15-rc.7                             
npm notice package size:  82.3 kB                                 
npm notice unpacked size: 557.8 kB                                
npm notice shasum:        37648fd517250ae6b790e01e283510a33f751e0e
npm notice integrity:     sha512-SMeG/qRA/ZnKk[...]EbC7Zp7a0QsCg==
npm notice total files:   4                                       
npm notice 
npm ERR! code E404
npm ERR! 404 Not Found - PUT <https://npm.pkg.github.com/@picortex%2fpi-monitor-client-sdk> - The expected resource was not found.
npm ERR! 404 
npm ERR! 404  '@picortex/pi-monitor-client-sdk@0.0.15-rc.7' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
npm ERR! 404 
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/runner/.npm/_logs/2021-08-25T13_02_19_085Z-debug.log

FAILURE: Build failed with an exception.

[Big Chungus]

And you said you did try to publish this exact config from your local?

[andylamax]

yes, and it worked

[Big Chungus]

Is the project open source so I could have a look?

[andylamax]

If you are familiar with github actions, I even tried this snippet, but still got the same error

- name: Setup per user .npmrc
        uses: actions/setup-node@v2
        with:
          node-version: '14.x'
          registry-url: '<https://npm.pkg.github.com>'
          scope: '@picortex'

Sadly, not. The project is not open source. But I can share any configuration file you would ask

[Big Chungus]

Ah, your repository config is incorrect (no clue why it works locally, it really shouldn't)

[andylamax]

only difference I have noticed with my CI, is that I have logged into to github package registry while it seems that I haven't logged on CI.

[andylamax]

Help me correct my repository config

[Big Chungus]

Registry url should be this https://npm.pkg.github.com/yourOrganizationOrAccount

[andylamax]

setting right away

[andylamax]

Tested on local with new package registry and worked. Going to test on CI now

[andylamax]

Failed again, same error

npm ERR! code E404
npm ERR! 404 Not Found - PUT <https://npm.pkg.github.com/@picortex%2fpi-monitor-client-sdk> - The expected resource was not found.
npm ERR! 404 
npm ERR! 404  '@picortex/pi-monitor-client-sdk@0.0.15-rc.8' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
npm ERR! 404 
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/runner/.npm/_logs/2021-08-25T13_29_00_452Z-debug.log

FAILURE: Build failed with an exception.

[andylamax]

Is there a working sample you have for github package npm registry? Looks like I can't publish without npm login --registry=... on CI

[Big Chungus]

There are a bunch of project floating around that use it, but the only one I recall is kvision-assets. But I'm not sure if Robert is publishing via gh actions.

[Big Chungus]

How are you setting GH_TOKEN in actions?

[Big Chungus]

I suspect it either is not being set or has insufficient permissions

[Big Chungus]

Use {{ github.token }}

[andylamax]

I was setting using my Personall Access Token.
Like so

env:
  GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} -> ${{ github.token }}

Let me change it {{ github.token }}

[Big Chungus]

github.token is especially tailored for gh actions and has all the required perms to publish and pull stuff

[Big Chungus]

FYI, the token needs these permissions to publish to npm

write:packages, read:packages and delete:packages.

[andylamax]

Sadly it is still failing.
The CI Task is configured like that

- name: Publish pi-monitor-client sdk
        env:
          GH_TOKEN: ${{ github.token }}
        run: ./gradlew :pi-monitor-client-sdk:publishJsNpmPublicationToGithub

[Big Chungus]

Hmm, ok give me a minute to setup an action myself

[andylamax]

Tried checking KVision's repo. The main.yml file only runs test.

[andylamax]

Very much appreciate. If it helps, I can create an issue on github. Should I?

[Big Chungus]

Nah, you're missing these (perms) in your GH. actions

[andylamax]

I was surely I was missing them. Already set, lets see what CI will say now

[andylamax]

Still same error 😞

[Robert Jaros]

I'm not publishing my npm package from CI

[Big Chungus]

Almost done with my sample

[Big Chungus]

Bingo! https://github.com/mpetuska/npm-publish/runs/3423237943?check_suite_focus=true

[Big Chungus]

GH packages are VERY restrictive so here are key takeaways:
1. Configure repo as

repository("GitHub") {
      registry = uri("<https://npm.pkg.github.com/>")
}

2. Declare repository in package.json pointing to your github repo where you want packages to be published to

packageJson {
  repository {
      type = "git"
      url = "<https://github.com/mpetuska/npm-publish.git>"
  }
}

3. Set your gradle group or npm.publish.organization property to match your GH repo owner's name

[Big Chungus]

Or just use this template I've setup and just make sure to add #1 & #2 from above

[Big Chungus]

Step #2 is only needed if you npm package name is different than your repo name (e.g. @mpetuska/both vs mpetuska/npm-publish from above action, would work without step #2 if my package name would be @mpetuska/npm-publish)

[andylamax]

My package is different from repo name.

Do I have to use command line arguments as here

./gradlew :${{ github.event.inputs.sandbox-module }}:publishJsNpmPublicationToGitHub \
              -Pnpm.publish.repository.GitHub.authToken="${{ github.token }}" \
              -Pversion=${VERSION/v} \
              -Pgroup=${{ github.repository_owner }}

or is the gradle script fine as well?

[Big Chungus]

No, I just made an action that way so most of the people could just copy/paste it and not have to manage too much in their build.gradle.kts

[Big Chungus]

But you can just as well hard-code all that in buildscript

[Big Chungus]

you can even leave group and version alone and instead set these (or pass the eqvivalent via cli as -Pnpm.publish.<<prop-path>>=XXXX

npmPublishing {
  organization = "GH username"
  version = "default npm package version"
}

[andylamax]

I prefer build script more. How do I set group in the build script? I think it is the only thing missing

[Big Chungus]

See the above 🙂

[andylamax]

But I already had all of that then. My build script looks like this

npmPublishing {
    repositories {
        repository("github") {
            registry = uri("<https://npm.pkg.github.com/picortex>")
            authToken = System.getenv("GH_TOKEN")
        }
    }

    publications {
        val js by getting {
            organization = "picortex"
            version = "${project.version}-rc.9"
            moduleName = "pi-monitor-client-sdk"
            readme = file("README.md")
//            files { from(".npmrc") }
            packageJson {
                repository {
                    type = "git"
                    url = "<https://github.com/picortex/monitor-client.git>"
                }
            }
        }
    }
}

[Big Chungus]

Ah, look like you're all set then

[Big Chungus]

EXCEPT npmPublishing.repositories.github.registry. That should be reverted to uri("<https://npm.pkg.github.com>")

[Big Chungus]

Not a blocker, but I'm wondering why are you including .npmrc file? That's not something you publish normally

[andylamax]

Currently testing on CI.

I was trying to include .npmrc as a way to solve the problem at first. Either way, it wasn't even being published

[Big Chungus]

Ah, must be implicitly ignored by npm

[andylamax]

Still fails on my CI. Trying with command line arguments

[andylamax]

Also, still passes on Local

[Big Chungus]

it passes on local because you're npm login i guess

[andylamax]

I think so too

[Big Chungus]

Send me your build.gradle.kts and gh action yml

[Big Chungus]

(if you can)

[andylamax]

build.gradle.kts

import org.jetbrains.kotlin.gradle.targets.js.nodejs.NodeJsRootExtension
import org.jetbrains.kotlin.gradle.targets.js.nodejs.NodeJsRootPlugin

plugins {
    kotlin("multiplatform")
    id("tz.co.asoft.library")
    id("dev.petuska.npm.publish")
}

rootProject.plugins.withType(NodeJsRootPlugin::class.java) {
    rootProject.the<NodeJsRootExtension>().versions.webpackDevServer.version = "4.0.0"
}

kotlin {
    jvm { library() }

    js(IR) {
        val main by compilations
        main.outputModuleName = "pi-monitor-client-sdk"
        browserLib()
        binaries.library()
    }

    sourceSets {
        val commonMain by getting {
            dependencies {
                api(project(":pi-monitor-core"))
                api(project(":bitframe-client-sdk-test"))
                api(project(":bitframe-client-viewmodels"))
            }
        }
    }
}

npmPublishing {
    repositories {
        repository("github") {
            registry = uri("<https://npm.pkg.github.com>")
            authToken = System.getenv("GH_TOKEN")
        }
    }

    publications {
        val js by getting {
            organization = "picortex"
            version = "${project.version}-rc.9"
            moduleName = "pi-monitor-client-sdk"
            readme = file("README.md")
            packageJson {
                repository {
                    type = "git"
                    url = "<https://github.com/picortex/monitor-client.git>"
                }
            }
        }
    }
}

[andylamax]

workfile.yml

[andylamax]

name: Build & Test

on:
  push:
    branches: [ master ]

env:
  REMOTE_HOST: ${{ secrets.MONITOR_HOST }}
  REMOTE_USER: ${{ secrets.MONITOR_USER }}
  REMOTE_PASS: ${{ secrets.MONITOR_PASS }}
  GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }} // required by setup node action
  REGISTRY: <http://ghcr.io|ghcr.io>
  IMAGE_NAME: ${{ github.repository }}

jobs:
  publish:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write
    steps:
      - uses: actions/checkout@v2
      - name: Set up JDK 16
        uses: actions/setup-java@v1
        with:
          java-version: 16
      - name: Cache Gradle
        uses: actions/cache@v2
        with:
          path: ~/.gradle/caches
          key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*') }}
          restore-keys: |
            ${{ runner.os }}-gradle-
      - name: Making gradlew executable
        run: chmod +x ./gradlew
        working-directory: .
      - name: Cache konan
        uses: actions/cache@v2
        with:
          path: |
            ~/.konan/dependencies
            ~/.konan/kotlin-native-macos*
          key: ${{ runner.os }}-konan-
          restore-keys: ${{ runner.os }}-konan-
      - name: Cache Node Modules
        uses: actions/cache@v2
        with:
          path: 'build/js'
          key: node-modules

      - name: Login into github docker registry
        uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ env.GH_TOKEN }}

      - name: Set Version
        id: versions
        run: |
          ./gradlew :pi-monitor:setVersions
          echo "::set-output name=current::$(cat ./pi-monitor/build/versioning/current.txt)"
          echo "::set-output name=current_safe::$(cat ./pi-monitor/build/versioning/current_safe.txt)"
          echo "::set-output name=previous::$(cat ./pi-monitor/build/versioning/previous.txt)"
          echo "::set-output name=previous_safe::$(cat ./pi-monitor/build/versioning/previous_safe.txt)"

      - name: Setup per user .npmrc
        uses: actions/setup-node@v2
        with:
          node-version: '14.x'
          registry-url: '<https://npm.pkg.github.com>'
          scope: '@picortex'

      - name: Publish pi-monitor-client sdk
        env:
          GH_TOKEN: ${{ github.token }}
        run: ./gradlew :pi-monitor-client-sdk:publishJsNpmPublicationToGithub

      - name: Extract metadata (tags, labels) for Server Docker
        id: metaserver
        uses: docker/metadata-action@98669ae865ea3cffbcbaa878cf57c20bbf1c6c38
        with:
          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
          labels: |
            org.opencontainers.image.title=PiMonitorServer
            org.opencontainers.image.vendor=PiCortex
          tags: |
            type=raw,value=server-${{ steps.versions.outputs.current }}

      - name: Extract metadata (tags, labels) for Server Docker
        id: metaclient
        uses: docker/metadata-action@98669ae865ea3cffbcbaa878cf57c20bbf1c6c38
        with:
          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
          labels: |
            org.opencontainers.image.title=PiMonitorClientBrowserReact
            org.opencontainers.image.vendor=PiCortex
          tags: |
            type=raw,value=client-browser-react-${{ steps.versions.outputs.current }}

      - name: Create Pi Monitor Server Dockerfile
        run: ./gradlew :pi-monitor-server:createDockerfile

      - name: Build and push Server Docker image
        uses: docker/build-push-action@ad44023a93711e3deb337508980b4b5e9bcdc5dc
        with:
          context: ./pi-monitor/pi-monitor-server/build/binaries
          push: true
          tags: ${{ steps.metaserver.outputs.tags }}
          labels: ${{ steps.metaserver.outputs.labels }}

      - name: Create Pi Monitor Browser React Dockerfile
        run: ./gradlew :pi-monitor-client-browser-react:createDockerfile

      - name: Build and push Client Web Docker image
        uses: docker/build-push-action@ad44023a93711e3deb337508980b4b5e9bcdc5dc
        with:
          context: ./pi-monitor/pi-monitor-client/browser/react/build/websites/js
          push: true
          tags: ${{ steps.metaclient.outputs.tags }}
          labels: ${{ steps.metaclient.outputs.labels }}

      - name: Create Pi Monitor Docker Compose Staging File
        run: ./gradlew :pi-monitor:createDockerComposeStagingFile

      - name: Copy Docker-Compose file via ssh password
        uses: appleboy/scp-action@master
        with:
          host: ${{ env.REMOTE_HOST }}
          username: ${{ env.REMOTE_USER }}
          password: ${{ env.REMOTE_PASS }}
          source: "pi-monitor/build/docker/docker-compose-staging.yml"
          rm: true
          target: /apps/pi-monitor/${{ steps.versions.outputs.current }}/staging

      - name: Deploy Staging Stack
        uses: appleboy/ssh-action@master
        with:
          host: ${{ env.REMOTE_HOST }}
          username: ${{ env.REMOTE_USER }}
          password: ${{ env.REMOTE_PASS }}
          script: |
            mkdir /apps/pi-monitor/${{ steps.versions.outputs.current }}/staging -p
            mv /apps/pi-monitor/${{ steps.versions.outputs.current }}/staging/pi-monitor/build/docker/docker-compose-staging.yml /apps/pi-monitor/${{ steps.versions.outputs.current }}/staging/docker-compose.yml
            cd /apps/pi-monitor/${{ steps.versions.outputs.current }}/staging
            rm pi-monitor -rf
            docker-compose pull
            docker stack rm pi-monitor-${{ steps.versions.outputs.previous_safe }}
            docker stack deploy -c docker-compose.yml pi-monitor-${{ steps.versions.outputs.current_safe }}

[andylamax]

I can reduct the irrelevant steps on ci if you'd like

[Big Chungus]

Few things that come to mind:
1. try setting repo as registry = uri("<https://npm.pkg.github.com/>")
2. Remove node-setup step (or move it after npm publish steps

[andylamax]

For number 1, do you mean repositor.url?

[Big Chungus]

No,

npmPublishing {
    repositories {
        repository("github") {
            registry = uri("<https://npm.pkg.github.com/>")
            authToken = System.getenv("GH_TOKEN")
        }
    }
}

[Big Chungus]

Note trailing /. It's stupid, but npm is often like that

[andylamax]

oooooh, I see it now. Setting that as wll

[Big Chungus]

P.S. Usually when debugging pesky things like that with GH actions I like to create new temporary builds that only do the things that don't work for me. Much quicker feedback cycle that way.

[andylamax]

damn it. It was the trailing /

[andylamax]

works now

[Big Chungus]

Told you, npm is stupid like that 😄

[Big Chungus]

Anyways, glad to hear it's working. I'll probably write up a short wiki page on the repo to cover this as it's likely to come back to haunt me again. Or you can do that if you have spare minute 😄

[andylamax]

I have been hustling with thing thing for 3 days now.

My build script now looks like this

registry = uri("<https://npm.pkg.github.com/>") // DO NOT REMOVE THE TRAILING SLASH

[andylamax]

Will surely write a wiki

[Big Chungus]

Should've pinged me after day 1. Always happy to help

[andylamax]

I really should've. Thanks for this amazing plugin man. with the current state of K/JS IR publishing js artifacts to be consumed is starting to make sense.

You have been a life savior today

[Big Chungus]

Yeah, it's a pain right now. Have a look at this and this discussions to get a feel :D

[Big Chungus]

So many layers of complexity:
• npm
• kotlin
◦ ir
◦ legacy
• npm dependencies
• bundled dependencies
• kotlin dependencies
• kotlin.js dependencies
Each with its own quirks...

[andylamax]

😃 I had gone over the first discussion while trying to debug this