Hi, I’m trying to create a KMP module for audio pr...

# javascript

[Ori Spokoini]

Hi, I’m trying to create a KMP module for audio processing and want to split the processor code into a separate JS file, similar to this example:
// script.js
await audioContext.audioWorklet.addModule("hiss-generator.js");
// hiss-generator.js
registerProcessor("hiss-generator", HissGeneratorProcessor);
I tried to creating the processor in another module with:

// processor/build.gradle.kts
commonWebpackConfig {
    outputFileName = "audio-processor.js"
}

But when I add it as a dependency to the app, audio-processor.js isn’t included in the dist output.
What am I missing?

[turansky]

Do you use Kotlin Warappers?

[Ori Spokoini]

yep, currently I do it like in this example:

context.audioWorklet.addModule(blobUrl)

where the blobUrl is raw JS string, but I’d prefer to write it in Kotlin.

[turansky]

We already support Worker creation without separate subproject creation - example.

[turansky]

AFAIK Webpack doesn't support worklet modules without loaders 😞

[turansky]

We can support worklet modules, but they will work only with Vite bundler (recommended).

[turansky]

Could you please create issue?
I will add example of worklet creation

[Ori Spokoini]

sure, Thanks! So, what do you think is the best approach right now? Do you have any examples on how to set it up?

[turansky]

It will be something like this:

// MyAydioWorkletModule.kt
val MyAydioWorkletModule = AudioWorkletModule {
    // this - AudioWorkletGlobalScope
}

// App.kt
audioContext.audioWorklet.addModule(MyAydioWorkletModule)

All additional magic Vite and Seskar will do 😉

[Ori Spokoini]

Too good to be true 😅.
BTW the issue has been created.

[turansky]

Please add links on JS examples in issue

[Ori Spokoini]

I saw AudioWorkletModule was merged and released — is it ready to go?

[turansky]

I added MDN example here.
Compiler plugin support - next (final) step.
Hope it will be ready this week 😉

[turansky]

In any case I will write in this thread when it will be ready.

[turansky]

Example in Seskar works!

[turansky]

Plugin will be released later

[turansky]

Plugin released.

[dwursteisen]

👋 I configured Seskar on one of my project to use an audio worklet.
Using ./gradlew jsViteDev , it's working: the worklet is loaded in the browser and it's working as expected. (it's loading the *.mjs audio module)

But I was not able to make it work using the task jsBundleProduction : the audio worklet is not created. When updating the vite configuration to use the Audio Worklet as entry point

rollupOptions: {
            input: {
                // HTML entry point - relative to root (kotlin directory)
                main: resolve(__dirname, 'kotlin/index.html'),
                // Audio worklet entry point (will be bundled separately)
                'audio-worklet': resolve(__dirname, 'kotlin/HissGeneratorWorkletModule__worklet__module.mjs')
            },
}

It's not working also as it generated a base64 string of the mjs file (without resolving its dependencies.)

I tried the same thing using this test from seskar and use the same command: the audio worklet is not generated also using ./gradlew jsBundleProduction. (see build/dist/js/production)

is the audio worklet can be bundled using vite and ./gradlew jsBundleProduction?
I do believe that it's a configuration issue with vite so it can generate the right bundle with the audio worklet. But as a fresh user of vite, I'm bit lost at the moment. Could you point me what to do to fix this issue?

thanks 🙂

[turansky]

Do you use NPM dependencies inside worklet?

[dwursteisen]

i don't use any npm dependencies in my project.
If I decode the base64 string from the generated audio worklet javascript file,

I can see import of mjs files: (example below)

import { Unit_instance1fbcbse1fwigr as Unit_instance } from '../kotlin-kotlin-stdlib/kotlin/Unit.mjs';

But as files are bundle, there is no mjs to download.

You can reproduce the issue by using this vite.config.mjs with the seskar example.

import { defineConfig } from 'vite'
import { dirname, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'

const __dirname = dirname(fileURLToPath(import.meta.url))

export default defineConfig({
    root: "kotlin",
    build: {
        rollupOptions: {
            input: {
                // HTML entry point - relative to root (kotlin directory)
                 main: resolve(__dirname, 'kotlin/index.html'),
                'audio-worklet': resolve(__dirname, 'kotlin/mdn-audioworklet-example/mdn/audioworklet/example/HissGeneratorWorkletModule__worklet__module.mjs'),
            },
        }
    }
})

By using the command: ./gradlew jsBrowserProductionVite --rerun-tasks,
The audio-worklet.js in seskar/tests/worklets/mdn-audioworklet-example/build/dist/js/production will contains a base64 string to be decoded by the javascript code.

If i'm serving the content of this directory (python3 -m http.server 8000) the page is not working and crash when playing an audio because Failed to resolve module specifier "./HissGeneratorParameters.mjs".

Am I doing something wrong? 🤔

[turansky]

Looks like Vite configuration problem.
It doesn't process audio worklet for some reason.
At the same moment worker processing works as expected.

[dwursteisen]

Thanks! I’ll investigate Vite with audio worklet then :) (wish me luck 😂)

[turansky]

Looks like fake Worker class can be used as workaround

[turansky]

Are you familiar with includeBuild ? 😉

[dwursteisen]

yes. But I would like to avoid as a fix to this issue. (But I'm very ok to test a fix for this issue)

[turansky]

Workaround will be hidden inside plugin ;)

[turansky]

Workaround with Worker add importScripts (and fail worklet)

[turansky]

I pushed "Fake worker" workaround for worklets

[turansky]

Could you please check (with includeBuild) if it work's for you?

[turansky]

https server required for check

[dwursteisen]

I tried to generate the worklet: It seems to work (the js file is created and loaded by the browser)
I need more work on my project to fully test the audio worklet. So I test with the seskar example:
• the worklet is generated
• the worklet is loaded by the browser.
• I put a console.log in the worklet: it didn't show up in the console.
I didn't investigate the generated worklet and would need a bit more time to understand if it's an issue or a mistake from me. (but it's a very good progress!)

What do you mean that a https server is required? I'm using an http server only. With an audio worklet created by myself, in Javascript. it was working with a http only server.

[turansky]

@dwursteisen I released Seskar with required workaround (Vite pack worklet as Worker 😉 )

[turansky]

Single problem for now - Kotlin/JS removes external process method override in production without reason.
cc @Artem Kobzar

[turansky]

But we can add temp workaround 😉

[dwursteisen]

Oh! Thanks! I was in holidays and was thinking to look at it again this week. Will try the new release tomorrow then.

[turansky]

Workaround for production 😜