Hi everyone! Did anybody manage to do computation ...

# javascript

[Császár Ákos]

Hi everyone!
Did anybody manage to do computation heavy work inside web workers? How did you do it?
I'm trying to put deserialization (kotlinx serialization) work inside a web worker to not block the main thread, and I'm curious if somebody has experience with it or if there is any already existing solution for it. Or maybe somebody knows a better solution for it :D

[David Herman]

Hello! I actually support this in Kobweb (https://kobweb.varabyte.com/docs/concepts/foundation/workers), but give me a bit and I'll try to recall the relevant parts if you want to do this on your own

[David Herman]

Writing a web worker is basically like writing a normal web script, except you don't have access to the DOM (attempting to access window throws an exception)

[David Herman]

There is a global called "self" which in a normal script is type Window (see also: https://developer.mozilla.org/en-US/docs/Web/API/Window/self) but in a worker is type DedicatedWorkerGlobalScope. Which you can use as part of the message passing between scripts.

[David Herman]

Finally, you want to familiarize yourself with the Worker API itself

[David Herman]

So your worker code should be in a different module from your main web script. It should also be set up with KJS in a way that generates a javascript file. In your final site, you should copy that JS file into a directory that will get served by your web host. Then in your main script, you create a Worker instance, passing in the path to that target JS file.

[David Herman]

That is the high level basics -- and based on what you wrote, you may already have done this.

[turansky]

So your worker code should be in a different module from your main web script

If you don't have plugin 😉

[David Herman]

Which plugin?

[turansky]

With Seskar you can write workers in same project.

[turansky]

Example

[turansky]

But it won't work with serialization, because you can't transfer Kotlin classes between Window in Worker

[David Herman]

Oh very interesting, there's another option. How do you set up dependencies in the worker you don't want in the main script?

[turansky]

If it's required in your case you hide Worker logic in separate subproject.

[David Herman]

I'm assuming the plugin ultimately creates two separate JS files?

[David Herman]

One of the issues with Kotlin/JS web workers is that BOTH your main script and the worker script need their own (potentially tree shaken) copy of the Kotlin runtime.

[David Herman]

Does Seskar improve that situation?

[turansky]

It requires per-file compilation

[turansky]

And for worker it creates additional JS file

[David Herman]

Just to finish up my original thoughts, the last thing I wanted to mention is there is a concept called "Transferables", which is a way to send large objects (probably in most realistic cases, a large file or bitmap image) from the main script to your worker script. I can jump into that more if this is relevant to you, but Kobweb handles that here (see also my own concept called Attachments, which is transferables plus a bit of other stuff)

[Császár Ákos]

Thank you @David Herman I will check!

That is the high level basics -- and based on what you wrote, you may already have done this.

Yes I have the same approach but struggling a little bit with the copy-ing part part since I don't have much experience with web. But its good to know that I'm on a good track then.
It was also a question for me if I will be able to pass deserialized kotlin objects or types from/to web workers but as I understood, its possible.

[David Herman]

Look up Transferable and Clonable docs on MDN, it should give you the idea of the kind of raw objects you are allowed to pass between main script - workers

[David Herman]

Good luck and feel free to ask more questions if you run into any!

[Császár Ákos]

Thanks! Will do! :D

[Tóth István Zoltán]

How much data do you plan to deserialize (or serialize)? I'm just curious about the need to move it into a worker. Another solution would be to use a binary protocol such as protobuf.

[Császár Ákos]

@Tóth István Zoltán We have huge backends sometimes with big responses that need to be deserialized, and with old devices it can take a relatively lot of time, which affects startup time. Even on Android we can feel that deserialization is computation-heavy for our devices.

[Császár Ákos]

Fortunatelly on others platforms we have more threads 😄

[Tóth István Zoltán]

With JSON it can be problematic especially if you use datetime classes without optimization. I send out histories with tens of thousands of records and it is still quite fast. The difference between Json and protobuf is immense.

[Császár Ákos]

Agree. But isn't protobuf hard to debug? in case of deserializing BE response I think it can be a nightmare, no?

[Császár Ákos]

Anyway, for us its not a viable option

[Tóth István Zoltán]

I’ve never had any problems with it.

[Tóth István Zoltán]

You could try to use custom serializers if you use kotlinx.serialization. They can be much-much faster than the default ones.