This seems like a generic question about desktop apps, rather than being Compose specific, so I'm not sure the folks here will have the relevant expertise.
That said, I was curious and did a bit of Googling. Unfortunately, it was surprisingly hard to find useful answers, but it looks like the basic approach is to register a protocol with the underlying operating system (which is a very platform-specific thing to do) and then have it launch your program (or a wrapper script around your program). The best doc I ran across was this one: https://support.shotgunsoftware.com/hc/en-us/articles/219031308-Launching-applications-using-custom-browser-protocols - but I didn't try it so your mileage may vary.
It is an interesting topic, and if you find (or create) a good solution, I'd love for you to share and please cc me (@jim).