Any suggestions on distributed kotlin web servers(...

# server

[Oğuzhan Soykan]

Any suggestions on distributed kotlin web servers(ktor/spring-boot): select a leader and ensure that only the leader executes this job, if leader fails other server takes the job.
So far the options I’ve checked:
• Distributed Lock
◦ Fencing Tokens
• Implement Raft algorithm in the apps, and make them communicate (little bit hard to grasp actually)
◦ Apache Ratis: https://github.com/apache/ratis
More context:
• my db is mongo (so, distributed locks would live on mongo)
• App is kotlin, spring-boot
What would you recommend?

[Johann Pardanaud]

push your job to a queue, then let one of your processes consume it and run it

there probably is a solution with mongodb but I can't help you with it, FYI I did this with PostgreSQL and SKIP LOCK

[James Richardson]

Johann's suggestion is a good one. Are you sure you really need a leader election? Are you trying to make sure something runs on a single node/once? Maybe if you can explain the problem, other possible solutions could be suggested? What sort of guarantees do you need? What happens in a partition? Etc etc...

[Oğuzhan Soykan]

Thanks for the suggestions.

Two problems actually I have,
• one is distributed task scheduler/executor for fire and forget operations
◦ User wants to execute long-running operation, take the command and execute it later
◦ Command scheduling. Take the command but schedule it 10 min later
◦ Recurring tasks, for example, every 20 min check users eligibility for sending emails
• the second is only one instance runs specific operations
◦ Create a MongoDB ChangeStreams operation and publish the documents to Kafka, from there ElasticSearch indexing. And ensure that this always runs, if leader/runner fails then other takes the stream and resumes from where it was left. (kafka-and elastic part is not relevant with the need, I want just one stream follower in the cluster. There are connectors, like mongo-to-kafka but I don’t want to add too much separate element to the stack due to cost optimization)

[Oğuzhan Soykan]

I am not sure if that gives the idea of the use-case that I want to implement. It is not a new problem nor unique, but when it comes to solution I have a b it hard time navigating in the ecosystem of Kotlin or Java

[JasonB]

you might consider Redis + Redisson for tasks

[Oğuzhan Soykan]

I will look into it @JasonB thanks!

[Oğuzhan Soykan]

I ended up using Zookeeper with Curator worked like a charm for delegating the leader election