Hey, I am currently looking at adding a few benchmarks to various things (for my stm pr first and later delimited continuations) and I've been wondering if we have a common setup/already set up benchmarks to look at. The

arrow-benchmarks-fx

module in

arrow-fx

seems like a good start but it does not have

fx-coroutines

benchmarks as far as I can tell (do we even have some and if so where?) so I am not sure if that package is still supposed to be used... Also does it make sense to automate running benchmarks and show results in prs if any relevant code changes? (Could be a manual task as well but just having a way to compare performance for a pr and visualize the changes if we have benchmarks set up would be nice). Any ideas? I'd love to do some groundwork here so that adding benchmarks throughout other arrow repos becomes a more straightforward process...

Hi Jannis, yes there is an effort to include benchmarks in the site on going that @Israel Pérez González @Rachel were looking into but there is no common approach to add benchmarks. I’d say for the time being just add them and when Raquel and Isra get that done they can adjust the modules and automate the PR and site aspects of it.

I'll start with stm benchmarks and add them to the benchmarks-fx module for now 👍. I really just want a way to verify optimization ideas and to compare against the scala implementations (cats-stm, stm4cats, zio etc). As far as I am aware those all use a global lock to protect updates, which I think is a terrible idea for a concurrency abstraction and I kind of want some proof of that^^ For reference ghc implements both global and fine grained locks and chooses at compile time and I believe the fine grained one is default for threaded compilation. (Although it is kind of hard to find references to that...) The benchmark and further work on delimited control has to wait on my side anyway as I won't be home for about a week, but I do want to finish up the stm pr as much as possible^^

Hi! Everything is ready!! Just waiting for benchmarks thresholds!! Hood works fine in Arrow and adds the comparison with master branch as a new comment in PRs (and adds a new comment with every new commit on the same PR).