On the parMap/parTupled topic... I did 2 PRs and m...
# arrow-contributors
s
On the parMap/parTupled topic... I did 2 PRs and most of the work is updating the test after a copy paste 🙈 Could we do the arity 10, use that fun as an oracle and express lower arity test in terms of that? Cc @simon.vergauwen
s
Sorry for the late response, the reason we're not doing that is because the implementations are so simple. Deriving it from the highest arity would mean that for arity 2 we're still scheduling 10 tasks.
s
I ment for tests only....
j
Looking at the your latest pr, it should be possible to write an abstract test for parMap/parTupled which works with n-ary implementations... But yeah having something n-ary is definitely preferred over copy pasta in tests as that becomes annoying if the functions do change at some point
I'll add me a reminder to think about this some more and maybe work something out when I have time, but in the meantime if you have a good idea @stojan go ahead this could reduce duplicate/boilerplate in the tests quite a bit and may even be applicable to other case where we provide n-ary methods
👍 2