Hmm I really don't understand why Spark is so cumb...
# datascience
a
Hmm I really don't understand why Spark is so cumbersome. I reduced my sample size to 1000 and it still throws heap space errors. Meanwhile python is doing fine with currently 8000 records