1. You need shadow (or any other preferred way of controlling includes/excludes) to exclude Spark and Scala from a jar that you're building to use with spark-submit.
2. This may be my bad. The original example was with earlier version of Kotlin, and when I updated the version to 1.4 I didn't take this new feature into account. I'll double check and update the example, if stdlib is not needed.
3. For Jupyter (or any other interactive work) we'll need to release kotlin spark shell. We are working on it, but it's not available in this first preview. Right now self-contained applications are supported.