A modern programming language that makes developers happier.

kotlinlang

Hey folks!  Using Datalore or a Jupyter notebook and spark is there anyway to access local files?  I’m getting:
```No FileSystem for scheme "file"
org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "file"```

You can't access files on your computer since DataLore does not know anything about it. You can add files to your datalore workspace though.

I’ve added the file to the workspace in datalore and then I try to read it as follows:
```val equipment = spark.read().json("allEquipment.json" )```
But I still get the same error.  Have you been able to read a file from the workspace using spark?

I would be glad to read the file using standard kotlin.  And if helpful convert to json and feed it into spark.  But I have no idea how to do anything like this.

Not to the workspace. To attached files. It works differently from local jupyter since each notebook has its own isolated environment.

I should probably mention it works with kotlin serialization and I’m able to get it working using the kotlin dataframe.  But I am really hoping I can connect spark.

You should ask <@U0159NT3R9T>, but if you know how to use spark with regular file, notebook is not different.

<@U5F0TT0UX> thank you for mentioning me! But all the experimenting is done by <@U034GU9M803> 
But it looks to me that you don't need to use scheme to access files entirely. But I know nothing about filesystem in datalore

<@UDRLK763S> <@U0159NT3R9T> I would love to see any samples of accessing files using spark on datalore or jupyter notebooks.  My experience with Spark was years ago (when spark was new-ish) in Scala and I set up my own environments.

I am not familiar with spark, but probably something like this should do: <https://spark.apache.org/docs/latest/sql-data-sources-text.html>

Yeah that’s what I’ve been following.  If you look at my example line it follows their example.  :thinking_face:

Oh that's weird, it appears I get
```No FileSystem for scheme "file"
org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "file"```
as well. Thanks for letting us know! Looks like a bug

Okay, I found a fix!
Related to <https://stackoverflow.com/questions/17265002/hadoop-no-filesystem-for-scheme-file/27532248#27532248>
Will be in the next release for sure! Until then, a temporary solution is to restart spark with the right parameters: <https://datalore.jetbrains.com/view/notebook/tMW3yTujL1lv08P4ZzBgtq>

Created an issue for it: <https://github.com/Kotlin/kotlin-spark-api/issues/164>

Thank you so much!  I really appreciate all your help!