I see…

# datascience

[bjonnh]

I see…

[altavir]

In fact, our team studied a lot of formats for convenien hierarchic data storage. HDF5 was one of primary candidates, but was discarded due to its complexity. You will never guess what we ended up with.

[altavir]

🥁

[altavir]

Zipped file-tree.

[bjonnh]

yeah lot of people are doing that now

[bjonnh]

https://en.wikipedia.org/wiki/BagIt

[bjonnh]

that's what I have seen people using

[altavir]

Yeah, we are doing something very similar in dataforge right now. With more advanced meta processing.

[bjonnh]

is that public?