A curious article argues that the biggest technica...

# datascience

[holgerbrandl]

A curious article argues that the biggest technical benefit of Python for data science is the Buffer Protocol, which allows having huge chunks of data (NumPy arrays) in the memory and read their content effishently without copying

Would something similar be useful/possible in Kotlin/Native or other targets?

https://jeffknupp.com/blog/2017/09/15/python-is-the-fastest-growing-programming-language-due-to-a-feature-youve-never-heard-of//
Slack Conversation

[holgerbrandl]

@voddan interesting perspective. But jvm arrays can also be shared by reference between libraries, couldn’t they?

[voddan]

@holgerbrandl I think the point is to share parts of the array (like with C pointers)

[holgerbrandl]

isn’t a jvm reference pretty close to a pointer?

[voddan]

The use case they mention is storing an image in a chunk and then allow image processing libraries to read certain regions of this image

[holgerbrandl]

you can certainly do this with libraries such as https://github.com/imglib/imglib2/
which has some array as data-backend and which defines various iterators to traverse it.

[voddan]

Can it provide C interop to work with GPU shaders?

[holgerbrandl]

I guess the reference to the data array can be exposed via JNI, but no clue about GPUs. But I’d not think that python arrays can be gpu-processed without copying.

[Nikky]

KN could certainly interface with C libraries and share pointers, not sure how much KN can do shaders or gpu processing though

[Nikky]

for that you would probably need a C lib

[holgerbrandl]

kn==Kotlin?

[Nikky]

kotlin/native

[voddan]

To add some perspective, here is a pythonist's explanation of what Buffers Protocol is useful for:
"Well, in scientific computing, most of the interesting algorithms are implemented in compiled code: for example, scipy is essentially a set of wrappers around NetLib utilities, which are well-tested implementations of scientific algorithms written mostly in Fortran and C. The ability for Python to natively share data with these compiled libraries is incredibly important for scientific computing. "
from https://jakevdp.github.io/blog/2014/05/05/introduction-to-the-python-buffer-protocol/

[elizarov]

I frankly think that 99.999% of Python users only care about having a concise scripting language (Pyton) and an ecosystem of libraries (NumPy and beyond) that work and solve their needs. What the hell library authors has to go though to write those libs they care less (and it is a hell in Python world despite the claims of the miraculous buffer protocol).

[thomasnield]

I agree with @elizarov. I don't think the implementation of libraries are paid much attention to by most library users. I also think few library users are pushing the limits of the libraries to where they start questioning the implementation.

[louiscad]

Something interesting is that Python is no longer the fastest growing language as seen in GitHub Octoverse… Kotlin is! You can now write headlines like: "Learn machine learning with the fastest growing programming language" 😜