Daru wrapper over apache arrow

[Sameer Deshmukh]

All:

The latest Apache Arrow includes bindings for Ruby. This is an ideal time to create a brand new daru extension called daru-arrow that will use the columnar database of arrow for lightning fast data analysis at minimum overhead.

I propose making a new extension that will monkey patch the Ruby daru library when a user does a `require “daru/arrow”`.

Thoughts?

Links : https://arrow.apache.org/
https://twitter.com/ktou/status/999830445062672384

Regards,
Sameer Deshmukh

[Prasun Anand]

This would be super cool :) .

Prasun

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rohit Ner]

Hey Sameer,

Thanks for the news. We can add a wrapper for the same in Daru::Accessors module. WDYT? I am ready to take this up.

Best

Rohit

[Kenta Murata]

Apache Arrow doesn’t have any data-frame operations, currently.

It’s just a in-memory columnar storage library.

When you try to make daru to support Apache Arrow, there are two options:

(1) Write Apache Arrow storage wrapper, and implement operations in daru side

(2) Contribute to Apache Arrow so that it has data-frame operations that can be used in production, and then make daru as a wrapper over Apache Arrow

I asked @ktou, Apache Arrow PMC and Red Data Tools organizer, about the above, and he said that only (2) is recommenadble and (1) is worst.  If daru team will contribute to Apache Arrow, people may rethink Rubyists' attitude to data science.

My opinion is same as @ktou.

Regards,
Kenta Murata

[Prasun Anand]

I agree with Kenta and @ktou.

Regards,
Prasun

[Sameer Deshmukh]

Hmmmm works. Lets go with the second approach and get things straight one by one.