Potential ideas for GSOC 2017

[Sameer Deshmukh]

Hello,

GSOC 2017 is upon us!

The application period for orgs opens on the 19th of January and I will start writing the application for SciRuby as soon as it is up. We've had some great ideas last year and me and Abinash also had the chance to visit the Google office for a splendid mentor's summit.

SciRuby has been a GSOC mentoring organization for the past four years and we have been very consistent with our results throughout, and we want to carry this trend forward.

In order to write a good application, we need good mentors and their ideas. So please go to https://github.com/SciRuby/sciruby/wiki/Google-Summer-of-Code-2017-Ideas and start editing that wiki!

I also have a few in mind, but cannot focus much on mentorship due to my duties as org admin. Here are a few (suggestions welcome):

• Rewrite slow parts of daru in Rubex. This project will involve benchmarking various algorithms in daru, and porting the slowest ones to Rubex for speed. It might also involve improvements to Rubex since it is also in heavy development.
  
• Feather Ruby wrapper. Feather is a lightweight and fast storage system for dataframes.
• Apache Arrow Ruby wrapper. https://arrow.apache.org/
• Performance improvements to NMatrix. One thing that NMatrix needs a SERIOUS overhaul in is speed. Seriously. People talk about how slow NMatrix can be in conferences (on whole slides dedicated this fact) and it makes me real sad to see that sometimes. This should be priority and the mentor should ideally point the student in the right direction by pointing out the slowest algorithms that currently exist in NMatrix.
• Paratext Ruby wrapper. Paratext is a super fast CSV reading library. Having an NMatrix or daru wrapper would be great.
• Port daru internals to NMatrix on MRI/JRuby. This is to reduce daru's dependence on Ruby Array and to take advantage of the speed up that NMatrix would offer. Of course it will require proper benchmaring and adequate research before it can be formulated.
• Improvements to Rubex.
  

This is just me. I'm sure many of you have ideas you'd like to see implemented. Let's pitch them all in and make this one more awesome year for SciRuby!

[Pjotr Prins]

On Mon, Jan 09, 2017 at 10:21:00AM -0800, Sameer Deshmukh wrote:
> * Performance improvements to NMatrix. One thing that NMatrix needs a

> SERIOUS overhaul in is speed. Seriously. People talk about how slow
> NMatrix can be in conferences (on whole slides dedicated this fact)
> and it makes me real sad to see that sometimes. This should be
> priority and the mentor should ideally point the student in the right
> direction by pointing out the slowest algorithms that currently exist
> in NMatrix.

One way to speed up would be GPU integration. Also I believe most
distributions ship suboptimal backend libraries (such as BLAS). GNU
Guix does a better job there because you can add dedicated backends.

Pj.

[Sameer Deshmukh]

By performance I'm talking about basic stuff like slicing and calculating a sum and things like that. We can use NArray as a benchmark.

[Lokesh Sharma]

Some that I have in mind:

1. Ruby interface to Matplotlib like PyPlot is for Julia. This I think it will hugely benefit not only SciRuby but whole Ruby community as there are plotting libraries but none of them are "complete".

2. Make statsample capable of handing other common models used in learning like random forests and others in addition to regression.

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[John Woods]

I think having a Ruby interface for Matplotlib would be a phenomenal project. However, we researched it last year, and found some problems — this is discussed in this Google Group, I believe.

Here are a few emails on the topic from this time last year (I've CCed the authors in case they want to weigh in, as well as other students who expressed interest):

• https://groups.google.com/d/msg/sciruby-dev/fcNGQVdsdhs/YaaeDFOMEAAJ
  
• https://groups.google.com/d/msg/sciruby-dev/p4VCc1O4VCQ/Qoas2NUGFAAJ
  
• https://groups.google.com/d/msg/sciruby-dev/Ozk1n1AO2v4/S70XMqEgEgAJ
  

Caveats aside, if someone has successfully created a matplotlib API for Julia, that would be a proof-of-concept.

As an additional incentive, we've had some money in the bank for a while allocated for student projects, donated by various individuals to the Ruby Science Foundation. If there aren't objections, I'd support allocating that money for follow-on work on Matplotlib — either by a different student or whoever our chosen GSoC student is — to ensure the job gets finished.

John

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev...@googlegroups.com.

[John Woods]

So, tell me if this is consistent with your experiences with NMatrix performance slides at different conferences:

The slow-down occurs mainly when doing element-wise operations (adding two matrices, subtracting, etc.).

Why is this the case?

Element-wise operations make use of iterations and yielding to Ruby operators (e.g. to :+, :-. :/, :==, etc.) across the elements of an NMatrix. Those yields are slow, because each one requires searching the left-hand object's current element for the relevant operator.

Can it even be fixed?

Yes, and it was faster before, but generally at the expense of a Ruby-like interface — and it made compilation of NMatrix unbearably slow. Here's an email where someone asks about this and I explain how this can be done.

There may be another way to achieve a speed-up, but I haven't investigated it before, and some design leg-work would need to be done. It may be that it requires patching Ruby-core.

https://www.igvita.com/2008/12/11/ruby-ast-for-fun-and-profit/

Basically, you'd need to think a lot about the Ruby abstract syntax tree and how it does symbol lookups in a yield.

Is this a problem with NMatrix?

No, not in NMatrix's design, per se. I can't actually think how I would have done this better and still gotten the same amount of functionality. There are a lot of math operators, and getting them all to work in C-land just like they do in Ruby-land would be tricky (and again, slow to compile). Plus, what happens if a user overrides one of these operators in a monkey-patch?

In short, it may be a limitation of Ruby. The question is whether the limitation can be circumvented by some clever patching. The answer: I have no idea.

John

[Isuru Fernando]

Hi,

If you want to go the Julia way, I would suggest implementing an interop library like PyCall.jl, which makes it easier to convert data between Julia and Python using Python's C API and Numpy's ABI and also to call Python functions. PyPlot.jl is built on top of PyCall.jl.

While a Python interop library can be a GSoC project itself, you can also limit the scope to whatever is needed for the matplotlib library and then use it. I'm just putting it out there for anyone interested as this approach wasn't mentioned in the previous discussions.

Regards,

Isuru Fernando

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev+unsubscribe@googlegroups.com.

[prasun]

I agree that to improve performance, we need some design
leg-work.

My guess is that we are not doing elementwise operations in the
Ruby C extension way. So when nmatrix.so is loaded, the MRI
can't optimize the syntax tree. If that was not the case
other operations(#dot, #det) for NMatrix would also have been slow.

I guess using ffi gem for elementwise operations will greatly improve
the speed.

Also, I will be putting some time to benchmark all the function calls
for the c extension part. I will share the results shortly.

Regards,
Prasun

On Tuesday 10 January 2017 09:27 PM, John Woods wrote:
> So, tell me if this is consistent with your experiences with NMatrix
> performance slides at different conferences:
>

> /The slow-down occurs mainly when doing element-wise operations (adding
> two matrices, subtracting, etc.)./
> *
> *
> *Why is this the case?*

> Element-wise operations make use of iterations and yielding to Ruby

> operators (e.g. to :+, :-. :/, :==, /etc./) across the elements of an

> NMatrix. Those yields are slow, because each one requires searching the
> left-hand object's current element for the relevant operator.

> *
> *
> *Can it even be fixed?*

>
> Yes, and it was faster before, but generally at the expense of a
> Ruby-like interface — and it made compilation of NMatrix unbearably
> slow. Here's an email where someone asks about this and I explain how
> this can be done.

> <https://groups.google.com/d/msg/sciruby-dev/fz6RRuhfbNY/dhcZqxOBtOUJ>

>
> There may be another way to achieve a speed-up, but I haven't
> investigated it before, and some design leg-work would need to be done.
> It may be that it requires patching Ruby-core.
>
> https://www.igvita.com/2008/12/11/ruby-ast-for-fun-and-profit/
>
> Basically, you'd need to think a lot about the Ruby abstract syntax tree
> and how it does symbol lookups in a yield.
>

> *Is this a problem with NMatrix?*

> No, not in NMatrix's design, per se. I can't actually think how I would
> have done this better and still gotten the same amount of functionality.

> There are a /lot /of math operators, and getting them all to work in

> <mailto:sciruby-dev%2Bunsu...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.
>

> --
> You received this message because you are subscribed to the Google
> Groups "SciRuby Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to sciruby-dev...@googlegroups.com

> <mailto:sciruby-dev...@googlegroups.com>.

[Kenta Murata]

> 1. Ruby interface to Matplotlib like PyPlot is for Julia. This I think it will hugely benefit not only SciRuby but whole Ruby community as there are plotting libraries but none of them are "complete".

I'm working on this by creating pycall.gem and matplotlib.gem as my business in individual contract with ITOC.

This contract will be finished on March, and I'll release these gem by April.

2017年1月10日(火) 21:09 Lokesh Sharma <lokeshh...@gmail.com>:

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev...@googlegroups.com.

[Lokesh Sharma]

Great!

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev+unsubscribe@googlegroups.com.

[Sameer Deshmukh]

Dear Kenta,

Can you accommodate this gem in a GSOC 2017 idea? You will get a potential future contributor to the gem and someone to help you out with the implementation.

[Abinash Meher]

Hello everyone,

I was looking for some GPGPU libraries in Ruby. Seems like there are two. Barracuda and sgc-ruby-cuda[1]. sgc-ruby-cuda isn't maintained anymore and Barracuda only supports OpenCL on OSX 10.6. Will it be appropriate to have such a library under SciRuby? It will be good to have such a library since it will make it easier for other gems to leverage the power of GPU in performance critical applications.

[1] https://groups.google.com/forum/#!topic/singapore-rb/CyqM-Fjrp4w

A paper that I found might be useful later: http://prg.is.titech.ac.jp/members/masuhara/papers/ramse2012.pdf

Kappa is another framework with ruby bindings. I failed to find any open source repository for that.

Regards,

Abinash Meher

On Wed, Jan 11, 2017 at 10:06 AM, Kenta Murata <mr...@mrkn.jp> wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev+unsubscribe@googlegroups.com.

[Kenta Murata]

Of course I'll do so.

I'll try to have time to describe the project for GSoC I this weekend.

Even if I cannot do it in this weekend due to childcare, I can do until February at the least.

2017年1月11日(水) 18:11 Sameer Deshmukh <sameer.d...@gmail.com>:

[Sameer Deshmukh]

Prasun has worked on a gem called ArrayFire that also uses GPU computing. You might want to get in touch with him about this.

Regards,

Abinash Meher

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev...@googlegroups.com.

[Abinash Meher]

Thanks! I remember seeing it in an email he sent to the group. I used to think it is a domain specific library but it turns out is a general-purpose one. The repo says the ruby bindings are a work in progress. This should be the way to go should we decide to have a project on GPGPU in GSoC.

Regards,

Abinash Meher

On Wed, Jan 11, 2017 at 3:20 PM, Sameer Deshmukh <sameer.d...@gmail.com> wrote:

Prasun has worked on a gem called ArrayFire that also uses GPU computing. You might want to get in touch with him about this.

On Wednesday, January 11, 2017 at 2:46:57 PM UTC+5:30, Abinash Meher wrote:

Hello everyone,

I was looking for some GPGPU libraries in Ruby. Seems like there are two. Barracuda and sgc-ruby-cuda[1]. sgc-ruby-cuda isn't maintained anymore and Barracuda only supports OpenCL on OSX 10.6. Will it be appropriate to have such a library under SciRuby? It will be good to have such a library since it will make it easier for other gems to leverage the power of GPU in performance critical applications.

[1] https://groups.google.com/forum/#!topic/singapore-rb/CyqM-Fjrp4w

A paper that I found might be useful later: http://prg.is.titech.ac.jp/members/masuhara/papers/ramse2012.pdf

Kappa is another framework with ruby bindings. I failed to find any open source repository for that.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Prasun Anand]

Hi ,

MRI work is mostly done for double dense 2D matrices.

Speed is 1e4 to 1e7 faster than NMatrix.

Can be easily used with narray and nmatrix. 

ArrayFire rb uses jitting which makes it very fast and works with both CUDA and Opencl.

Regards,

Prasun 

[Prasun Anand]

Hi,

I think a general purpose Machine Learning library with off the shelf algorithms like scikit learn would be great to have. Though it won't be a summer's work but it will definitely push a lot of users to adopt Ruby for scientific computing.

Regards,

Prasun

[Sameer Deshmukh]

So if we have good ideas in mind can you all please go ahead and edit the ideas page? More the ideas the better.

Here's the link: https://github.com/SciRuby/sciruby/wiki/Google-Summer-of-Code-2017-Ideas

I think we can reuse some ideas from last year too.

[John Woods]

This is incorrect. The functions are written in C/C++. Let's not guess when we can look at the code. :)

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev...@googlegroups.com.

[Pjotr Prins]

On Fri, Jan 13, 2017 at 01:15:52PM +0000, John Woods wrote:
> This is incorrect. The functions are written in C/C++. Let's not guess
> when we can look at the code. :)

Yeah, we should measure before optimizing also. Early optimizations
are the root of all evil says Knuth ;)

Pj.

[John Woods]

I would strongly discourage a Ruby-Python interface for using matplotlib. It'll be too slow and painful to debug.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev...@googlegroups.com.

[Sameer Deshmukh]

John,

Does the PyPlot.jl package for Julia suffer from speed issues? If not, how did they go about it?

[Zhuanhao Wu]

Hi John, Sameer, Isuru and Kenta,

I have been away from the SciRuby community (and Ruby) for a long time and I am picking up my ruby skills now.

I think I am still interested in the idea of matplotlib.

So generally speaking, one idea is to replace codes written in Python in matplotlib, while still using C++ part of that project (which is suggested by Yuning Zhang in https://groups.google.com/forum/#!msg/sciruby-dev/fcNGQVdsdhs/YaaeDFOMEAAJ).

Another idea would be implementing a Ruby `wrapper' around Python code (which possibly resembles PyPlot.jl/PyCall.jl).

I will research on these 2 ideas more deeply in the next few days.

Thanks,

Zhuanhao Wu

[John Woods]

I don't know the answer to that. I haven't looked at Julia at all. However, we've had numerous experiences here at SciRuby with APIs to packages in other languages, and particularly for plotting big data stuff, we can't afford the overhead.

John

[Isuru Fernando]

Sameer,

On Mon, Jan 16, 2017 at 12:20 AM, Sameer Deshmukh <sameer.d...@gmail.com> wrote:

John,

Does the PyPlot.jl package for Julia suffer from speed issues? If not, how did they go about it?

No, it doesn't. Julia loads the python dynamic library and ccall into that. Therefore, calls by the CPython interpreter and the calls by the Julia JIT compiler are basically the same.


Isuru Fernando

[Sameer Deshmukh]

Isuru,

Then theoretically it should be possible to make something like that with Ruby right?

[Isuru Fernando]

Sameer,

Yes, but there'll be overheads of using Ruby as opposed to Julia's JIT compiler.

To give you more information on how data communication happens, say you have a NMatrix double type array you want to pass to matplotlib, what should happen is,

Step 1. Extract a C pointer to the data from NMatrix.
Step 2. Create a numpy object that wraps this C pointer using NumPy's C API. (No memory copy done here.)
Step 3. Call matplotlib function with this numpy object.

In Julia, Step 1 can be avoided since you work with C pointer.
In Ruby and Julia, both steps 1 & 2 can be avoided, if we work directly with NumPy arrays. In PyCall.jl, you can directly create a NumPy array (step 3 only) or use Julia arrays to create one. (steps 2 & 3 only)

Does that make sense? Let me know if you have any more questions.


Regards,

Isuru Fernando

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev+unsubscribe@googlegroups.com.

[Pjotr Prins]

Binding python lib against Ruby is fraught with problems.

> --
> You received this message because you are subscribed to the Google Groups
> "SciRuby Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to [2]sciruby-dev...@googlegroups.com.
> For more options, visit [3]https://groups.google.com/d/optout.
>
> References
>
> Visible links
> 1. javascript:
> 2. mailto:sciruby-dev...@googlegroups.com
> 3. https://groups.google.com/d/optout


--

[Sameer Deshmukh]

Isuru,

So if we make something like PyCall.jl for Ruby, we can theoretically get over the need to create Python data types from Ruby right? And even if we must create Python data, maybe a gem like Pycall.gem can abstract that away from the programmer?

Pjotr,

So what you're saying is true, but given the fact that Python libraries are atleast 5-10 years ahead of Ruby, to me it does not make sense to repeat the exact same effort that they have already put in.

Take the simple example of daru. Daru is now two years old but pandas still consistently beats it at features and performance simply because its been around for a longer time. Or even NMatrix or statsample for that matter. Numpy or statsmodels are better equipped to solve a wider variety of problems than their Ruby counterparts.

Because of this, Ruby programmers wanting to use scientific libraries still have to migrate to other languages. Also, we are not able to leverage the effort put in by the Python people in spite of all the code being open source and the community being open to suggestions.

Therefore, here's what I propose: we can have two projects this summer - one that can be mentored by Alexej or Will Levine (they are  yet to confirm this) that will build a machine learning library similar to scikit-learn from the ground up in Ruby using the traditional approach of writing Ruby code and extending with C extensions, and the second project that can be mentored by Kenta Murata (he has confirmed) that will build a similar machine learning library by building on top of Python (by using Kenta's pycall.gem Ruby gem).

I think we can get a much better perspective on the correct approach once both these projects reach a logical point of conclusion and then we can focus on the more appropriate approach based on ease of use, speed and maintainability.

I say dedicate this summer to experimentation and find out the best approach to solving the problem of Ruby's lack of  scientific libraries. What do you think?

[Pjotr Prins]

On Wed, Jan 25, 2017 at 04:56:00AM -0800, Sameer Deshmukh wrote:
> Isuru,
>
> So if we make something like PyCall.jl for Ruby, we can theoretically get
> over the need to create Python data types from Ruby right? And even if we
> must create Python data, maybe a gem like Pycall.gem can abstract that
> away from the programmer?
>
> Pjotr,
>
> So what you're saying is true, but given the fact that Python libraries
> are atleast 5-10 years ahead of Ruby, to me it does not make sense to
> repeat the exact same effort that they have already put in.

Binding has been tried before - and only works on old versions of
both. Most of the trouble arises around threads and garbage
collection. If you want Ruby to communicate with Python I suggest
using some message paradigm.

PJ.

[Sameer Deshmukh]

Hmmm ok we can consider that too. Kenta, will it possible to briefly tell us what your approach to PyCall.gem will be like? Might clear some air here.

[Kenta Murata]

Python uses reference counting while Ruby uses mark-and-sweep.

I consider that both can be interoperable, but I haven't performed any experiments, yet.

So I want to know what the troubles especially around GC you've been experienced beforehand.

Could you please tell me such knowledge if you can remind them?

2017年1月25日(水) 22:14 Pjotr Prins <pjotr...@gmail.com>:

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev...@googlegroups.com.

[Pjotr Prins]

google. Not meant as an insult, but it is some time ago that I looked
into this. It is non-trivial.

On Wed, Jan 25, 2017 at 01:55:48PM +0000, Kenta Murata wrote:
> Python uses reference counting while Ruby uses mark-and-sweep.
> I consider that both can be interoperable, but I haven't performed any
> experiments, yet.
> So I want to know what the troubles especially around GC you've been
> experienced beforehand.
> Could you please tell me such knowledge if you can remind them?

> 2017年1月25日(水) 22:14 Pjotr Prins <[1]pjotr...@gmail.com>:

>
> On Wed, Jan 25, 2017 at 04:56:00AM -0800, Sameer Deshmukh wrote:
> >    Isuru,
> >
> >    So if we make something like PyCall.jl for Ruby, we can
> theoretically get
> >    over the need to create Python data types from Ruby right? And even
> if we
> >    must create Python data, maybe a gem like Pycall.gem can abstract
> that
> >    away from the programmer?
> >
> >    Pjotr,
> >
> >    So what you're saying is true, but given the fact that Python
> libraries
> >    are atleast 5-10 years ahead of Ruby, to me it does not make sense
> to
> >    repeat the exact same effort that they have already put in.
>
> Binding has been tried before - and only works on old versions of
> both. Most of the trouble arises around threads and garbage
> collection. If you want Ruby to communicate with Python I suggest
> using some message paradigm.
>
> PJ.
>
> --
> You received this message because you are subscribed to the Google
> Groups "SciRuby Development" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to [2]sciruby-dev...@googlegroups.com.
> For more options, visit [3]https://groups.google.com/d/optout.

>
> --
> You received this message because you are subscribed to the Google Groups
> "SciRuby Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to [4]sciruby-dev...@googlegroups.com.
> For more options, visit [5]https://groups.google.com/d/optout.
>
> References
>
> Visible links
> 1. mailto:pjotr...@gmail.com
> 2. mailto:sciruby-dev%2Bunsu...@googlegroups.com
> 3. https://groups.google.com/d/optout
> 4. mailto:sciruby-dev...@googlegroups.com
> 5. https://groups.google.com/d/optout

--

[John Woods]

I think messaging is going to have too much overhead.

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev...@googlegroups.com.

[Pjotr Prins]

On Fri, Jan 27, 2017 at 08:40:00PM +0000, John Woods wrote:
> I think messaging is going to have too much overhead.

I agree in principle, though one can be smart about it. If one figure
is one message and the data is passed by reference or file name it can
be acceptable. Basically that is what we did with nyaplot. No bindings
there.

Bindings won't work between Python and Ruby, that is my main message.
But you can prove me wrong ;)

Pj.

> On Wed, Jan 25, 2017 at 7:14 AM Pjotr Prins <[1]pjotr...@gmail.com>

> an email to [2]sciruby-dev...@googlegroups.com.
> For more options, visit [3]https://groups.google.com/d/optout.

>
> --
> You received this message because you are subscribed to the Google Groups
> "SciRuby Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an

[John Woods]

I actually used Nyaplot extensively and found this to be really frustrating for big data. =/

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[John Woods]

So, I want to bump this discussion back up because I think it's really important.

• We really, really need a native plotting library for Ruby which works for big data, and which resembles Matlab and Matplotlib in terms of its interface (people already know this API).
  
  
• Limiting dependencies is an important goal. My boss was trying to run a Ruby script the other day on a brand new Ruby installation, and he asked, "How do I plot these data quickly?" and I couldn't actually give him an easy answer. That makes Ruby unusable in most engineering and science contexts.
  
  
• I think the Ruby Science Foundation would be willing to fund this project post-GSOC since it's clearly too large to just be a GSOC project.
  
  
• I suggest we start it as a GSOC project and finish it under our own steam ($). That means it'd ultimately be a student project, since a student would be designing it with feedback from mentors.
  
  
• It doesn't need to be as complicated as Matplotlib. We can learn a lot from Matplotlib. We could even possibly develop a language-independent plotting API which has interfaces in multiple languages, with Ruby being the first interface.

Thoughts?

John

--

You received this message because you are subscribed to the Google Groups "SciRuby Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sameer Deshmukh]

Works. A native plotting library would be great.

I think the Ruby Association can also fund our efforts if they think we're showing promise.

[Zhuanhao Wu]

Hi John,

Glad to see that you've added matplotlib as an idea in the GSoC page. Actually I'm applying for GSoC as a student this year and hope there's no misunderstandings.

Would you give some tips on how the language-independent idea works? This sounds quite different from just copying codes from matplotlib (converting python to ruby and reserve c++ codes). Suggest that we want to draw pictures through the same process as matplotlib. Currently one way I think of is to put basic elements in matplotlib, such as Artist/Renderer/Axes down to C++ and expose data structures that are easy to wrap in different languages (perhaps SWIG). Another way is that we can use Client/Server mode and/or RPC calls, which might be easier for other languages to call. In this way passing data between server/client could be just some shared memory and hopefully this won't be so slow for even big arrays. But all these ideas, while reducing work needed to port the lib to other languages, mean that the difficult drawing part lies in C++ side.

What do you think?

Zhuanhao Wu

[John Woods]

I think this needs to be straightforward — put basic elements in C++ and expose them, either via SWIG or manually.

Some basic engineering requirements:

1. It needs to have limited external dependencies, which probably rules out the client–server architecture.
2. It needs to quickly produce plots in a very simple GUI.
3. Plots need to be exportable in a variety of formats (SVG, PNG, PDF, etc.).
4. It needs to be similar to the Matlab/Matplotlib plotting APIs.
5. It should be a language-independent API, so it can be easily interfaced from Python, Ruby, etc.
6. It should be able to handle extremely large datasets quickly.

Glad to hear you're interested!

John

[Victor Shepelev]

So, it looks like we are officially accepted? https://summerofcode.withgoogle.com/organizations/5462926128316416/

Hoorraay or something? :)

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev+unsubscribe@googlegroups.com.

[Pjotr Prins]

On Mon, Feb 27, 2017 at 07:10:47PM +0200, Victor Shepelev wrote:
> So, it looks like we are officially

> accepted? [1]https://summerofcode.withgoogle.com/organizations/5462926128316416/
> Hoorraay or something? :)

Indeed, I just got the message :)

Pj.

[Prasun Anand]

Hoorraay!

Congrats! Sameer.

>> email to sciruby-dev...@googlegroups.com.

>> For more options, visit https://groups.google.com/d/optout.
>
>

> --
> You received this message because you are subscribed to the Google Groups
> "SciRuby Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to sciruby-dev...@googlegroups.com.

[Lokesh Sharma]

Great news 

Cheers 

>> email to sciruby-dev+unsubscribe@googlegroups.com.

>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "SciRuby Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to sciruby-dev+unsubscribe@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev+unsubscribe@googlegroups.com.

[Athitya Kumar]

Awesome news! 😁

[John Woods]

Oops, just saw this — after sending my email. :P

On Mon, Feb 27, 2017 at 11:24 AM Athitya Kumar <athity...@gmail.com> wrote:

Awesome news! 😁

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev...@googlegroups.com.

[Ananyo Maiti]

Great news!

ᐧ

On Mon, Feb 27, 2017 at 11:32 PM, John Woods <john.o...@gmail.com> wrote:

Oops, just saw this — after sending my email. :P

On Mon, Feb 27, 2017 at 11:24 AM Athitya Kumar <athity...@gmail.com> wrote:

Awesome news! 😁

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev+unsubscribe@googlegroups.com.

[Abinash Meher]

This might be helpful. http://aosabook.org/en/matplotlib.html It discusses the architecture of Matplotlib and the lessons the developers learnt from it.

Regards,

Abinash Meher

[John Woods]

Good find.

I want to encourage some GSoC students to pursue this project. It's our most important idea this year.