GSoC 2017 Project [Creating the fastest math libraries for Ruby/JRuby by using the GPU through OpenCL and ArrayFire]

[Prasun Anand]

Hi,

I am very grateful to SciRuby Community for selecting my proposal for
the project 'Creating the fastest math libraries for Ruby by using the
GPU through OpenCL and ArrayFire'. The proposal application can be found
here[0]. I congratulate fellow SciRuby students for getting selected.

I am a fifth-year student at BITS Pilani KK Birla Goa Campus, Goa,
India pursuing a dual degree in M.Sc. in Biological Sciences and B.E.
in Chemical Engineering. My programming interests lie in
Bioinformatics, High Performance Computing, Systems Programming and
Compiler Design.

I have been an active SciRuby contributor since last year and was a
GSoC 2016 student with SciRuby[1],[2] and I am very grateful to SciRuby as it
has helped me become a good programmer.

This Summer I would be working on building ArrayFire gem, a GPGPU
library for Ruby as well as JRuby. It will be suitable for any
application that requires numeric processing. What excites me
particularly is that it will create endless possibilities for
scientific computation. My gem will bring with  it the large collection of 

functionality that is already addressed by ArrayFire algorithms for  linear algebra, 

data science, machine learning, computer vision,  computational fluid dynamics, 

bioinformatics, etc.

Looking forward to a great learning experience in the summers.

Regards
Prasun Anand
Github: https://github.com/prasunanand

[0]. https://github.com/prasunanand/resume/wiki/GSoC-2017-proposal 

[1]. http://sciruby.com/blog/2016/10/24/gsoc-2016-port-nmatrix-to-jruby/

[2]. https://github.com/SciRuby/nmatrix/pull/558

[Prasun Anand]

Hi, 

This is a slight modification in the plan for GSoC 2017.

Currently, the work in the field of Ruby Scientific Computing is available mostly for MRI.

Hence, my main focus would be to make ArrayFire completely usable for MRI, making it 

easy for other Rubyists to switch to ArrayFire for fast computations on a GPU backend . I will

be writing blogs and examples every week. Docs will be updated every week.

ArrayFire for JRuby would be a prototype with same functionalities as ArrayFire - MRI but the blogs

and examples may not be as rich as that of ArrayFire-MRI.

Plan:

Week 1 - Week2 -mid (May 30 - June 10)

Make ArrayFire-MRI ready with tests

Strides and Enumerators

Helper Functions

Week 2-mid - mid to Week 3 (May 30 - June 19)

CTC 2017

(I will be attending Complex Trait Community Conference.

So, I would not be able to work in this period.)

Week 3-mid to Week 4 (June 20 - June 28 )

Implement the following functions using ArrayFire C++ APIs.

Arithmetic operations: +,-, *, /, >>, <<

Exponential and logarithmic functions: exp, log, expm1, log1p, etc.

Hyperbolic functions: sinh, cosh, tanh, etc.

Logical operations: &&, ||, |, &, <, >, <=, >=,==, !

Numeric functions: floor, round, min, max, etc.

Trigonometric functions: sin, cos, tan, etc.

Af_Array#+ has been implemented as:

First - Evaluation Deliverables

Linear Algebra Support for N-dimensional matrices using ArrayFire on MRI

Linear Algebra Support for 2-dimensional matrices using ArrayFire on JRuby

Week 5 (June 30 - July 7)

Start implementing ArrayFire-JRuby

Creation of Af_Array

ArrayFire-BLAS

Week 6 (July 7 - June 14)

ArrayFire- JRuby LAPACK functionalities

Week 7 (July 14 - July 21)

Strides and Enumerators

Helper Functions

Week 8 (July 21 - July 28)

Mathematical Functions

Implement the following functions using ArrayFire Java.

Arithmetic operations: +,-, *, /, >>, <<

Exponential and logarithmic functions: exp, log, expm1, log1p, etc.

Hyperbolic functions: sinh, cosh, tanh, etc.

Logical operations: &&, ||, |, &, <, >, <=, >=,==, !

Numeric functions: floor, round, min, max, etc.

Trigonometric functions: sin, cos, tan, etc.

Af_Array#+ would be implemented as:

Second Evaluation July 24 - 28, 2017 - Deliverables

Linear Algebra Support for N-dimensional matrices using ArrayFire on JRuby

NMatrix integrated with ArrayFire with maximum test coverage for dense stype and double dtype.

Implement Statistics and Reduction APIs for vectors

Week 9 ( July 29 - August 6)

Interface ArrayFire with NMatrix

Week 10 (July 7 - August 14)

Interface ArrayFire with mixed_models

Week 11 (August 14 - August 21 )

Searching and Sorting

Implementing Basic Statistics functions, like corrcoef, cov, mean

Week 12( August 21 - August 30 )

Release docs for the ArrayFire gem.

Write blog posts explaining how to utilize the ArrayFire gem for High Performance Computing. This will involve writing iruby notebooks.

Final - Evaluation Deliverables

ArrayFire gem for Ruby supported on MRI as well as JRuby.

NMatrix integrated with ArrayFire.

Regards,

Prasun

[Prasun Anand]

Hi,

I presented my poster at CTC 2017 which can be found here:

https://github.com/prasunanand/resume/blob/master/CTC_2017_Poster_Faster_LMM_D.pdf

It was a great experience and it taught me the importance of scientific software in academia and research.

Now, I would be working at full pace to keep up with GSoC deliverables.

I have written two blogs which summarises my work till now:

1. http://www.prasunanand.com/arrayfire/2017/06/23/gsoc17-arrayfire-ruby-bindings-part-1-installation.html

2. http://www.prasunanand.com/ruby-c-extensions/2017/06/23/gsoc17-ruby-c-extensions-for-complex-projects.html

Please review.

One more blog will be there in 4 days as I get more code in.

Regards,

Prasun

[Sameer Deshmukh]

My apologies for being inactive. Can you update us on further progress and any new blogs?

Regards,

Sameer Deshmukh

[Sameer Deshmukh]

Present blogs look good.

Regards,

Sameer Deshmukh

[Prasun Anand]

Hi,

I have been making up for the time I was absent.

ArrayFire bindings for most of the functions are functional. I am currently working on

tests and docs now.

New blog post:

http://www.prasunanand.com/arrayfire/2017/07/04/gsoc17-arrayfire-ruby-bindings-part-2-af_array.html

Regards,

Prasun

[Alexej Gossmann]

Hi Prasun!

I ran the code from your blog posts on an AWS p2.xlarge instance, and it seems to work fine:

[2] pry(main)> ArrayFire::Device.info()

ArrayFire v3.2.2 (OpenCL, 64-bit Linux, build )

[0] NVIDIA  : Tesla K80 OpenCL 1.2 CUDA Device driver 367.48 FP64 Support(True)

Great job! I'm especially excited about 3 and 4-dimensional arrays.

However, this line from your first blog returns an error:

[7] pry(main)> c.array

NoMethodError: undefined method `array' for #<ArrayFire::Af_Array:0x005597c33f2540>

It's probably nothing serious, but I just wanted to let you know.

Best,

Alexej

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Prasun Anand]

Hi Alex,

Thanks for trying ArrayFire :) .

I have changed the method name #array to #elements (alias #to_cpu) becasue
#array seemed to be confusing. I will update my blog.

By the end of this week I will be creating an arrayfire-rb-benchmark-suite where
anybody can benchmark arrayfire functionalities on his device.

Regards,
Prasun

>> email to sciruby-dev...@googlegroups.com.

[Prasun Anand]

Hi,

I have created two blogs summarising my work in the last two weeks.

ArrayFire-rb is almost ready for 2d matrices, Hence, I can soon start 

working on NMatrix interfacing :) . There needs some work on managing multiple 

GPU cards but NMatrix can be interfaced to ArrayFire. I have posted a blog about the Algorithm class. 

Link: http://www.prasunanand.com/arrayfire/2017/07/22/gsoc17-arrayfire-ruby-bindings-part-3-minitest-algorithm.html

Last week I had been busy hacking on D bindings for Ruby headers.  The work on Calling D from Ruby is still a work in progress. 

Link :  http://www.prasunanand.com/gpu-computing/2017/07/25/gsoc17-calling-d-from-ruby-for-gpu-computing.html

I think SciRuby will find it interesting. I request your feedback.

Regards,

Prasun

[Prasun Anand]

Hi,

I have got BLAS, LAPACK, vector algorithms, statistics methods implemented for arrayfire.

I am working on implementing multiple datatypes and handling exceptions currently.

Next I will work on interfacing ArrayFire to NMatrix.

Regards,

Prasun

[Prasun Anand]

Hi,

You will be delighted to know that ArrayFire-rb can also run on CPU now as I have got the

unified backend(CPU + CUDA + OpenCL) successfully running. A user can also change the

backend on the fly.

I have added OpenCL and CUDA specific APIs but some of the bindings cannot be implemented.

These bindings help in plugging custom OpenCL and CUDA code. Unlike Python, which has

PyOpenCL and PyCUDA , Ruby has no such alternatives. So there is no use implementing those

bindings.

I am currently working on reinterpret_cast to support all the data-types

https://github.com/prasunanand/arrayfire-rb.

Regards,

Prasun

[Prasun Anand]

Hi,

Sparse support has been added to arrayfire-rb.

I have created an arrayfire-rb-benchmark-suite.

https://github.com/prasunanand/arrayfire-rb-benchmark-suite

This repo will help in reproducing the benchmark plots that I will publish in the upcoming blogs.

I have started working on arrayfire-rb examples that will help Ruby community to

explore ArrayFire-rb.

https://github.com/prasunanand/arrayfire-rb/tree/master/examples

Regards,

Prasun

[Prasun Anand]

Hi,

I have published a blog post about the BLAS and LAPACK routines in ArrayFire.

http://www.prasunanand.com/arrayfire/2017/08/16/gsoc17-arrayfire-ruby-bindings-part-4-blas-lapack.html

The benchmark results are nice. 

Please review :) .

Regards,

Prasun

[Prasun Anand]

Hi,

I have published a new blog regarding using Random generators and Statistics methods.

http://www.prasunanand.com/arrayfire/2017/08/17/gsoc17-arrayfire-ruby-bindings-part-4-statistics-and-random-engine.html

Another blog about using multiple backend is coming up :)

Regards,

Prasun

[Prasun Anand]

I have opened up a PR in arrayfire-rb rep.

https://github.com/arrayfire/arrayfire-rb/pull/3

Please review my PR. I am also working on refactoring my code and removing depreceated Ruby -C  APIS

Regards,

Prasun

[Prasun Anand]

Hi,

I have successfully interfaced nmatrix and arrayfire.

$ rake pry

pry -r './lib/arrayfire.rb'

[1] pry(main)> require 'nmatrix'

=> true

[2] pry(main)> a = ArrayFire::Af_Array.new 2, [4,4], [1, 2, 2, 0,  -2, 2 , 1, 3, 1, 4, 3 , 1, 0, -3, 2, 9]

No Name Array

[4 4 1 1]

    1.0000    -2.0000     1.0000     0.0000 

    2.0000     2.0000     4.0000    -3.0000 

    2.0000     1.0000     3.0000     2.0000 

    0.0000     3.0000     1.0000     9.0000 

=> #<ArrayFire::Af_Array:0x000000019bd6f0>

[3] pry(main)> b = a.to_nmatrix

=> 

[

  [ 1.0,  2.0, 2.0, 0.0]   [-2.0,  2.0, 1.0, 3.0]   [ 1.0,  4.0, 3.0, 1.0]   [ 0.0, -3.0, 2.0, 9.0] ]

[4] pry(main)> c = b.to_af_array

=> #<ArrayFire::Af_Array:0x00000001a4d340>

[5] pry(main)> c.elements

=> [1.0, 2.0, 2.0, 0.0, -2.0, 2.0, 1.0, 3.0, 1.0, 4.0, 3.0, 1.0, 0.0, -3.0, 2.0, 9.0]

[6] pry(main)> ArrayFire::Util.print_array(c)

No Name Array

[4 4 1 1]

    1.0000    -2.0000     1.0000     0.0000 

    2.0000     2.0000     4.0000    -3.0000 

    2.0000     1.0000     3.0000     2.0000 

    0.0000     3.0000     1.0000     9.0000 

=> true

[7] pry(main)> ArrayFire::Util.print_array(c)

No Name Array

[4 4 1 1]

    1.0000    -2.0000     1.0000     0.0000 

    2.0000     2.0000     4.0000    -3.0000 

    2.0000     1.0000     3.0000     2.0000 

    0.0000     3.0000     1.0000     9.0000 

=> true

[8] pry(main)> c = b.to_af_array

=> #<ArrayFire::Af_Array:0x00000001436088>

[10] pry(main)> exit

Now, I will write blogs and examples from http://arrayfire.org/docs/examples.htm.

I will also work on refactoring the code and proper exception handling.

Regards,

Prasun

[Pjotr Prins]

Can you give an example of a largish multiplication and timings?

On Mon, Aug 21, 2017 at 07:00:21AM -0700, Prasun Anand wrote:
> Hi,
> I have successfully interfaced nmatrix and arrayfire.
> $ rake pry
> pry -r './lib/arrayfire.rb'
> [1] pry(main)> require 'nmatrix'
> => true

> [2] pry(main)> a = ArrayFire::Af_Array.new 2, [4,4], [1, 2, 2, 0, Ā -2,

> 2 , 1, 3, 1, 4, 3 , 1, 0, -3, 2, 9]
> No Name Array
> [4 4 1 1]

> Ā Ā 1.0000 Ā Ā -2.0000 Ā Ā 1.0000 Ā Ā 0.0000Ā
> Ā Ā 2.0000 Ā Ā 2.0000 Ā Ā 4.0000 Ā Ā -3.0000Ā
> Ā Ā 2.0000 Ā Ā 1.0000 Ā Ā 3.0000 Ā Ā 2.0000Ā
> Ā Ā 0.0000 Ā Ā 3.0000 Ā Ā 1.0000 Ā Ā 9.0000Ā

> => #<ArrayFire::Af_Array:0x000000019bd6f0>
> [3] pry(main)> b = a.to_nmatrix

> =>Ā
> [
> Ā [ 1.0, Ā 2.0, 2.0, 0.0] Ā [-2.0, Ā 2.0, 1.0, 3.0] Ā [ 1.0, Ā 4.0,
> 3.0, 1.0] Ā [ 0.0, -3.0, 2.0, 9.0] ]

> [4] pry(main)> c = b.to_af_array
> => #<ArrayFire::Af_Array:0x00000001a4d340>
> [5] pry(main)> c.elements
> => [1.0, 2.0, 2.0, 0.0, -2.0, 2.0, 1.0, 3.0, 1.0, 4.0, 3.0, 1.0, 0.0,
> -3.0, 2.0, 9.0]
> [6] pry(main)> ArrayFire::Util.print_array(c)
> No Name Array
> [4 4 1 1]

> Ā Ā 1.0000 Ā Ā -2.0000 Ā Ā 1.0000 Ā Ā 0.0000Ā
> Ā Ā 2.0000 Ā Ā 2.0000 Ā Ā 4.0000 Ā Ā -3.0000Ā
> Ā Ā 2.0000 Ā Ā 1.0000 Ā Ā 3.0000 Ā Ā 2.0000Ā
> Ā Ā 0.0000 Ā Ā 3.0000 Ā Ā 1.0000 Ā Ā 9.0000Ā

> => true
> [7] pry(main)> ArrayFire::Util.print_array(c)
> No Name Array
> [4 4 1 1]

> Ā Ā 1.0000 Ā Ā -2.0000 Ā Ā 1.0000 Ā Ā 0.0000Ā
> Ā Ā 2.0000 Ā Ā 2.0000 Ā Ā 4.0000 Ā Ā -3.0000Ā
> Ā Ā 2.0000 Ā Ā 1.0000 Ā Ā 3.0000 Ā Ā 2.0000Ā
> Ā Ā 0.0000 Ā Ā 3.0000 Ā Ā 1.0000 Ā Ā 9.0000Ā

> => true
> [8] pry(main)> c = b.to_af_array
> => #<ArrayFire::Af_Array:0x00000001436088>
> [10] pry(main)> exit
> Now, I will write blogs and examples from
> http://arrayfire.org/docs/examples.htm.
> I will also work on refactoring the code and proper exception handling.
> Regards,
> Prasun

--

[Prasun Anand]

I have added the plot here.

http://www.prasunanand.com/arrayfire/2017/08/16/gsoc17-arrayfire-ruby-bindings-part-4-blas-lapack.html

Regards,
Prasun

[Prasun Anand]

Hi,

I got selected to give a talk on "GPU Computing with Ruby" at Ruby World Conference 2017 :).

Thank you all.

Regards,

Prasun

[Gaurav Tamba]

Great going Prasun!

I've been reviewing your code and blog posts, and am trying to see if I can quickly put together an interesting example using array-fire that highlights its capabilities (as a learning experience for me as well), will let you know how that goes!

Best,
Gaurav

[Prasun Anand]

Hi,

I have published a blog post regarding using the Device and Util class of ArrayFire-rb.

http://www.prasunanand.com/arrayfire/2017/08/22/gsoc17-arrayfire-ruby-bindings-part-6-device.html

Thank you @Gaurav for reviewing. Please open issues as you find them.

Regards,

Prasun

[Prasun Anand]

Hi,

I have published another blog post

http://www.prasunanand.com/arrayfire/2017/08/24/gsoc17-arrayfire-ruby-bindings-part-7-backend-cuda-and-opencl.html

It explains using CPU, OpenCL or CUDA for computing with ArrayFire.

The next blog is about ArrayFire-NMatrix interface and benchmarks.

Regards,

Prasun

[Alexej Gossmann]

Hi Prasun,

Thanks for the updates!

I'll try to find more time to play with the ArrayFire gem this week.

Do you think I could recreate this logistic regression example http://arrayfire.org/docs/machine_learning_2logistic_regression_8cpp-example.htm using only Ruby?

Best,

Alexej

[Prasun Anand]

Hi Alexei,

Yes, you can do that :).

Regards,

Prasun

--
You received this message because you are subscribed to the Google Groups "SciRuby Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sciruby-dev+unsubscribe@googlegroups.com.

[Prasun Anand]

Hi,

I have filled in the final evaluations.

The report can be found here:

http://www.prasunanand.com/arrayfire/cuda/opencl/ruby/2017/08/28/gsoc17-project-report.html

Another blog post about NMatrix-ArrayFire-rb is missing some benchmark figures. I am working on them 

now. I already published it uncomplete because I didn't want to miss the evaluation.

http://www.prasunanand.com/arrayfire/2017/08/24/gsoc17-arrayfire-ruby-bindings-part-8-nmatrix-interface.html

Regards,

Prasun

[Prasun Anand]

Hi,

I have completed the blog post about ArrayFire-NMatrix interface. 

http://www.prasunanand.com/arrayfire/2017/08/24/gsoc17-arrayfire-ruby-bindings-part-8-nmatrix-interface.html

The blog explains how adding small chunk of code can easily speedup the code and how NMatrix

and ArrayFire can easily be plugged together.

Performance figure added :) .

Regards,

Prasun

[Prasun Anand]

Hi,

The PR to arrayfire/arrayfire-rb has been merged.

I am working on finishing up my blog for SciRuby.

I have been invited as speaker to Ruby Conf Indonesia to talk on "High Performance GPU computing with Ruby" :) .

Regards,

Prasun