calculations with arrays

[Tim Meehan]

Is there a way to use Racket vectors similarly to how one might use NumPy?

(Or are vectors even the right thing to use?)

[Veit Heller]

Hey,

I’d say that depends on what you’re trying to do, but you might want to take a look at the math[1] library, specifically math/array[2] and math/matrix. Hope that helps!

If you need more specific help, it might be good to speak a little bit about your use case.

Cheers,
V

[1] https://docs.racket-lang.org/math/index.html
[2] https://docs.racket-lang.org/math/array.html
[3] https://docs.racket-lang.org/math/matrices.html

[Jens Axel Søgaard]

Den ons. 18. nov. 2020 kl. 14.15 skrev Tim Meehan <btme...@gmail.com>:

Is there a way to use Racket vectors similarly to how one might use NumPy?

(Or are vectors even the right thing to use?)

If you are after numerical calculations for mxn matrices, then look at `flomat`
which uses the same underlying C/Fortran library that NumPy uses.

https://docs.racket-lang.org/manual-flomat/index.html

If you need rrays with more dimensions than 2 use `math/array`.

If you need simple computation of 1-dimensional data, you can use
standard Racket vectors (or flomats or arrays).

/Jens Axel

[Tim Meehan]

Hello and thanks for the reply. I found your math/matrix library earlier this morning. That does look like it does most of what I need.

I’m looking to convert some Python scripts that load in data, process it with NumPy and plot (SVG, PDF, etc) with Matplotlib. I wanted to do it in a natural-looking Racket-y way.

Some of the datasets are large enough that speed might be a concern. Does that mean typed Racket? I don’t mind learning, but I’m dragging along some folks that are newer to Racket than I am.

Thanks again 

On Nov 18, 2020, at 08:23, Jens Axel Søgaard <jens...@soegaard.net> wrote:



[Dominik Pantůček]

Hello,

>
> If you are after numerical calculations for mxn matrices, then look at
> `flomat`
> which uses the same underlying C/Fortran library that NumPy uses.
>
> https://docs.racket-lang.org/manual-flomat/index.html
>
> If you need rrays with more dimensions than 2 use `math/array`.
>
> If you need simple computation of 1-dimensional data, you can use
> standard Racket vectors (or flomats or arrays).

I didn't know about `flomat'. Thanks for (yet another) good hint.

However, for my project I needed really fast matrix multiplication (and
other basic linear algebra functions). It turned out that most of the
available options are sub-optimal at best. To my surprise `flomat` is
one of those.

Mostly I am concerned with 3D and 4D matrices and vectors.

During the past few months I hacked together a fairly optimized
module[1] for performing these operations that defines algebras for
given dimension during macro expansion stage and all the procedures are
constructed in a way that helps Racket (mainly CS) perform all
operations unboxed.

In the repository, there is a benchmark script, which yields the
following (self-explanatory) results:

==== Welcome to Racket v7.9.0.5 [cs].

# 1000000 iterations

## flalgebra

mat3x3*mat3x3! cpu time: 13 real time: 13 gc time: 0
mat3x3*mat3x3 cpu time: 23 real time: 23 gc time: 4

## math/matrix

matrix*matrix cpu time: 55892 real time: 55911 gc time: 463

## math/typed/matrix

matrix*matrix cpu time: 6861 real time: 6862 gc time: 1045

## flomat

matrix*matrix! cpu time: 887 real time: 887 gc time: 4
matrix*matrix cpu time: 1825 real time: 1825 gc time: 7

==== Welcome to Racket v7.9 [bc].

# 1000000 iterations

## flalgebra

mat3x3*mat3x3! cpu time: 145 real time: 145 gc time: 7
mat3x3*mat3x3 cpu time: 163 real time: 163 gc time: 2

## math/matrix

matrix*matrix cpu time: 53817 real time: 53788 gc time: 733

## math/typed/matrix

matrix*matrix cpu time: 3852 real time: 3851 gc time: 730

## flomat

matrix*matrix! cpu time: 745 real time: 745 gc time: 1
matrix*matrix cpu time: 1621 real time: 1620 gc time: 1


What puzzles me the most: when I read `flomat' documentation, I thought
it must beat my implementation by far margin - it's a native code for
really basic number crunching. When using the `times!' variant of matrix
multiplication, I would expect it to outperform anything implemented in
in pure Racket.

Is there something I miss or is it really the case, that carefully
crafted Racket implementation is the fastest option right now?

I actually wanted to write a similar email some time ago - in the spring
- about the typed math/matrix variant. But I was almost certain I am
doing something wrong and should figure it out. Now I am more and more
convinced that the performance of all number crunching libraries - as
they can be used in Racket - might be the real issue here.


Cheers,
Dominik

[1] https://gitlab.com/racketeer/flalgebra

[Jens Axel Søgaard]

Den tir. 24. nov. 2020 kl. 15.12 skrev Dominik Pantůček <dominik....@trustica.cz>:

Hello,

> If you are after numerical calculations for mxn matrices, then look at
> `flomat`
> which uses the same underlying C/Fortran library that NumPy uses.
>
> https://docs.racket-lang.org/manual-flomat/index.html
>
> If you need rrays with more dimensions than 2 use `math/array`.
>
> If you need simple computation of 1-dimensional data, you can use
> standard Racket vectors (or flomats or arrays).

I didn't know about `flomat'. Thanks for (yet another) good hint.

However, for my project I needed really fast matrix multiplication (and
other basic linear algebra functions). It turned out that most of the
available options are sub-optimal at best. To my surprise `flomat` is
one of those.

Mostly I am concerned with 3D and 4D matrices and vectors.

During the past few months I hacked together a fairly optimized
module[1] for performing these operations that defines algebras for
given dimension during macro expansion stage and all the procedures are
constructed in a way that helps Racket (mainly CS) perform all
operations unboxed.

In the repository, there is a benchmark script, which yields the
following (self-explanatory) results:

==== Welcome to Racket v7.9.0.5 [cs].

... 

What puzzles me the most: when I read `flomat' documentation, I thought
it must beat my implementation by far margin - it's a native code for
really basic number crunching. When using the `times!' variant of matrix
multiplication, I would expect it to outperform anything implemented in
in pure Racket.

Is there something I miss or is it really the case, that carefully
crafted Racket implementation is the fastest option right now?

I think the main reason that flomat matrices are slower than your flalgebra ones
is the cost of the FFI call. The cost of calling a C foreign function doesn't
depend on the size of the matrix though - so for large matrices the cost
is negligible. However for small matrices, it is faster to stay in the Racket world.

I believe game libraries in C also implement their own 3x3 matrices rather 
than using BLAS/LAPACK. BLAS can handle very large matrices and 
makes an effort to give good results even for ill-conditioned matrices.
 

I actually wanted to write a similar email some time ago - in the spring
- about the typed math/matrix variant. But I was almost certain I am
doing something wrong and should figure it out. Now I am more and more
convinced that the performance of all number crunching libraries - as
they can be used in Racket - might be the real issue here.

I have somewhat regretted that the matrices in math/matrix used math/array
arrays to represent matrices directly. A traditional approach would have been 
better. However math/matrix matrices are more general - they also work
when you need matrices over exact integers or over rationals.

In order from slowest to fastest (when used for floats over 3x3 matrices):

   math/matrix   (most general, very slow)
   flomat            (only floats, arbitrary size)

   flagebra         (floats, square only?)

To satisfy my curiosity, of the performance of flomat:

One way to get a little extra performance out of  flomat! is to use times!
instead of times. This avoids the repeated allocation. The code of times! is:

For a more direct call to the C multiplication routine:
  

So `(flomat! A B C)` will compute A*B and store it in C.

Maybe there is a specialized A*B only routine in BLAS that 
can be used instead?

The benchmarks suggest that the  `times` in flomat can be improved for
smaller matrices by calling specialized routines written in Racket.

If you (or anyone else) are interested in improving `flomat` for multiplying
smaller matrices, I would be glad to receive patches.

Btw - you have some fancy tricks in `flalgebra.rkt` !

/Jens Axel

[Bertrand Augereau]

I believe game libraries in C also implement their own 3x3 matrices rather 
than using BLAS/LAPACK. BLAS can handle very large matrices and 
makes an effort to give good results even for ill-conditioned matrices.

They sure do. Generally as 4x4 or implicit homogeneous 4x3 matrices to help leverage a little SIMD.