Arithmetic operations with complex matrices

[Shankar (krishnan2098)]

Gonum doesn't provide support for complex matrix arithmetic operations (such as addition, subtraction, division, multiplication, transpose etc). The CDense type for matrix creation is barebones. An interesting use case arrived when we were simulating a Telecommunication system where support for complex matrix addition, subtraction, etc. were required. 

In the discussion in reddit, Vladimir Ch. suggests that CDense matrix lacks matrix operations. They can be implemented using BLAS Level 1,2,3 implementation for complex64/128 in pure Go. I would love to contribute towards implementing these matrix operations. But where do I start?

[Dan Kortschak]

Hi Shankar,

The plan that I had was to refactor the files in mat so that we could
autogenerate the complex128 cases from the float64 case. This is the
first part. The second (and probably more important part) is making
sure that there is test coverage.

Dan

On Mon, 2019-07-22 at 05:55 -0700, Shankar (krishnan2098) wrote:
> Gonum doesn't provide support for complex matrix arithmetic
> operations
> (such as addition, subtraction, division, multiplication, transpose
> etc). The
> CDense type for matrix creation is barebones. An interesting use
> case
> arrived when we were simulating a Telecommunication system where
> support
> for complex matrix addition, subtraction, etc. were required.
> In the discussion in reddit
> <

> https://www.reddit.com/r/golang/comments/cf6giw/arithmetic_operations_with_complex_matrices/?utm_source=share&utm_medium=web2x>,

[Shankar (krishnan2098)]

Hi Dan,

I would love to be of help. How would you refactor it? Or shall I work on it?

[Dan Kortschak]

The plan was to pull the simpler things that are easily generalisable
into separate files that you can apply scripts similar to what we have
in the blas/gonum package.

I will leave this to you, but I'm happy to advise.

Dan

[Shankar (krishnan2098)]

I'll go through the blas/gonum package and try to figure out how the scripts for 

generalising works. If i don't get it I'll ask you for further mentoring, if that's alright with you?

Shankar

[Dan Kortschak]

That sounds good to me.

Dan

[Brendan Tracey]

I think the easiest thing to start with would be to try and auto-generate CDense.Add(a, b CMatrix). The basic function is trivial, it doesn't have very many special cases (yet), but would give some exposure to some of the implementation details (untranspose, reuseAs). I would suggest a rough implementation plan would be to move the Dense.Add method to its own file, try and auto-generate the complex version, adding as many shims as you need. For instance, you could do a partial implementation where the equivalent of checkOverlap just returns true,  to give a basic working version where we could iterate on implementation details before you get too far.

[Shankar (krishnan2098)]

Hi Dan,

I tried going through the files but I wasn't able to understand it. Could you help me with it by sharing some resources or by guiding me. Also, is there any other method to reach you rather than this group?

~ Shankar

[Dan Kortschak]

Hi Shankar,

Which files?

Looking at Add, there's dependency on untransposeExtract,
Dense.checkOverlapMatrix/Dense.checkOverlap and
Dense.isolatedWorkspace. These need to handled first, and even just
untransposeExtract is non-trivial (done by hand in first instance since
otherwise you need to implement all the complex types first).

We have go more complex than I remember.

Dan

[Shankar (krishnan2098)]

Hi Dan,

I am unable to understand you. Can we have a walkthrough, if possible? Moreover, I had gone through this repository.

[Dan Kortschak]

What I was saying is that it's more complex than I remember.

I have sent a PR to add shadow detection[1] to CDense. I will (or you
can) also need to send PRs add the other functions and methods that I
mentioned below to the CDense type (essentially copy/paste from the
Dense equivalents). I'm afraid I don't have more time at the moment to
really go through this in detail.

Dan

[1]https://github.com/gonum/gonum/pull/1047

On Mon, 2019-08-05 at 22:09 -0700, Shankar (krishnan2098) wrote:
> Hi Dan,

> I am unable to understand you. Can we have a walkthrough, if
> possible?
> Moreover, I had gone through this repository

> <https://github.com/gonum/gonum/tree/master/blas>.

[Brendan Tracey]

Could you be more specific in what you don't understand? That way we can be more targeted here and/or in code documentation on how it can be more clear. Similarly, if you see things that could be more clear that you do understand, please submit a PR. 

--
You received this message because you are subscribed to the Google Groups "gonum-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gonum-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gonum-dev/f575760f-d810-4c9b-962d-3de0d9bf51be%40googlegroups.com.

[Matt Braunstein]

Has there been any progress on this? If not, I would like to get the ball rolling on this. I'm trying to convert a program that I've written in Python. It requires the ability to handle complex number linear algebra.

My plan was to reference and convert the routines here. This seems to be the most accessible form of the routines that I've found, although they are all in Fortran. I don't think just refactoring the float routines will work. It will for some, but not all.

Based on the plan to actually port existing Fortran routines, I'm looking for some more information on the structure/architecture of the gonum library. I want to make sure to stick with this as closely as possible. Based on my initial perusing, it looks like the combination of BLAS and LAPACK routines are being implemented a little differently. Maybe there's a better resource for the routines that were used for the initial port?

Keep in mind, I'm just learning Go. So I'm still wrapping my head around how Go implements its quasi-OOP setup. But, I know I can figure it out as I go. The best way to do that is to just get my hands dirty.

[Dan Kortschak]

Hi Matt,

No nothing significant has happened here beyond the complex BLAS
implementations.

We would certainly be interested in contributions of complex LAPACK
routines. The netlib site is the primary source of information for our
implementations, with the code in our implementations coming originally
from the Fortran. Note though that we are row major (for the most part
- there are exceptions where it doesn't make sense to deviate) and zero
indexed, so the translations can get subtle.

The approximate process is to write a test for the routine and add it
to lapack/testlapack and add a test and wrapper for the Cgo
implementation in gonum/netlib/lapack/netlib, and then with that
passing (not necessarily merged into the repo), port the pure-Go
implementation. It works best to look at the call graph and work
backwards.

I'm not sure what you mean by the BLAS and LAPACK routines being
implemented differently; they are intended to be architecturally
identical. Can you explain what you mean?

Dan

> --
> You received this message because you are subscribed to the Google
> Groups "gonum-dev" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to gonum-dev+...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/gonum-dev/7154d1bb-7f6b-4228-90d8-c16f1144e04c%40googlegroups.com
> .

[Matt Braunstein]

Thanks for the pointers on porting the library. Maybe I wasn't looking at the library properly. I'll figure it out as I get into this process.

[Matt Braunstein]

OK, so I've been looking through this over the last couple of days. Unfortunately, I haven't had a significant amount of time to spend on it, since it is basically any free time I have. But, I'm hitting a bit of a roadblock. It may just be a misunderstanding on my part. Here's where I think I should be starting based on your recommendations.

I've installed the gonum LAPACK package. In the "gonum.org/v1/netlib/lapack/lapacke/lapacke.go" file, I've added a wrapper function for the proposed new function. Then in the "gonum.org/v1/gonum/lapack/testlapack/" directory, I've added a new go file for testing. In this file, I've added a test function to call the previous wrapper function. But when I try to run the test, I'm getting all kinds of compiler linking errors regarding LAPACK.

I took your advice and chose a function at the very end of the caller graph. This one happened to be DLARUV. Here's my wrapper function.

func Dlaruv(iseed [4]int32, n int, x []float64) bool {
 var _iseed *int32
 if len(iseed) > 0 {
 _iseed = &iseed[0]
 }
 var _x *float64
 if len(x) > 0 {
 _x = &x[0]
 }
 return isZero(C.LAPACKE_dlaruv_work((C.int)(rowMajor), (*C.lapack_int)(_iseed), (C.lapack_int)(n), (*C.float)(_x)))
}

Here's the simple little test function to try and run the routine.

func TestDlaruv(t *testing.T) {
 var iseed [4]int
 n := 1
 x := make([]float64, n)

 iseed[0] = 5
 iseed[1] = 3
 iseed[2] = 55
 iseed[3] = 67

 lapacke.Dlaruv(iseed, n, x)
}

Am I going about this correctly? I noticed the lapacke.h file does not contain a definition for LAPACKE_dlaruv_work. Does this mean that I need to go further and incorporate this? If so, are there any additional steps I need to do in order to provide access to the underlying function?

[Vladimír Chalupecký]

Hi Matt,

Dlaruv is probably not the best start, it's too Fortran specific, not tremendously useful and also not provided by LAPACKE (that's why you cannot find it in lapacke.h).

If you want to work on complex Lapack, then for example Ztrtrs would be a good start. The function itself is trivial, therefore easy to understand and test, is exported by LAPACKE and is in the call graph of Zgels which you mentioned before as useful for you.

Alternatively, you can work in the mat package on adding basic arithmetic operations to the CDense type following what exists for the Dense type.

[Matt Braunstein]

Thanks for the pointer. I started looking into this and quickly realized that several utility functions from the floats package are used. I decided to start with generating a cfloats equivalent. I completed that over the weekend. I need to get that merged into the gonum package.

I've just submitted a pull request to add myself to the contributors list. Once that's approved, I can add a pull request to add the cfloats package. Then this can be utilized for access in the complex number functions from LAPACK.

[Jakub Korsak]

Hello all,

It is my first post here, and I'd like to ask whether there has been progress since 1537? 

I'm interested in contributing to the implementation of CDense, in particular things like Add, Mul, MulVec, Scale and such.

What should my first steps be?

Thanks,

JK

[Dan Kortschak]

On Mon, 2023-01-09 at 11:31 -0800, Jakub Korsak wrote:
> Hello all,
> It is my first post here, and I'd like to ask whether there has been
> progress since 1537? 
> I'm interested in contributing to the implementation of CDense, in
> particular things like Add, Mul, MulVec, Scale and such.
> What should my first steps be?
> Thanks,
> JK

There has been no progress on this.

If you are interested in implementing the basic CDense arithmetic
operations, I would suggest you make copies of the Dense operations and
pool code as was done in the linked PR; working from the original code
in the gonum/gonum repo's mat package in dense_arithmetic.go should be
fine (copy to cdense_arithmetic.go would be the most sensible).

You will also need to add tests. This can, in the first instance, be
derived from the relevant tests in dense_test.go for the methods being
added. These tests make use of the list test infrastructure which is
likely too large to bring in for CDense to start with (this can be done
in a later PR - this should be largely mechanical).

Dan