ilp64 build -> odd ipiv vector

6 views
Skip to first unread message

Paul Karlshöfer

unread,
Apr 27, 2021, 11:12:37 AMApr 27
to MAGMA User
Dear developers,

I am using Magma to compute the LU decomposition of a large matrix with the function 
magmaf_dgetrf_m(...).

The matrices are larger than the combined GPU memory - so to deal with a seg-fault once we exceed the memory size, we built Magma with ipl64. This works great.

However, we encounter a problem with the ipiv vector (permutation matrix) which is a integer(8), allocatable   :: ipiv(:)
In int8 representation, the return vector reads something like:
25769803781
[..]
I noticed that each value represents 2 int4s, as 8589934593 in hex is 0x200000001
so i tried to split each int8 into 2 int4s with some bitwise operations. However, there are also a lot of 0s or other random but small and reoccurring numbers in the ipiv vector which don't seem to represent anything meaningful in the ipiv vector.
Furthermore, I counted the numbers in the ipiv vector which exceed 0x100000000 and they precisely match the matrix dimension divided by 2.
So I conclude that all those numbers (split in 2 int4s) should somehow represent my ipiv vector but so far, I didn't manage to fully recover it.

The source of the issue seems to be, that Magma does not properly fill the ipiv vector for int8?!
Does anyone have an idea? I am grateful for all tips!


This is part of the code:
              integer, pointer        :: ipiv(:)
              integer(8), allocatable   :: ipiv_copy(:)
              call magmaf_dgetrf_m(1, d_dim, d_dim, asigma(id)%mat, d_dim, ipiv_copy, info_lapack )

              do i = 1, d_dim
if (ipiv_copy(i) .GE. 4294967296) then ! >0x100000000
  id3 = id3 + 1
  asigma(id)%ipiv(id3) = int(ISHFT(ISHFT(ipiv_copy(i),32), -32), kind=4)
  id3 = id3 + 1
  asigma(id)%ipiv(id3) = int(ISHFT(ipiv_copy(i),-32), kind=4)
end if
              end do
 
IPIV Vector
          int8                          int4 (least sig)   int4(most sig)
          8589934593           1           2
           17179869187           3           4
           25769803781           5           6
           34359738375           7           8
           42949672969           9          10
           51539607563          11          12
           60129542157          13          14
           68719476751          15          16
           77309411345          17          18
           85899345939          19          20
           94489280533          21          22
          103079215127          23          24
          111669149721          25          26
          120259084315          27          28
          128849018909          29          30
          352187318353          81          82
          360777252947          83          84
          369367187541          85          86
          377957122135          87          88
          386547056729          89          90
        28441273440733        6621        6622
        28449863375327        6623        6624
        28458453309921        6625        6626
        28467043244515        6627        6628
        28475633179109        6629        6630
          523986010233         121         122
          532575944827         123         124
[...]
                   256         ?                ?
                   256        ?                ?
                   256        ?                ?
                   256         ?                ?
                   256        ?                ?
                   256         ?                ?
                   256        ?                ?
                   256         ?                ?
                   256         ?                ?



Mark Gates

unread,
Apr 27, 2021, 3:12:08 PMApr 27
to Paul Karlshöfer, MAGMA User
What BLAS & LAPACK library are you linking with? What is your make.inc file, or the list of libraries using `ldd your_application`? For example, using MKL, it should have ilp64 in the name: like -lmkl_gf_ilp64.

With ilp64, the ipiv vector should have 64-bit integers in it. You shouldn't need to unpack them as 32-bit ints in any way.

Mark
Reply all
Reply to author
Forward
0 new messages