MAGMA 1.0.0 zlatrd needs zdotc

[fletchjp]

I am installing MAGMA 1.0.0 having previously used RC5. I have copied my make.inc for gotoblas and lapack.

Code: Select all

# using GotoBLAS
LIB       = -lgoto2  -lpthread -lcublas -lcudart -llapack -lm

When I make the main installation it fails with this:

Code: Select all

gcc -O3 -DADD_ -DGPUSHMEM=200 -fPIC -Xlinker -zmuldefs -DGPUSHMEM=200  testing_zhetrd.o  -o testing_zhetrd lin/liblapacktest.a -L../lib \
          -lcuda -lmagma -lmagmablas -lmagma -L/home/fletcher/GotoBLAS2 -L/usr/lib64 -L/usr/local/cuda/lib64  -lgoto2  -lpthread -lcublas -lcudart -llapack -lm 
../lib/libmagma.a(zlatrd.o): In function `magma_zlatrd':
zlatrd.cpp:(.text+0x3ce): undefined reference to `zdotc'
zlatrd.cpp:(.text+0xd16): undefined reference to `zdotc'

From the Changelog this is one of the new testing routines and clearly it needs something not supplied before.

What do I need to do, please?

John

[Stan Tomov]

Hi John,

This problem comes from zdot not having the same interface in the different BLAS implementations. We didn't realize there would be a problem for GotoBLAS with this change. The way it is now will work for MKL. If you open file zlatrd.cpp, before calling blasf77_zdotc, there is a call to cblas_zdotc_sub that is commented. This is an alternative to calling the blasf77_zdotc but you would have to add linking to cblas (if it is not part of the GOTO BLAS). The other way is to see what is the ZDOT interface in GotoBLAS and call it the correct way. Meanwhile probably for the next release we will make all BLAS functions to use CBLAS and require linking to CBLAS to avoid problems like this.

Best regards,
Stan

[fletchjp]

Stan

Thank you for the prompt reply. I could not figure out what was happening, as zdotc was (I thought) part of BLAS. I thought the whole point of BLAS was that it had a standard interface. I have learned something today!

Cheers

John

[Stan Tomov]

John,
Actually, there is no problem with the interface, which is in FORTRAN. There is no problem with the optimized FORTRAN implementations either. The problem is when we call FORTRAN functions from C as it is not clear how to return the values. Some C implementations (e.g., wrappers to FORTRAN) will make it subroutine where the result will be returned in an argument (in the case of CBLAS).
Stan

[fletchjp]

Stan

In the case of GotoBLAS2 the easy thing to do is to switch to using the cblas interface, which GotoBLAS supports as default, so I do not need to recompile it. There is a second call to zdotc in the code of zlatrd.cpp and that needs to be replaced as well which I have done like this:

Code: Select all

          cblas_zdotc_sub(i, W(0, iw), ione,
                        A(0, i), ione, &value);
          /* blasf77_zdotc(&value, &i, W(0, iw), &ione, A(0, i), &ione); */


The same two changes have to be done in clatrd.cpp as well, with cblas_cdotc_sub.

With that it all builds again.

Thanks

John

[fletchjp]

I am trying out the resulting code. The GPU results are slower than the CPU ones with a four threaded GotoBLAS2.

Code: Select all

fletcher@fletcher-desktop:~/magma_1.0.0/testing$ ./testing_zhetrd
device 0: GeForce GTX 460, 1400.0 MHz clock, 2047.2 MB memory

Usage:
  testing_zhetrd -L|U -N 1024



  N    CPU GFlop/s    GPU GFlop/s   |A-QHQ'|/N|A|  |I-QQ'|/N
=============================================================
 1024    21.12         17.74
 2048    19.48         13.18
 3072    19.49         13.65
 4032    18.91         13.14
 5184    19.20         13.27
 6016    18.68         13.39
 7040    18.21         13.27
 8064    17.07         13.27
^C
fletcher@fletcher-desktop:~/magma_1.0.0/testing$ ./testing_zhetrd_gpu
device 0: GeForce GTX 460, 1400.0 MHz clock, 2047.2 MB memory

Usage:
  testing_zhetrd_gpu -L|U -N 1024



  N    CPU GFlop/s    GPU GFlop/s   |A-QHQ'|/N|A|  |I-QQ'|/N
=============================================================
 1024    21.50         20.79
 2048    19.48         13.18
 3072    19.27         13.65
 4032    18.35         13.14
^C

[fletchjp]

The single precision results show the same pattern.

Code: Select all

fletcher@fletcher-desktop:~/magma_1.0.0/testing$ ./testing_chetrd
device 0: GeForce GTX 460, 1400.0 MHz clock, 2047.2 MB memory

Usage:
  testing_chetrd -L|U -N 1024



  N    CPU GFlop/s    GPU GFlop/s   |A-QHQ'|/N|A|  |I-QQ'|/N
=============================================================
 1024    38.89         32.69
 2048    35.74         28.92
 3072    36.26         28.64
 4032    35.61         25.83
 5184    36.15         28.02
 6016    36.66         28.04
 7040    36.47         27.68
^C
fletcher@fletcher-desktop:~/magma_1.0.0/testing$ ./testing_chetrd_gpu
device 0: GeForce GTX 460, 1400.0 MHz clock, 2047.2 MB memory

Usage:
  testing_chetrd_gpu -L|U -N 1024



  N    CPU GFlop/s    GPU GFlop/s   |A-QHQ'|/N|A|  |I-QQ'|/N
=============================================================
 1024    38.92         38.93
 2048    35.89         28.94
 3072    35.96         28.74
 4032    35.73         25.88
^C

John

[fletchjp]

Postscript on GotoBLAS2. The source code for the zdotc interface inside GotoBLAS2 has some defines to choose at compile time between different interfaces. I have not found anything in the notes to explain how to choose between them.

My understanding is that GotoBLAS is not being further developed.

John

[jgpallero]

Hello,

Postscript on GotoBLAS2. The source code for the zdotc interface inside GotoBLAS2 has some defines to choose at compile time between different interfaces. I have not found anything in the notes to explain how to choose between them.

My understanding is that GotoBLAS is not being further developed.

John

GotoBLAS2 is continued as a new project called OpenBLAS: https://github.com/xianyi/OpenBLAS

[fletchjp]

This problem is still present in MAGMA 1.1.0 and the same fixes to zlatrd.cpp and clatrd.cpp sort it out.

I don't know why they are not needed in zlatrd2.cpp and clatrd2.cpp as well.

John