LAPACK freezing during the test

[anmunoz]

Hi All!

I was wondering if someone could help me, I'm trying to build LAPACK on an 64 bit Intel Xeon 3.2MHz with Linux SUSE 10.0 using the ATLAS BLAS libraries.

The problem is that when LAPACK is on this step:

SEP: Testing Symmetric Eigenvalue Problem routines
./xeigtsts < sep.in > ssep.out 2>&1

it stays there forever every time, although is using 100% of my processor. I already installed the patch mentioned on this forum in several topics. I'm also using gcc version 3.3.5

My make.inc file is:

####################################################################
# LAPACK make include file. #
# LAPACK, Version 3.0 #
# June 30, 1999 #
####################################################################
#
SHELL = /bin/sh
#
# The machine (platform) identifier to append to the library names
#
PLAT = _LINUX
#
# Modify the FORTRAN and OPTS definitions to refer to the
# compiler and desired compiler options for your machine. NOOPT
# refers to the compiler options desired when NO OPTIMIZATION is
# selected. Define LOADER and LOADOPTS to refer to the loader and
# desired load options for your machine.
#
FORTRAN = g77
OPTS = -funroll-all-loops -O3
DRVOPTS = $(OPTS)
NOOPT =
LOADER = g77
LOADOPTS =
#
# The archiver and the flag(s) to use when building archive (library)
# If you system has no ranlib, set RANLIB = echo.
#
ARCH = ar
ARCHFLAGS= cr
RANLIB = ranlib
#
# The location of the libraries to which you will link. (The
# machine-specific, optimized BLAS library should be used whenever
# possible.)
#
#BLASLIB = ../../blas$(PLAT).a
BLASLIB = -L/disk/data/munoz/ATLAS/lib/Lin_Fil/ -lf77blas -latlas
LAPACKLIB = lapack$(PLAT).a
TMGLIB = tmglib$(PLAT).a
EIGSRCLIB = eigsrc$(PLAT).a
LINSRCLIB = linsrc$(PLAT).a
#############################################



and my Makefile is:



############################################
# Top Level Makefile for LAPACK
# Version 3.0
# June 30, 1999
#

include make.inc

all: install lib testing blas_testing timing blas_timing

lib: lapacklib tmglib
#lib: blaslib lapacklib tmglib

clean: cleanlib cleantesting cleantiming

install:
( cd INSTALL; $(MAKE); ./testlsame; ./testslamch; \
./testdlamch; ./testsecond; ./testdsecnd; \
cp lsame.f ../BLAS/SRC/; cp lsame.f ../SRC; \
cp slamch.f ../SRC/; cp dlamch.f ../SRC/; \
cp second.f ../SRC/; cp dsecnd.f ../SRC/ )

#blaslib:
# ( cd BLAS/SRC; $(MAKE) )

lapacklib:
( cd SRC; $(MAKE) )

tmglib:
( cd TESTING/MATGEN; $(MAKE) )

testing:
( cd TESTING ; $(MAKE) )

blas_testing:
( cd BLAS/TESTING; $(MAKE) -f Makeblat1 )
( cd BLAS; ./xblat1s > sblat1.out; \
./xblat1d > dblat1.out; \
./xblat1c > cblat1.out; \
./xblat1z > zblat1.out )

( cd BLAS/TESTING; $(MAKE) -f Makeblat2 )
( cd BLAS; ./xblat2s < sblat2.in ; \
./xblat2d < dblat2.in ; \
./xblat2c < cblat2.in ; \
./xblat2z < zblat2.in )

( cd BLAS/TESTING; $(MAKE) -f Makeblat3 )
( cd BLAS; ./xblat3s < sblat3.in ; \
./xblat3d < dblat3.in ; \
./xblat3c < cblat3.in ; \
./xblat3z < zblat3.in )

timing:
( cd TIMING; $(MAKE) )

blas_timing:
( cd TIMING/LIN; $(MAKE) )
( cd TIMING; ./xlintims < sblasa.in > sblasa.out ; \
./xlintims < sblasb.in > sblasb.out ; \
./xlintims < sblasc.in > sblasc.out )
( cd TIMING; ./xlintimd < dblasa.in > dblasa.out ; \
./xlintimd < dblasb.in > dblasb.out ; \
./xlintimd < dblasc.in > dblasc.out )
( cd TIMING; ./xlintimc < cblasa.in > cblasa.out ; \
./xlintimc < cblasb.in > cblasb.out ; \
./xlintimc < cblasc.in > cblasc.out )
( cd TIMING; ./xlintimz < zblasa.in > zblasa.out ; \
./xlintimz < zblasb.in > zblasb.out ; \
./xlintimz < zblasc.in > zblasc.out )

cleanlib:
( cd INSTALL; $(MAKE) clean )
( cd BLAS/SRC; $(MAKE) clean )
( cd SRC; $(MAKE) clean )
( cd TESTING/MATGEN; $(MAKE) clean )

cleantesting:
( cd TESTING/LIN; $(MAKE) clean )
( cd TESTING/EIG; $(MAKE) clean )
( cd TESTING; rm xlin* xeig* )

cleantiming:
( cd TIMING/LIN; $(MAKE) clean )
( cd TIMING/LIN/LINSRC; $(MAKE) clean )
( cd TIMING/EIG; $(MAKE) clean )
( cd TIMING/EIG/EIGSRC; $(MAKE) clean )
( cd TIMING; rm xlin* xeig* )
##########################################

If someone has had a similar problem or know how to help me I would greatly appreciate it.

[Julien Langou]

hello

so I doubt it is because of the BLAS you are using, but just removing the doubt from ATLAS BLAS, can you try:

- change the make.inc so that: BLASLIB = ../../blas$(PLAT).a
- make blaslib
- cd TESTING
- rm -f xeigtsts
- cd EIG
- make single
- cd ..
- ./xeigtsts < sep.in

and then ... maybe it does not hang ....
but I doubt.

then can you try in LAPACK/SRC to replace the file slaed6.f with the following:
http://www.cs.utk.edu/~langou/slaed6.f

(
1- This file is from Ren-Cang Li from Uiversity of Kentucky, thanks!
2- it's going to be released soon, and should help in your case
)

then:
- cd SRC
- make single
- cd ../TESTING
- rm -f xeigtsts
- cd EIG
- make single
- cd ..
- ./xeigtsts < sep.in

and then jsut reply that's it does not hang .... please ....
otherwise we got a problem.

you can also get dlaed6.f at:
http://www.cs.utk.edu/~langou/dlaed6.f


Julien

[anmunoz]

Hi Julien:

Thank you very much for your prompt answer, unfortunately we have a problem :cry:

after following both your suggestions the test hung in the same part, here is the output all the way from the start to the part it stalls:

filament:/disk/data/munoz/LAPACK/TESTING> ./xeigtsts < sep.in
Tests of the Symmetric Eigenvalue Problem routines

LAPACK VERSION 3.0, released June 30, 1999

The following parameter values will be used:
M: 0 1 2 3 5 20
N: 0 1 2 3 5 20
NB: 1 3 3 3 10
NBMIN: 2 2 2 2 2
NX: 1 0 5 9 1

Relative machine underflow is taken to be 0.117549E-37
Relative machine overflow is taken to be 0.340282E+39
Relative machine precision is taken to be 0.596046E-07

Routines pass computational tests if test ratio is less than 50.00


SST routines passed the tests of the error exits (147 tests done)


SEP: NB = 1, NBMIN = 2, NX = 1
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 2, JTYPE= 1, ISEED=( 1238, 3814, 1038, 2501)

SST -- Real Symmetric eigenvalue problem
Matrix types (see SCHKST for details):

Special Matrices:
1=Zero matrix. 5=Diagonal: clustered entries.
2=Identity matrix. 6=Diagonal: large, evenly spaced.
3=Diagonal: evenly spaced entries. 7=Diagonal: small, evenly spaced.
4=Diagonal: geometr. spaced entries.
Dense Symmetric Matrices:
8=Evenly spaced eigenvals. 12=Small, evenly spaced eigenvals.
9=Geometrically spaced eigenvals. 13=Matrix with random O(1) entries.
10=Clustered eigenvalues. 14=Matrix with large random entries.
11=Large, evenly spaced eigenvals. 15=Matrix with small random entries.
16=Positive definite, evenly spaced eigenvalues
17=Positive definite, geometrically spaced eigenvlaues
18=Positive definite, clustered eigenvalues
19=Positive definite, small evenly spaced eigenvalues
20=Positive definite, large evenly spaced eigenvalues
21=Diagonally dominant tridiagonal, geometrically spaced eigenvalues

Test performed: see SCHKST for details.

N= 2, seed=1238,3814,1038,2501, type 1, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 2, JTYPE= 2, ISEED=( 1238, 3814, 1038, 2501)
N= 2, seed=1238,3814,1038,2501, type 2, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 2, JTYPE= 3, ISEED=( 1238, 3814, 1038, 2501)
N= 2, seed=1238,3814,1038,2501, type 3, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 2, JTYPE= 4, ISEED=( 1292, 3649, 2091, 541)
N= 2, seed=1292,3649,2091, 541, type 4, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 2, JTYPE= 5, ISEED=( 1270, 3808, 1580, 949)
N= 2, seed=1270,3808,1580, 949, type 5, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 2, JTYPE= 6, ISEED=( 154, 2832, 2055, 1165)
N= 2, seed= 154,2832,2055,1165, type 6, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 2, JTYPE= 7, ISEED=( 697, 3633, 1078, 2725)
N= 2, seed= 697,3633,1078,2725, type 7, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 2, JTYPE= 8, ISEED=( 491, 637, 17, 3069)
N= 2, seed= 491, 637, 17,3069, type 8, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 2, JTYPE= 9, ISEED=( 2184, 481, 2621, 3973)
N= 2, seed=2184, 481,2621,3973, type 9, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 2, JTYPE= 10, ISEED=( 1267, 234, 2125, 77)
N= 2, seed=1267, 234,2125, 77, type 10, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 2, JTYPE= 11, ISEED=( 2870, 2302, 1216, 85)
N= 2, seed=2870,2302,1216, 85, type 11, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 2, JTYPE= 12, ISEED=( 3460, 1287, 2245, 413)
N= 2, seed=3460,1287,2245, 413, type 12, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 2, JTYPE= 13, ISEED=( 3331, 1843, 2879, 1573)
N= 2, seed=3331,1843,2879,1573, type 13, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 2, JTYPE= 14, ISEED=( 2412, 2407, 18, 929)
N= 2, seed=2412,2407, 18, 929, type 14, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 2, JTYPE= 15, ISEED=( 287, 1761, 2318, 4077)
N= 2, seed= 287,1761,2318,4077, type 15, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 2, JTYPE= 16, ISEED=( 2838, 442, 1510, 2249)
N= 2, seed=2838, 442,1510,2249, type 16, test(18)= 0.839E+07
SCHKST: SPTEQR(V) returned INFO= 1.
N= 2, JTYPE= 17, ISEED=( 3993, 603, 3137, 1913)
N= 2, seed=3993, 603,3137,1913, type 17, test(14)= 0.839E+07
SCHKST: SPTEQR(V) returned INFO= 1.
N= 2, JTYPE= 18, ISEED=( 2889, 1084, 3667, 1833)
N= 2, seed=2889,1084,3667,1833, type 18, test(14)= 0.839E+07
SCHKST: SPTEQR(V) returned INFO= 1.
N= 2, JTYPE= 19, ISEED=( 688, 2391, 2431, 2009)
N= 2, seed= 688,2391,2431,2009, type 19, test(14)= 0.839E+07
SCHKST: SPTEQR(V) returned INFO= 1.
N= 2, JTYPE= 20, ISEED=( 815, 1456, 184, 2441)
N= 2, seed= 815,1456, 184,2441, type 20, test(14)= 0.839E+07
SCHKST: SPTEQR(V) returned INFO= 1.
N= 2, JTYPE= 21, ISEED=( 164, 2270, 1921, 3129)
N= 2, seed= 164,2270,1921,3129, type 21, test(14)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 3, JTYPE= 1, ISEED=( 119, 1817, 3309, 4073)
N= 3, seed= 119,1817,3309,4073, type 1, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 3, JTYPE= 2, ISEED=( 119, 1817, 3309, 4073)
N= 3, seed= 119,1817,3309,4073, type 2, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 3, JTYPE= 3, ISEED=( 119, 1817, 3309, 4073)
N= 3, seed= 119,1817,3309,4073, type 3, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 3, JTYPE= 4, ISEED=( 2826, 3723, 3039, 1429)
N= 3, seed=2826,3723,3039,1429, type 4, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 3, JTYPE= 5, ISEED=( 3780, 184, 3715, 81)
N= 3, seed=3780, 184,3715, 81, type 5, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 3, JTYPE= 6, ISEED=( 150, 3784, 1497, 2269)
N= 3, seed= 150,3784,1497,2269, type 6, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 3, JTYPE= 7, ISEED=( 1621, 3912, 51, 249)
N= 3, seed=1621,3912, 51, 249, type 7, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 3, JTYPE= 8, ISEED=( 1149, 1441, 1813, 3941)
N= 3, seed=1149,1441,1813,3941, type 8, test(18)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 3, JTYPE= 9, ISEED=( 1244, 2051, 1301, 1465)
N= 3, seed=1244,2051,1301,1465, type 9, test(18)= 0.839E+07
SCHKST: SSTEQR(V) returned INFO= 2.
N= 3, JTYPE= 10, ISEED=( 1148, 1848, 3525, 2141)
N= 3, seed=1148,1848,3525,2141, type 10, test( 9)= 0.839E+07
SCHKST: SSTEBZ(A) returned INFO= 1.
N= 3, JTYPE= 11, ISEED=( 2791, 3827, 970, 1169)
N= 3, seed=2791,3827, 970,1169, type 11, test(18)= 0.839E+07

I appreciate very much your help since I'm not too experienced so I don't know how to tackle this problem, if you have any other suggestions I will be happy to try.

[isaac]

Hi,

I had a similar problem with stalling tests on an opteron running suse 10.0 with gcc 3.3.5.
For example:
./xeigtstz < sep.in > zsep.out 2>&1
would stall.

Another thing that puzzled me was the following:

hailperi@frontend:~/LAPACK/INSTALL>./testsecond
Time for 1,000,000 SAXPY ops = 0.00 seconds
*** Error: Time for operations was zero
Including SECOND, time = 0.00 seconds
Average time for SECOND = 0.00 milliseconds

First I thought that the opteron I used was just too fast, so I modified INSTALL/secondtst.f on line 10: PARAMETER ( NMAX = 100, ITS = 5000 )
ITS gives the iteration number, so I thought the measured time should be linear in ITS. But even with a couple of zeros added and runtime of a couple of seconds (human estimate), the measured time was still 0.

I finally tried gfortran (same options as with g77) which is part of gcc 4.0.2 (and ships with suse 10.0), et voila, I got timings different form zero. And also no more stalls during testing.
(If anyone is interested in further investigation I can open a new thread)


Isaac

[Julien Langou]

Thanks a lot for taking the time to investigate all this. Julien.

[MattR]

Hi there,

I am attempting to build LAPACK on an OSF1/alpha machine using the HP fortran compiler (f90) and am encountering stalling behavior in the testing phase of the installation. In particular

Code: Select all

SEP: Testing Symmetric Eigenvalue Problem routines
./xeigtstz < sep.in > zsep.out 2>&1


never terminates while using all of the available processor time.

The previous posts show that switching compilers allows the installation to complete successfully, but I was hoping someone out there had some insight into getting around this issue for a given compiler, namely the HP Fortran Compiler T5.6a-104657-48FB1.

I have tried many combinations of compiler flags with no beneficial effect.
Many combinations of the following flags were tried.

Code: Select all

-arch host -tune host -align all -O0,1,2,3,4 -fpe0,1,2

Any suggestions on additional things to try to get the testing phase to complete successfully? I tried retrieving the replacement slaed6.f and dlaed6.f files referenced in a previous post, but the links are dead. Do those versions of the files correct any of this stalling behavior? If so, might someone suggest where I could obtain those updated source files? Any help is appreciated.

Thanks,
--Matt

[MattR]

Hello,

Following up on this issue again. I am still unable to get LAPACK to finish testing the "Symmetric Eigenvalue Problem routines" Has anyone come upong any information about preventing this stalling behavior when testing LAPACK?

I am unable to fully build and test it using the Intel fortran compiler (ifort) on i686-Linux as well as the HP compiler on alpha-OSF1.

I've tried all the combinations of compiler/loader flags that I have seen posted anywhere with no success. Ironically, I had LAPACK building and testing perfectly fine using ifort last week. Now for no apparent reason it is stalling just like the OSF1 build.

Any suggestions are heartily appreciated. Thanks.

--Matt

[Julien Langou]

Hello Matt,
can you check with the very fresh brand-new release? LAPACK-3.1.
You can download it at
http://www.netlib.org/lapack/lapack-3.1.0.tgz
There is good chances your problem have been removed. If not, there is good chances, we won't tolerate it much longer :) and we'll get back to you.
Julien

[MattR]

Hi Julien,

I will start testing the new release you suggested right away. Hopefully it clears things up. I shall let you know how this turns out as soon as I can.
Thank you very much.

--Matt

[MattR]

Julien,

As per your suggestion, I built and tested the new LAPACK-3.1.0 using
ifort on i686-Linux,
lf95 on i686-Linux and
f90(hp) on OSF1-alpha
and all completed successfully!

You and the other LAPACK developers have done excellent work and have been most helpful.

Thanks again,
-=_-Matt

[anmunoz]

Dear Julien:

It's Andrés again, I wanted to ask you for help again, I'm installing LAPACK on a Sun4 UltraSPARC III Cu and I'm having again problems with the tests, this time it stalls on:

Testing REAL LAPACK linear equation routines
./xlintsts < stest.in > stest.out 2>&1

I typed

./xlintsts < sep.in

like you told me to last time and this is the output:

Tests of the REAL LAPACK routines
LAPACK VERSION 3.0, released June 30, 1999

The following parameter values will be used:
M : 0 1 2 3 5 20
N : 1 3 3 3 10
NRHS: 1 0
Invalid input value: NNB = 50; must be <= 12
list io: [1012] Unknown error
logical unit 5, named 'stdin'
lately: reading sequential list external IO
part of last format: ext list io
part of last data: rror exits^J1|
Abort
Exit 134

do you have any ideas?

Also I don't know if it matters at all but when I run the sanity test of ATLAS I get the warnings of this type:

"../invtst.c", line 552: warning: enum type mismatch: arg #2

or

"../invtst.c", line 635: warning: enum type mismatch: op "="

Thank you very much for your help,

Andrés

[Julien Langou]

If you typed: ' ./xlintsts < sep.in ' , there is no way this will work, you are using the
linear system tester executable ( ./xlintsts ) with an eigenvalue testing input file
( 'sep.in' ) .

Either type ' ./xlintsts < stest.in ' to test the linear system solvers or type:
' ./xlintsts < sep.in ' to test the symmetric eigenvalue problem drivers.

Julien

[anmunoz]

Hi Julien:

Sorry for that mistake, that only shows how experienced I'm at this :(, so I ran it typing the correct command and it stalls when it reaches this part:

M = 50, N = 5, type 5, test 2, ratio = 13532.
M = 50, N = 5, type 5, test 3, ratio = 0.42136E+06
M = 50, N = 5, type 6, test 1, ratio = 0.44150E+06
M = 50, N = 5, type 6, test 2, ratio = 58113.
M = 50, N = 5, type 6, test 3, ratio = 0.66361E+06
M = 50, N = 10, type 2, test 2, ratio = 0.62030E+35
M = 50, N = 10, type 2, test 3, ratio =Inf
M = 50, N = 10, type 3, test 1, ratio = 0.49955E+06
M = 50, N = 10, type 3, test 2, ratio = 26888.
M = 50, N = 10, type 3, test 3, ratio = 0.57525E+06
M = 50, N = 10, type 4, test 1, ratio = 0.49437E+06
M = 50, N = 10, type 4, test 2, ratio = 0.15715E+06
M = 50, N = 10, type 4, test 3, ratio = 0.73951E+06
M = 50, N = 10, type 5, test 1, ratio = 0.36321E+07
M = 50, N = 10, type 5, test 2, ratio = 39036.
M = 50, N = 10, type 5, test 3, ratio = 0.55326E+06
M = 50, N = 10, type 6, test 1, ratio = 0.49428E+06
M = 50, N = 10, type 6, test 2, ratio = 0.18626E+06
M = 50, N = 10, type 6, test 3, ratio = 0.72327E+06

the strange thing is that in my previous installation on a linux I was having a similar problem but with the eigenvalues test and I don't know how did I fix it :(.

Thank you very much for your help,

Andrés

[anmunoz]

Hi Julien:

I wanted to ask you, in the ATLAS errata they mention that the LAPACK testers flag the ATLAS TSRMs as bad, do you think this might be the problem?

thank you very much,

Andrés

[Dominique d'Humieres]

If this may help:

> [karma] dominiq/test% diff -u LAPACK-gfc-O3u2/SRC/slarre.f~ LAPACK-gfc-O3u5/SRC/slarre.f
> --- LAPACK-gfc-O3u2/SRC/slarre.f~ Thu Nov 4 20:26:29 1999
> +++ LAPACK-gfc-O3u5/SRC/slarre.f Sun Feb 5 20:35:34 2006
> @@ -251,7 +251,11 @@
> J = IBEGIN
> DO 110 I = 1, IN - 1
> WORK( I ) = D( J ) + S
> - WORK( 2*IN+I ) = ONE / WORK( I )
> + IF (WORK(I).EQ.ZERO) THEN
> + WORK( 2*IN+I ) = ONE / (TAU*DELTA)
> + ELSE
> + WORK( 2*IN+I ) = ONE / WORK( I )
> + END IF
> * WORK( N+I ) = ( E( I ) * D( I ) ) / WORK( I )
> WORK( IN+I ) = ( E( J )*D( J ) )*WORK( 2*IN+I )
> S = S*WORK( IN+I )*E( J ) - TAU
>
> The problem is that under certain circumstances, WORK(I) can be zero.
> This induces arithmetic with INF and an infinite loop in some tests of
>
> ./xeigtstc < sep.in > csep.out 2>&1
>
> I traced the problem a year ago and I don't remember very well the details,
> in particular at what level of optimization this appears. Also this can be
> architecture dependent: I am using a G5 under OSX 10.3.9.

This was for gfortran 4.3 with -O3 -funroll-loops. It is possible that
other compilers and architectures have a similar problem on different
kind of real/complex. Note also this was for lapack 3.0 and seems
to have disappeared in lapack 3.1.

Dominique