diagonalizing large matrices with dsyevr

Post here if you have a question about LAPACK performance

diagonalizing large matrices with dsyevr

Postby apologia » Thu Oct 06, 2011 1:48 pm

I am having issues with using dsyevr to diagonalize matrices with a size beyond 16400^2
(anything smaller works). For such matrices I get errors of the form (generated
from within dsyevr)

flapack_renorm_b(5776) malloc: *** mmap(size=18446744071998529536) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug

The matrix size where things start to go badly is about 2GB. If this was a 32bit machine it would make
sense but I'm on a 64bit mac pro with 16GB of memory. I see that the mmap error corresponds to
a size 18446744071998529536 ~ 2^64 which I guess would be at the edge of the ability of malloc to
map addresses, but I am unsure of what dsyevr could be doing to generate such errors. Assuming the
problem is not hidden or at least set up to be brought out by dsyevr by my own code, it looks like dsyevr
is causing severe memory fragmentation.

When such an error occurs, the program does not at first crash but instead continues
to generate such errors for a while. It however does eventually throw out a segmentation fault.

Has anyone reported similar issues with large matrices? I've tried a variety of lapack libraries (the ones
that come with atlas through fink (which was set for 64bit) as well as the ones apple includes with its Developer toolkit.

Any thoughts would be appreciated.

Thanks, Robert
apologia
 
Posts: 4
Joined: Tue Jan 25, 2011 1:41 pm

Re: diagonalizing large matrices with dsyevr

Postby Julien Langou » Thu Oct 06, 2011 3:56 pm

We might use a 32-bit integer to represent a number above N^2. That would explain the bug as an integer overflow. We need to have a look at this. Julien.
Julien Langou
 
Posts: 727
Joined: Thu Dec 09, 2004 12:32 pm
Location: Denver, CO, USA

Re: diagonalizing large matrices with dsyevr

Postby apologia » Thu Oct 06, 2011 6:20 pm

Ok thanks.

Robert
apologia
 
Posts: 4
Joined: Tue Jan 25, 2011 1:41 pm

Re: diagonalizing large matrices with dsyevr

Postby seobts » Fri Dec 02, 2011 10:32 pm

1)How we can investiagte the original problem (with dsyevr) you reported?
>>> for packed storage with dspevx...
2)Do you mean dspevx executed > then 2 days!. What was the order of the input matrixes?
3)The list of threaded functions, you can find into User's Guide, chapter 6, Using the Intel® MKL Parallelism.
seobts
 
Posts: 1
Joined: Fri Dec 02, 2011 10:22 pm

Re: diagonalizing large matrices with dsyevr

Postby yurii_ » Tue Dec 06, 2011 6:43 pm

I do not advise to use algorithms of diagonalization of the big matrixes from LAPACK package. It is better to use Intel MKL package. You receive magnification of speed approximately in 10 times.
yurii_
 
Posts: 4
Joined: Tue Dec 06, 2011 10:40 am


Return to Performance

Who is online

Users browsing this forum: No registered users and 1 guest