ScaLAPACK Archives

[Scalapack] Process Grid Shape Trouble

Let throw in my $0.02.
For the most part I agree with Julien.
It happened to me before that a vendor MPI implementation was buggy in the
shared memory part.
Definitely trying a different MPI is a good idea.
I saw the word "hyperthreading".
Perhaps you meant "hypertransport"?
I don't think hypertransport changes the memory consistency model, so it
should not be an issue.
I hope you're not using hyperthreading.
There is no performance gain to be expected from that for ScaLAPACK.

On Wed, Oct 7, 2009 at 10:47 AM, Julien Langou

This is indeed baffling (as you put it). There is not much reason for a
ScaLAPACK code to perform correctly on a given machine and to perform
incorrectly on another one. Moreover if you managed to get the incorrect
installation to correctly work from time to time. A suggestion is to try a
different MPI ... I am not at all convinced that this will change the
problem but I have no other suggestion.

On Tue, 6 Oct 2009, Jason Sadowski wrote:

 Hello All,

I am currently using ScaLapack to perform research at the University of
Saskatchewan and have encountered some peculiar problems.  I am using the
PDSYEVD routine to
perform the diagonalization of a matrix however occasionally I get the
wrong results.  I believe the problem may be a memory issue, but I am unable
to determine what
is causing it.  I am hoping somebody can give me an idea as to where this
problem may be occurring.

The matrix I am trying to diagonalize is 3200x3200 and I am using a block
size of 32x32.  I've noticed that the eigenvalues returned by PDSYEVD are
depending on the particular shape of the process grid I choose.  Inside my
program I have the lines which perform:

PRINT Matrix A
Diagonalize ( A, Z)
PRINT Matrix(Z)

Where A is the input matrix, and Z is the matrix of eigenvectors returned
from PDSYEVD.  I want to perform this calculation using 4 processors, so I
can choose
process grids of 1x4, 2x2, or 4x1.  Here are the results of performing
such calculations:

1) Matrix A is independent of the grid shape.  It is the same in all
2) Matrix Z is INCORRECT for grid shapes of 2x2 and 1x4.
3) Process grid 4x1 gives the CORRECT values for MatrixZ.

As I have mentioned before, I believe this may be some kind of memory
issue.  The reason I think this is because I can perform the previous
calculations on a
different machine (called vortex) with no errors.  Identical code ran on
Vortex gives correct values of MatrixZ for all grid shapes.  Only on this
specific machine
(iglu) do the process grid problems arise.  To my knowledge the only
difference between the machines is that Vortex is a quad core machine with 8
GB of RAM, while
iglu is a dual core machine ( with hyperthreading enabled ) and 4 GB of

First I should ask are there any known issues with ScaLapack's memory
distribution scheme and hyperthreading technology?  I realize this email is
quite lengthy, but I
am completely baffled as to why this problem is occurring.  Any ideas or
comments as to where I should be looking would be greatly appreciated.


Jason Sadowski
Jason Sadowski
Cell: 1-306-227-6066
"He who never made a mistake never made a discovery" - Samuel Smiles

Scalapack mailing list

-------------- next part --------------
An HTML attachment was scrubbed...

<Prev in Thread] Current Thread [Next in Thread>

For additional information you may use the LAPACK/ScaLAPACK Forum.
Or one of the mailing lists, or