ScaLAPACK Archives

[Scalapack] Process Grid Shape Trouble

The MPI that was giving problems is:

 /home/sw/ifort10.0mpich2/bin/mpich2version
MPICH2 Version:         1.0.6
MPICH2 Release date:    Unknown, built on Tue Oct  2 14:37:41 CST 2007
MPICH2 Device:          ch3:sock
MPICH2 configure:       --prefix=/home/sw/ifort10.0mpich2
MPICH2 CC:      gcc  -O2
MPICH2 CXX:     c++  -O2
MPICH2 F77:     /opt/intel/fc/10.0.023/bin/ifort  -O2
MPICH2 F90:     /opt/intel/fc/10.0.023/bin/ifort  -O2
MPICH2 Patch level:      none

The MPI that I have recently installed is:

/home/sadowski/LIBS/mpich2-install/bin/mpich2version
MPICH2 Version:         1.1.1p1
MPICH2 Release date:    Unknown, built on Wed Oct  7 10:57:10 CST 2009
MPICH2 Device:          ch3:nemesis
MPICH2 configure:       --prefix=/home/sadowski/LIBS/mpich2-install
MPICH2 CC:      gcc  -O2
MPICH2 CXX:     c++  -O2
MPICH2 F77:     g77  -O2
MPICH2 F90:     ifort  -O2


Hope this helps.

Jason

Julien Langou wrote:

Just for information which is the nonworking MPI? which is the working 
one?

Note:
* This MPI may be simply buggy
* BLACS may rely on specificty of MPI that are not explicitly in the 
standard but in most of MPI versions and so well

In any case, look like BLACS and this MPI are happy together. It would 
be good for us to know.

--j



On Thu, 8 Oct 2009, Jason Sadowski wrote:

Hi Jakub and Julien,

Thank you so much for your replies.  I've tried installing a new 
version of MPI and this seems to have resolved my issue.  I now get 
the correct
results independent of the shape of the process grid.

Thanks for the suggestions!

Jason

Jakub Kurzak wrote:
      Let throw in my $0.02.
      For the most part I agree with Julien.
      It happened to me before that a vendor MPI implementation was 
buggy in the shared memory part.
      Definitely trying a different MPI is a good idea.
      I saw the word "hyperthreading".
      Perhaps you meant "hypertransport"?
      I don't think hypertransport changes the memory consistency 
model, so it should not be an issue.
      I hope you're not using hyperthreading.
      There is no performance gain to be expected from that for 
ScaLAPACK.
      Jakub


      On Wed, Oct 7, 2009 at 10:47 AM, Julien Langou 
<julien.langou@Domain.Removed> wrote:

            This is indeed baffling (as you put it). There is not 
much reason for a ScaLAPACK code to perform correctly on a given
            machine and to perform incorrectly on another one. 
Moreover if you managed to get the incorrect installation to correctly
            work from time to time. A suggestion is to try a 
different MPI ... I am not at all convinced that this will change the
            problem but I have no other suggestion.
            --j


            On Tue, 6 Oct 2009, Jason Sadowski wrote:

                  Hello All,

                  I am currently using ScaLapack to perform research 
at the University of Saskatchewan and have encountered some
                  peculiar problems.  I am using the PDSYEVD routine to
                  perform the diagonalization of a matrix however 
occasionally I get the wrong results.  I believe the problem may
                  be a memory issue, but I am unable to determine what
                  is causing it.  I am hoping somebody can give me an 
idea as to where this problem may be occurring.

                  The matrix I am trying to diagonalize is 3200x3200 
and I am using a block size of 32x32.  I've noticed that the
                  eigenvalues returned by PDSYEVD are different
                  depending on the particular shape of the process 
grid I choose.  Inside my program I have the lines which
                  perform:

                  PRINT Matrix A
                  Diagonalize ( A, Z)
                  PRINT Matrix(Z)

                  Where A is the input matrix, and Z is the matrix of 
eigenvectors returned from PDSYEVD.  I want to perform this
                  calculation using 4 processors, so I can choose
                  process grids of 1x4, 2x2, or 4x1.  Here are the 
results of performing such calculations:

                  1) Matrix A is independent of the grid shape.  It 
is the same in all cases.
                  2) Matrix Z is INCORRECT for grid shapes of 2x2 and 
1x4.
                  3) Process grid 4x1 gives the CORRECT values for 
MatrixZ.

                  As I have mentioned before, I believe this may be 
some kind of memory issue.  The reason I think this is because
                  I can perform the previous calculations on a
                  different machine (called vortex) with no errors.  
Identical code ran on Vortex gives correct values of MatrixZ
                  for all grid shapes.  Only on this specific machine
                  (iglu) do the process grid problems arise.  To my 
knowledge the only difference between the machines is that
                  Vortex is a quad core machine with 8 GB of RAM, while
                  iglu is a dual core machine ( with hyperthreading 
enabled ) and 4 GB of RAM. 

                  First I should ask are there any known issues with 
ScaLapack's memory distribution scheme and hyperthreading
                  technology?  I realize this email is quite lengthy, 
but I
                  am completely baffled as to why this problem is 
occurring.  Any ideas or comments as to where I should be looking
                  would be greatly appreciated.

                  Sincerely,

                  Jason Sadowski
                  --
                  Jason Sadowski
                  jason.sadowski@Domain.Removed
                  Cell: 1-306-227-6066
                  "He who never made a mistake never made a 
discovery" - Samuel Smiles


_______________________________________________
Scalapack mailing list
Scalapack@Domain.Removed
http://lists.eecs.utk.edu/mailman/listinfo/scalapack



-- 
Jason Sadowski
jason.sadowski@Domain.Removed
Cell: 1-306-227-6066
"He who never made a mistake never made a discovery" - Samuel Smiles



-- 
Jason Sadowski
jason.sadowski@Domain.Removed <mailto:jason.sadowski@Domain.Removed>
Cell: 1-306-227-6066
/"He who never made a mistake never made a discovery"/ - Samuel Smiles

<Prev in Thread] Current Thread [Next in Thread>


For additional information you may use the LAPACK/ScaLAPACK Forum.
Or one of the mailing lists, or