ScaLAPACK Archives

[Scalapack] MPI errors with BLACS on Ubuntu 9.10

Dear BLACS developers

I have installed the default Ubuntu 9.10 packages with Blacs/Scalapack: 
libblacs-mpi1, libblacs-mpi-dev and so on, using OpenMPI.  I get an error 
when attempting to call Cblacs_gridinit with certain MPI communicators.

I have attached a simple example program (blacs.c and a corresponding 
makefile) which exhibits this behaviour.  The same program runs fine on 
several different non-Ubuntu computers with BLACS/Scalapack, which leads 
me to believe that it could be related to the debian package.

The program is run with 8 cpus (mpirun -np 8 testblacs).  If 
Cblacs_gridinit is called on a subcommunicator for ranks 0, 1, 2, 3, then 
everything works.  If instead it is called on ranks 0, 2, 4, 6, then it 
gives the following error:

[askm:3868] *** An error occurred in MPI_Group_incl
[askm:3868] *** on communicator MPI_COMM_WORLD
[askm:3868] *** MPI_ERR_RANK: invalid rank
[askm:3868] *** MPI_ERRORS_ARE_FATAL (your MPI job will now abort)
--------------------------------------------------------------------------
mpirun has exited due to process rank 0 with PID 3868 on
node askm exiting without calling "finalize". This may
have caused other processes in the application to be
terminated by signals sent by mpirun (as reported here).
--------------------------------------------------------------------------

(The above error is caused specifically by Cblacs_gridinit, not the 
explicit creation of the MPI group in the program)

In case this is relevant, here are the output and error files from running 
the cblacs tests as per the command 'mpirun -np 4 
cblacs_test_shared-openmpi' (-np 8 gives identical output):

http://www.student.dtu.dk/~ashj/opendir/cblacstest.out
http://www.student.dtu.dk/~ashj/opendir/cblacstest.err

None of the tests fail, but some of them are skipped.

Any help to understand or fix this, or other places to direct this 
question or report it if it is a bug, would be greatly appreciated.

Best regards
Ask Hjorth Larsen

Center for Atomic-scale Materials Design
Technical University of Denmark
-------------- next part --------------
A non-text attachment was scrubbed...
Name: blacs.c
Type: text/x-csrc
Size: 1357 bytes
Desc: 
Url : 
http://lists.eecs.utk.edu/mailman/private/scalapack/attachments/20091127/35928191/blacs.bin
 
-------------- next part --------------
ubuntu: blacs.c
        mpicc -o testblacs -lblacs-openmpi -lblacsCinit-openmpi 
-lscalapack-openmpi -llapack blacs.c

thul: blacs.c
        /opt/openmpi/1.3.3-1.el5.fys.gfortran43.4.3.2/bin/mpicc -o testblacs 
-I/opt/openmpi/1.3.3-1.el5.fys.gfortran43.4.3.2/include 
-L/opt/openmpi/1.3.3-1.el5.fys.gfortran43.4.3.2/lib64 
-L/opt/acml/4.3.0/gfortran4364/lib 
-L/opt/blacs/1.1/24.el5.fys.gfortran43.4.3.2.openmpi.1.3.3/lib64 
-L/opt/scalapack/1.8.0/1.el5.fys.gfortran43.4.3.2.openmpi.1.3.3.acml.4.3.0.acml.4.3.0/lib64
 -lgfortran -lscalapack -lmpiblacsF77init -lmpiblacs -lacml -lmpi -lmpi_f77 
-Wl,-rpath=/opt/openmpi/1.3.3-1.el5.fys.gfortran43.4.3.2/lib64,-rpath=/opt/acml/4.3.0/gfortran4364/lib,-rpath=/opt/blacs/1.1/24.el5.fys.gfortran43.4.3.2.openmpi.1.3.3/lib64,-rpath=/opt/scalapack/1.8.0/1.el5.fys.gfortran43.4.3.2.openmpi.1.3.3.acml.4.3.0.acml.4.3.0/lib64
 blacs.c

newton: blacs.c
        /usr/lib/openmpi/1.3.2-gcc/bin/mpicc -o testblacs 
-I/usr/lib/openmpi/1.3.2-gcc/include 
-L/opt/blacs/1.1/24/1.el5.fys.gfortran.4.1.2.openmpi.1.3.2/lib 
-L/opt/scalapack/1.8.0/1.el5.fys.gfortran.4.1.2.openmpi.1.3.2.blas.3.0.37.el5.lapack.3.0.37.el5/lib/
 -L/usr/lib/openmpi/1.3.2-gcc/lib/ -lmpiblacs -lblas -llapack -lmpi 
-lmpiblacsCinit -lmpi_f77 
-Wl,-rpath=/usr/lib/openmpi/1.3.2-gcc/lib/,-rpath=/opt/blacs/1.1/24/1.el5.fys.gfortran.4.1.2.openmpi.1.3.2/lib,-rpath=/opt/scalapack/1.8.0/1.el5.fys.gfortran.4.1.2.openmpi.1.3.2.blas.3.0.37.el5.lapack.3.0.37.el5/lib/
 blacs.c

.PHONY: clean

clean:
        rm testblacs

<Prev in Thread] Current Thread [Next in Thread>


For additional information you may use the LAPACK/ScaLAPACK Forum.
Or one of the mailing lists, or