Using Matlab Mex file to wrap around ScaLAPACK

Open discussion regarding features, bugs, issues, vendors, etc.

Using Matlab Mex file to wrap around ScaLAPACK

Postby sket16 » Mon May 30, 2011 4:16 pm

Dear Forum Users,

Most of the code developed within my group is written in Matlab and there are specific tasks such as solving a dense linear system very computationally intensive. My idea was to write the Matlab Mex file in C to wrap around the needed functions in ScaLAPACK, and present the Matlab interface to other group members.

Currently, I was able to write a mex file that contains calls to MPI_Init()/MPI_Finalize(). A TestHarness.m was created to call this mex file, and the TestHarness was invoked on the command line like

Code: Select all
mpiexec -np 10 matlab -nodesktop -nosplash -nojvm -r TestHarness


However, the Segmentation Violation was detected when I embedded the Cblacs_gridinit() in the mex function. A portion of the mex file contains the following lines

Code: Select all
 

  MPI_Init(0, 0);                                                                                                           
  MPI_Comm_rank(MPI_COMM_WORLD, &myrank_mpi);                                                                               
  MPI_Comm_size(MPI_COMM_WORLD, &nprocs_mpi);                                                                               
  printf("number of processes %d and my rank %d\n", nprocs_mpi, myrank_mpi);                                                 
                                                                                                                             
  npcol = 2; nprow = 2;                                                                                                     
                                                                                                                             
  if (nprow*npcol > nprocs_mpi) {                                                                                           
    MPI_Finalize(); exit(1);                                                                                                 
  }                                                                                                                         
                                                                                                                             
  /* Initilize the processor grid */                                                                                         
  Cblacs_pinfo(&iam, &nprocs);    /* executed OK */                                                                                         
                                                                                                                             
  Cblacs_get(-1, 0, &ictxt);   /* executed OK */                                                 

  Cblacs_gridinit(&ictxt, "Col-major", nprow, npcol); /* throw segmentation violation error in Matlab */                                 
                                                                                                                             
  Cblacs_gridinfo(ictxt, &nprow, &npcol, &myrow, &mycol); /* not executed */



I have not identified anyone on the internet with a report of mex wrapper of Matlab. My question is: is this idea valid/possible or is there any more appropriate solution? My second question is: what could possibly be the cause of the segmentation violation?

Any comment is welcome. Thank you for reading this post.


Kevin
sket16
 
Posts: 8
Joined: Tue May 17, 2011 4:10 pm

Re: Using Matlab Mex file to wrap around ScaLAPACK

Postby admin » Tue May 31, 2011 4:31 pm

Your code seems fine.
I think you should check your BLACS and ScaLAPACK libraries.
Did you try running the C code without matlab?
Julie
admin
Site Admin
 
Posts: 499
Joined: Wed Dec 08, 2004 7:07 pm

Re: Using Matlab Mex file to wrap around ScaLAPACK

Postby sket16 » Thu Jun 02, 2011 8:28 am

Hi Julie,

Thank you for the reply. I indeed fell back to a C implementation before debugging with the Matlab mex function. I notice that in the Makefile, I have a mysterious "-fPIC" flag when compiling the function contains ScaLAPACK calls.

Code: Select all
all:
   gcc -c ../src/utility.c -I../include
   gcc -c -fPIC ../src/solveDenseRealSystem.c -I../include -I$(MPI_INC)   
   mpicc utility.o solveDenseRealSystem.o testSMI.c -o testSMI -I../include  -I$(MPI_INC) -L$(MPI_LIB) -L$(LIB) -llapack_atlas -lscalapack -lreflapack -lblacsF77 -lblacs -lrefblas -lmpi_f77


I am not entirely sure about the use of "-fPIC" and could not remember the exact rational why -fPIC was placed there. It vaguely recalls that -fPIC generate the position independent code. Running

Code: Select all
mpirun -np 6 testSMI


gives the following error

Code: Select all
[windu:10047] *** Process received signal ***
[windu:10048] *** Process received signal ***
[windu:10049] *** Process received signal ***
[windu:10048] Signal: Segmentation fault (11)
[windu:10048] Signal code: Address not mapped (1)
[windu:10048] Failing at address: 0xbbb86a38
[windu:10049] Signal: Segmentation fault (11)
[windu:10049] Signal code: Address not mapped (1)
[windu:10049] Failing at address: 0x5565a38
[windu:10052] *** Process received signal ***
[windu:10052] Signal: Segmentation fault (11)
[windu:10052] Signal code: Address not mapped (1)
[windu:10052] Failing at address: 0xba68fa38
[windu:10047] Signal: Segmentation fault (11)
[windu:10047] Signal code: Address not mapped (1)
[windu:10047] Failing at address: 0x49127a38
[windu:10048] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfc60) [0x7fe9ba88ac60]
[windu:10048] [ 1] /home/kzhu/devcode/openmpi-1.4.3/lib/libmpi.so.0(PMPI_Comm_group+0x46) [0x7fe9bb884c46]
[windu:10048] [ 2] testSMI(BI_TransUserComm+0x26) [0x45eaa6]
[windu:10048] [ 3] testSMI(Cblacs_gridmap+0x1c5) [0x45ac9d]
[windu:10048] [ 4] testSMI(Cblacs_gridinit+0x10d) [0x45aac9]
[windu:10048] [ 5] testSMI(solveDenseRealSystem+0x146) [0x404fca]
[windu:10048] [ 6] testSMI(main+0x67) [0x40563f]
[windu:10048] [ 7] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xff) [0x7fe9ba505eff]
[windu:10049] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfc60) [0x7f5b04269c60]
[windu:10049] [ 1] /home/kzhu/devcode/openmpi-1.4.3/lib/libmpi.so.0(PMPI_Comm_group+0x46) [0x7f5b05263c46]
[windu:10049] [ 2] testSMI(BI_TransUserComm+0x26) [0x45eaa6]
[windu:10049] [ 3] testSMI(Cblacs_gridmap+0x1c5) [0x45ac9d]
[windu:10049] [ 4] testSMI(Cblacs_gridinit+0x10d) [0x45aac9]
[windu:10049] [ 5] testSMI(solveDenseRealSystem+0x146) [0x404fca]
[windu:10049] [ 6] testSMI(main+0x67) [0x40563f]
[windu:10049] [ 7] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xff) [0x7f5b03ee4eff]
[windu:10049] [ 8] testSMI() [0x404d09]
[windu:10049] *** End of error message ***
[windu:10048] [ 8] testSMI() [0x404d09]
[windu:10048] *** End of error message ***
[windu:10052] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfc60) [0x7fdfb9393c60]
[windu:10052] [ 1] /home/kzhu/devcode/openmpi-1.4.3/lib/libmpi.so.0(PMPI_Comm_group+0x46) [0x7fdfba38dc46]
[windu:10052] [ 2] testSMI(BI_TransUserComm+0x26) [0x45eaa6]
[windu:10052] [ 3] testSMI(Cblacs_gridmap+0x1c5) [0x45ac9d]
[windu:10052] [ 4] testSMI(Cblacs_gridinit+0x10d) [0x45aac9]
[windu:10052] [ 5] testSMI(solveDenseRealSystem+0x146) [0x404fca]
[windu:10052] [ 6] testSMI(main+0x67) [0x40563f]
[windu:10052] [ 7] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xff) [0x7fdfb900eeff]
[windu:10052] [ 8] testSMI() [0x404d09]
[windu:10052] *** End of error message ***
[windu:10047] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfc60) [0x7fc847e2bc60]
[windu:10047] [ 1] /home/kzhu/devcode/openmpi-1.4.3/lib/libmpi.so.0(PMPI_Comm_group+0x46) [0x7fc848e25c46]
[windu:10047] [ 2] testSMI(BI_TransUserComm+0x26) [0x45eaa6]
[windu:10047] [ 3] testSMI(Cblacs_gridmap+0x1c5) [0x45ac9d]
[windu:10047] [ 4] testSMI(Cblacs_gridinit+0x10d) [0x45aac9]
[windu:10047] [ 5] testSMI(solveDenseRealSystem+0x146) [0x404fca]
[windu:10047] [ 6] testSMI(main+0x67) [0x40563f]
[windu:10047] [ 7] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xff) [0x7fc847aa6eff]
--------------------------------------------------------------------------
mpirun noticed that process rank 2 with PID 10049 on node windu exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------
[windu:10051] *** Process received signal ***
[windu:10051] Signal: Segmentation fault (11)
[windu:10051] Signal code: Address not mapped (1)
[windu:10051] Failing at address: 0xe18a9a38
[windu:10051] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfc60) [0x7f83e05adc60]
[windu:10051] [ 1] /home/kzhu/devcode/openmpi-1.4.3/lib/libmpi.so.0(PMPI_Comm_group+0x46) [0x7f83e15a7c46]
[windu:10051] [ 2] testSMI(BI_TransUserComm+0x26) [0x45eaa6]
[windu:10051] [ 3] testSMI(Cblacs_gridmap+0x1c5) [0x45ac9d]
[windu:10051] [ 4] testSMI(Cblacs_gridinit+0x10d) [0x45aac9]
[windu:10051] [ 5] testSMI(solveDenseRealSystem+0x146) [0x404fca]
[windu:10051] [ 6] testSMI(main+0x67) [0x40563f]
[windu:10051] [ 7] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xff) [0x7f83e0228eff]
[windu:10051] [ 8] testSMI() [0x404d09]
[windu:10051] *** End of error message ***
[windu:10050] *** Process received signal ***
[windu:10050] Signal: Segmentation fault (11)
[windu:10050] Signal code: Address not mapped (1)
[windu:10050] Failing at address: 0x6230aa38
[windu:10050] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfc60) [0x7f6e6100ec60]
[windu:10050] [ 1] /home/kzhu/devcode/openmpi-1.4.3/lib/libmpi.so.0(PMPI_Comm_group+0x46) [0x7f6e62008c46]
[windu:10050] [ 2] testSMI(BI_TransUserComm+0x26) [0x45eaa6]
[windu:10050] [ 3] testSMI(Cblacs_gridmap+0x1c5) [0x45ac9d]
[windu:10050] [ 4] testSMI(Cblacs_gridinit+0x10d) [0x45aac9]
[windu:10050] [ 5] testSMI(solveDenseRealSystem+0x146) [0x404fca]
[windu:10050] [ 6] testSMI(main+0x67) [0x40563f]
[windu:10050] [ 7] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xff) [0x7f6e60c89eff]
[windu:10050] [ 8] testSMI() [0x404d09]
[windu:10050] *** End of error message ***


If I remove "-fPIC" in the Make file, the code also compiles and finishes with the correct result. I do not understand what difference the -fPIC flag has played here. Do you have some feeling about this flag in the context of ScaLAPACK? Why would the position independent code cause the segmentation fault? Luckily, I could get the C code running smoothly and move on to the compiling the mex file.


Kevin
sket16
 
Posts: 8
Joined: Tue May 17, 2011 4:10 pm

Re: Using Matlab Mex file to wrap around ScaLAPACK

Postby admin » Thu Jun 02, 2011 3:41 pm

"-fPIC" is used for shared library.
By default we provide a build for static library for ScaLAPACK.
Glad it works now.
Julie
admin
Site Admin
 
Posts: 499
Joined: Wed Dec 08, 2004 7:07 pm


Return to User Discussion

Who is online

Users browsing this forum: Bing [Bot], Yahoo [Bot] and 3 guests