pdgemr2d usage,

Open discussion regarding features, bugs, issues, vendors, etc.

pdgemr2d usage,

Postby yildirim » Thu Aug 09, 2007 1:22 pm

Hi,

I am trying to distribute a matrix on root process to other processes. In order to do that, I choose scalapack subrouinte pdgemr2d,

It works for some cases, but it sometimes gives error, which is as follows
"xxmr2d:out of memory"

I checked all allocation very carefully and it does correctly, but it is still problematic for some cases,

has anybody encontoured this kind of problem when using pdgemr2d subroutine,

I have copied important part of my applications which is written in C++,

sl_init( &ictxt, nprow, npcol );
bool isRootNode = false;

if ( myrow == 0 && mycol == 0 ) isRootNode = true;

//loaad matrix on root node and create desc for it
int descA0[9], lm_a0, info, rsrc = 0, csrc = 0;
if ( isRootNode ) {
lm_a0 = numroc( M, M, myrow, rsrc, nprow);
assert( input->getVecSpaceTimeMtrx().at(dim).rows() == lm_a0 );
descinit( descA0, M, N, M, N, rsrc, csrc, rootNodeContext, std::max(1,lm_a0), &info );
assert( !info );
} else {
for ( int i = 0; i < 9; i++ ) descA0[i] = 0;
descA0[1] = -1;
}

//create desc for distributed matrix
int lm_a, ln_a;
rsrc = 0; csrc = 0;

lm_a = numroc( M, block_size, myrow, rsrc, nprow);
ln_a = numroc( N, block_size, mycol, csrc, npcol);
assert( input->getLocalVecSpaceTimeMtrx().at(dim).rows() == lm_a );
assert( input->getLocalVecSpaceTimeMtrx().at(dim).cols() == ln_a );
descinit( descA.at(dim).data(), M, N, block_size, block_size, rsrc, csrc, ictxt, std::max(1,lm_a), &info);
assert( !info );

}


//distribute data
int IA = 1, JA = 1, IB = 1, JB = 1;

pdgemr2d( M, N, input->getVecSpaceTimeMtrxP(dim), IA, JA, descA0, input->getLocalVecSpaceTimeMtrxP(dim),
IB, JB, descA.at(dim).data(), descA.at(dim)(1));


I hope that you can help me out,

thanks anyway,
yildirim
 
Posts: 3
Joined: Thu Jun 28, 2007 2:09 pm

Postby thombos » Tue May 13, 2008 8:20 pm

I have the same problem. The error occurs as soon my matrix is larger than 2GB. I'm investigating the issue now, but if anyone knows a solution, I'd be glad to hear it.

Thomas
thombos
 
Posts: 30
Joined: Mon Nov 26, 2007 8:41 pm

Postby Julien Langou » Tue May 13, 2008 8:34 pm

Julien Langou
 
Posts: 734
Joined: Thu Dec 09, 2004 12:32 pm
Location: Denver, CO, USA

Postby thombos » Tue May 13, 2008 10:23 pm

Interesting. Unfortunaly this is not the problem here. What we are looking at is p_gemr2d calling mr2d_malloc(int n), which is already negative at the sizes we are looking at. I'll try to work around it, by making it unsigned and long, but its also in the long term not a good idea that p_gemr2d needs a temporary buffer the size of the matrix its supposed to distribute.

This is a bit off-topic, but has anyone looked at BLACS or tried to optimize the whole BLACS communication shebang in ScaLapack in recent years ?

Thomas

Edit: changing the function mr2d_malloc to have an argument unsigned long int n instead of int n, does the trick.
thombos
 
Posts: 30
Joined: Mon Nov 26, 2007 8:41 pm

Re: pdgemr2d usage,

Postby papis » Mon Jul 21, 2008 7:45 am

Can you tell us ,where did you do the changes,and what changes exactly.?
Does scalapack 1.8 has a fix or sth?

Ty very much
papis
 
Posts: 11
Joined: Wed Nov 14, 2007 9:49 am

Re: pdgemr2d usage,

Postby thombos » Tue Jul 22, 2008 8:34 am

Hello,

it is very simple:
You need to modify the file REDIST/SRC/pgemraux.c and change
void *
mr2d_malloc(n)
unsigned int n;
{

to
void *
mr2d_malloc(n)
unsigned long int n;
{

That should do it. Needless to say this will only work on a 64 bit operating system.

Thomas
thombos
 
Posts: 30
Joined: Mon Nov 26, 2007 8:41 pm

Re: pdgemr2d usage,

Postby papis » Wed Jul 23, 2008 6:24 am

Thank you very much for the info,It was really helpfull.I just made the changes and everything seems to work ok.

Thanks a lot
papis
 
Posts: 11
Joined: Wed Nov 14, 2007 9:49 am


Return to User Discussion

Who is online

Users browsing this forum: Bing [Bot] and 6 guests