PLASMA 2.4.0 and OpenMP compatibility issue?

Open forum for general discussions relating to PLASMA.

Re: PLASMA 2.4.0 and OpenMP compatibility issue?

Postby srdegraaf » Thu Jul 07, 2011 10:39 am

Mathieu,

That's great that you found a/the bug! As far as I can tell, I'm NOT using hwloc, so I'm looking forward to your more general patch tomorrow.

Thanks!

Stuart
srdegraaf
 
Posts: 12
Joined: Sat Jul 02, 2011 12:00 am

Re: PLASMA 2.4.0 and OpenMP compatibility issue?

Postby srdegraaf » Thu Jul 07, 2011 12:37 pm

Mathieu,

Actually, I don't know what hwloc is beyond a cursory googling of it. However, when I do
find / -mount -name "*hwloc*"
it doesn't turn anything up, and neither does
rpm -qa |grep hwloc
so I conclude that I'm not "using" it.

I tried to do "patch <yourpatchfile" down in the build/plasma_2.4.0/control directory to update the three files, and patch barfed with the message:
patch: **** malformed patch at line 16: }

Anyway, I did the patching by hand, and then figured out how to rebuild (since the top level python installer over-writes the three patched files with their originals). I cd-ed to build/plasma_2.4.0 and did a "make lib", and then copied the resulting lib/libplasma.a up into the install/lib directory.

In any event, having done this and linking with the new library, I see no change in my code's behaviour. I guess/hope that this confirms that I'm not using hwloc.

Thanks,

Stuart
srdegraaf
 
Posts: 12
Joined: Sat Jul 02, 2011 12:00 am

Re: PLASMA 2.4.0 and OpenMP compatibility issue?

Postby srdegraaf » Thu Jul 07, 2011 9:17 pm

Mathieu,

I just realized that "using" hwloc is a conscious choice, i.e. not something over which I have no control.
So, I did "yum install hwloc hwloc-devel", determined where I needed to add link libraries and include files, rebuilt plasma with your patch, and my code.

!!! IT WORKS NOW !!!

Thank you very much, sir!

Stuart
srdegraaf
 
Posts: 12
Joined: Sat Jul 02, 2011 12:00 am

Re: PLASMA 2.4.0 and OpenMP compatibility issue?

Postby mateo70 » Fri Jul 08, 2011 3:26 am

Glad to hear that :).

So just to add some informations:
- hwloc is a library developed by the OpenMPI team and which provides functionalities to bind threads, information about the hierarchy of the machine, memory placement, ... and so it's better to use it because it will hide the different systems and their different interfaces to PLASMA. The most important point in using hwloc is that it will detect if hyperthreading is activated or not and if yes, will bind only one thread per physical core.
- The way PLASMA detect if you have hwloc or not, and so the way you can detect if you have it or not, is to use the command line:
Code: Select all
pkg-config --libs hwloc

- Concerning the patch, you have to go in the PLASMA root directory and do :
Code: Select all
patch -p 0 < patch_file


As I said, I will try to add the fix for every architecture we support without hwloc today, and I will add the new release as soon as it is done.

However, this is not the best solution for the general case because if you have a loop over the block (PLASMA call, OpenMP section) you don't want to call init and finalize at each iteration. In this case, the solution is to add a fake openmp section before the first call to PLASMA. This way openmp will create non-binded threads and PLASMA will create later its own binded thread without interfering with openmp.

Regards,
Mathieu
mateo70
 
Posts: 98
Joined: Fri May 07, 2010 3:48 pm

Re: PLASMA 2.4.0 and OpenMP compatibility issue?

Postby uhle89 » Fri Jul 08, 2011 10:46 am

Mathieu,
I can also confirm, even iterating OMP/PLASMA/OMP/PLASMA... now works fine for my application,
omp_get_num_procs() always yields correct results.

Thank you for your fast response and efforts, Stephan
uhle89
 
Posts: 6
Joined: Wed Jul 06, 2011 10:30 am

Re: PLASMA 2.4.0 and OpenMP compatibility issue?

Postby mateo70 » Fri Jul 08, 2011 11:06 am

Perfect !!! :D

Just to know, do you call PLASMA_Init and Finalize in your loop, or is it done outside the loop ?

Mathieu
mateo70
 
Posts: 98
Joined: Fri May 07, 2010 3:48 pm

Re: PLASMA 2.4.0 and OpenMP compatibility issue?

Postby uhle89 » Thu Jul 28, 2011 1:09 pm

Dear Mathieu,
I call PLASMA_Init and Finalize within the loop. As as far as I understand this is still
neccessary if I want to change the number of OMP threads to be used dynamically.
However, the potential overhead of calling PLASMA_Init and Finalize wont bother me, since
each step in my loop takes minutes to hours.
regards stephan
uhle89
 
Posts: 6
Joined: Wed Jul 06, 2011 10:30 am

Re: PLASMA 2.4.0 and OpenMP compatibility issue?

Postby nchilton » Mon Jul 02, 2012 2:46 am

Hi,

I'm having the same problem as in this old thread and it seems to be similar to viewtopic.php?f=2&t=61 and viewtopic.php?f=2&t=224.

I have tried installing hwloc and rebuilding plasma, but no luck. I have seen the mention of using plasma_init_affinity(cores,affinity) or using a fake openMP parallel section, but I'm not exactly sure what is meant. Here is how I am using OpenMP/plasma currently:

Code: Select all
call OMP_set_num_threads(2)
!$OMP PARALLEL SECTIONS SHARED(totaldim) COPYIN(LWORK)
   !$OMP SECTION
      ThreadID = OMP_get_thread_num()
      allocate(dummyR(1,1))
      allocate(EigVals(totaldim))
      call PLASMA_INIT(2,plas_info)
      write(6,*) ThreadID,"plasma initialized",plas_info
      call PLASMA_ALLOC_WORKSPACE_DSYEV(totaldim,totaldim,T,plas_info)
      write(6,*) ThreadID,"plasma allocated",plas_info
      call PLASMA_DSYEV(301,121,totaldim,EigVecsR,totaldim,EigVals,T,dummyR,totaldim,plas_info)
      write(6,*) ThreadID,"plasma diaged!",plas_info
      call PLASMA_DEALLOC_HANDLE(T,plas_info)
      write(6,*) ThreadID,"plasma dealloced",plas_info
      call PLASMA_FINALIZE(plas_info)
      write(6,*) ThreadID,"plasma finalized",plas_info
      deallocate(dummyR,EigVals)
   !$OMP SECTION            
      ThreadID = OMP_get_thread_num()
      allocate(dummyR(1,1))
      allocate(EigVals(totaldim))
      call PLASMA_INIT(2,plas_info)
      write(6,*) ThreadID,"plasma initialized",plas_info
      call PLASMA_ALLOC_WORKSPACE_DSYEV(totaldim,totaldim,T,plas_info)
      write(6,*) ThreadID,"plasma allocated",plas_info
      call PLASMA_DSYEV(301,121,totaldim,EigVecsR,totaldim,EigVals,T,dummyR,totaldim,plas_info)
      write(6,*) ThreadID,"plasma diaged!",plas_info
      call PLASMA_DEALLOC_HANDLE(T,plas_info)
      write(6,*) ThreadID,"plasma dealloced",plas_info
      call PLASMA_FINALIZE(plas_info)
      write(6,*) ThreadID,"plasma finalized",plas_info
      deallocate(dummyR,EigVals)
!$OMP END PARALLEL SECTIONS


This code executes with only two cores (not the 2*2 = 4 as expected) and the diagonalizations complete at virtually the same time (always one then the other however).

Any suggestions?

Cheers,
Nick
nchilton
 
Posts: 16
Joined: Wed Apr 04, 2012 9:04 pm

Previous

Return to User discussion

Who is online

Users browsing this forum: No registered users and 3 guests