Multithreading Magma

Open discussion for MAGMA library (Matrix Algebra on GPU and Multicore Architectures)
Post Reply
Hidra2000
Posts: 3
Joined: Tue Mar 18, 2014 3:15 am

Multithreading Magma

Post by Hidra2000 » Wed May 14, 2014 7:03 am

Hello,

I'm trying to make a dedicated GPU calculator that uses multiple threads on several GPUs, each thread with their own stream and attached to a specific GPU.
Since the need arose to use lapack capabilities I turned to MAGMA, unfortunately i couldn't find any recent discussions about thread safety in magma.

So, the question is: How thread safe is Magma 1.4.1?

my initial thought was to initialize Magma for each thread but then i noticed that in magma_init() there are at least 2 global variables, defined in line 27-28 inferfaces.cpp.
Another setback was the fact that the cudaStream is internal, as well as the cublasHandle. I've made my own modification to the functions that interested me but i'm sure not the only one to encounter these issues.

Is there an approved way to use Magma across different threads safely?
If not, are there plans to change the API to support such operations? Maybe change the API so functions need to get magma_context_t or something of that nature?

Thank you
Hidra

mgates3
Posts: 916
Joined: Fri Jan 06, 2012 2:13 pm

Re: Multithreading Magma

Post by mgates3 » Wed May 14, 2014 9:34 am

The globals in magma_init() should be fine. Just call magma_init() once for the whole application; each thread does not need to initialize it.

Other functions are not yet thread safe, unfortunately. Particularly, setting the stream using magmaSetKernelStream is not thread safe.

There is a plan to change the API to pass streams/handles into each function to make them thread safe, similar to the change from CUBLAS v1 to v2.

-mark

mgates3
Posts: 916
Joined: Fri Jan 06, 2012 2:13 pm

Re: Multithreading Magma

Post by mgates3 » Wed May 14, 2014 9:43 am

Also, remember that MAGMA functions are in general hybrid -- part of the computation is done on the CPU, part on the GPU. This means you need to set your BLAS library to use the appropriate number of threads. For instance, set MKL_NUM_THREADS to 1. Otherwise you may have, say, 8 application threads, each spawning 8 MKL threads, making 64 total threads that swamp your system.

In this case, the performance of MAGMA may be significantly reduced, as the GPU waits for the single-threaded CPU to finish its work. You may try adjusting the relevant blocksize in control/get_nb.cpp, making it smaller.

-mark

Hidra2000
Posts: 3
Joined: Tue Mar 18, 2014 3:15 am

Re: Multithreading Magma

Post by Hidra2000 » Mon Jun 23, 2014 2:30 am

another issue i encountered is the various magma_xmalloc_pinned functions which are not thread safe.
I encountered multiple errors until I encapsulated each call with a lock.

If there are plans to change the api to include externally supplied streams i suggest adding a lock as well, or at least have an internal lock to use while allocating pinned memory.

mgates3
Posts: 916
Joined: Fri Jan 06, 2012 2:13 pm

Re: Multithreading Magma

Post by mgates3 » Mon Jun 23, 2014 10:58 am

We'll look into the malloc_pinned routines. Those functions are simply wrappers around cudaMallocHost. MAGMA doesn't do anything additional that would not be thread safe.
-mark

mgates3
Posts: 916
Joined: Fri Jan 06, 2012 2:13 pm

Re: Multithreading Magma

Post by mgates3 » Thu Jul 24, 2014 3:27 pm

Which binary release are you referring to? MAGMA is released only in source code, not binary.

There are OpenMP calls, if compiled with openmp, and pthread calls in MAGMA. Even without threading, MAGMA supports multiple GPUs.

-mark

Noran
Posts: 27
Joined: Fri Sep 26, 2014 3:47 am

Re: Multithreading Magma

Post by Noran » Thu Mar 12, 2015 2:34 am

Since the last answer of this topic some time has gone. So i would like to ask, if the function magmablasSetKernelStream is thread safe now? And what is about thread safety in the LU-Solver called magma_dgesv_gpu?

Greetz and thanks in advance
Noran

mgates3
Posts: 916
Joined: Fri Jan 06, 2012 2:13 pm

Re: Multithreading Magma

Post by mgates3 » Thu Nov 12, 2015 11:46 pm

Sorry that this has been a long time coming. The next release -- hopefully due out by the end of December -- fixes this threading issue. The magmablas GPU kernels all now take a queue (which encapsulates a stream and cublas handle), instead of using a global stream.
-mark

eja
Posts: 1
Joined: Mon Jan 04, 2016 1:23 pm

Re: Multithreading Magma

Post by eja » Mon Jan 04, 2016 1:46 pm

December 2015?

Seriously, this would be a much appreciated improvement; personally I am stuck with the zombie remains of CULA for this reason only.

mgates3
Posts: 916
Joined: Fri Jan 06, 2012 2:13 pm

Re: Multithreading Magma

Post by mgates3 » Wed Jan 06, 2016 8:53 pm

MAGMA 2.0 beta is now available.
http://icl.cs.utk.edu/magma/software/index.html

-mark

Post Reply