MAGMA use for steady CFD simualtions

Open discussion for MAGMA library (Matrix Algebra on GPU and Multicore Architectures)

MAGMA use for steady CFD simualtions

Postby dreamer » Mon Feb 13, 2017 6:23 am

I am testing MAGMA sparse to solve linear systems arising from NS equations in a CFD steady simulation. Existing code uses Paralution ( cusparse ILU) and works fine. I am interested on having a faster ILU than what Paralution provides. I adapted the example_sparse.c to read an mtx file for the matrix and a txt file for the rhs. It works fine and I got performance results for ILU with cusparse and ILU with ISAI.

I have though observed that the repeated use of the ILU and the solver increases the memory consumption (attached file: memoryCheck.c).
I an other test I solve the same system twice but this time every run is between a magma_init() and magma_finalize() (attached file memoryCheck2.c).
My Matrix has 18400 nnz and the memory consumption status with nvidia-smi after each step is like that:
    ILU1
    | 0 27593 C ./memoryCheck2 138MiB |
    FGMRES1
    | 0 27593 C ./memoryCheck2 138MiB |
    After magma_finalize()
    | 0 27593 C ./memoryCheck2 132MiB |

    ILU2
    | 0 27593 C ./memoryCheck2 148MiB |
    FGMRES2
    | 0 27593 C ./memoryCheck2 148MiB |
    After magma_finalize()
    | 0 27593 C ./memoryCheck2 142MiB |

Am I forgetting to free some of the used structure between the first and the second use?
Attachments
memoryCheck2.c
A memory check for repeated use ILU and fgmres separated by magma_finalize (need mtx and rhs file, I could provide my files)
(4.65 KiB) Downloaded 32 times
memoryCheck.c
A memory check for repeated use of ILu and fgmres (need mtx and rhs file, I could provide my files)
(2.96 KiB) Downloaded 30 times
dreamer
 
Posts: 2
Joined: Sun Feb 12, 2017 10:25 am

Re: MAGMA use for steady CFD simualtions

Postby hartwig anzt » Mon Feb 13, 2017 10:41 am

Dear "dreamer",

thank you for pointing out this issue. I appreciate attaching code that I can follow what is happening.

The increased memory usage comes from the fact that the preconditioner is not deallocated after being applied. This means, the ILU factorization stays in the GPU memory, and when the preconditioner setup is called the second time, it does not update the existing preconditioner, but allocates a new one.

to avoid this, just call

magma_dprecondfree( &opts.precond_par,queue);

to free the preconditioner, or

magma_dsolverinfo_free( &opts.solver_par, &opts.precond_par,queue);

to free all information in the preconditioner and the solver. I admit that this should happen automatically in case the same preconditioner gets generated, and will fix it in the next release.

I don't have all details about your NS solver, but I could imagine this being a very nice setting for not completely regenerating a preconditioner, but updating an existing factorization via
magma_dpariluupdate( A, &precond, sweeps, queue );
see : http://www.netlib.org/utk/people/JackDo ... LU_GPU.pdf

If you are interested to look into this, please just send me an email.

Also, I see you are using GMRES solver, maybe try IDR with restart 4: My experience is that IDR is often much faster...

Hartwig
hartwig anzt
 
Posts: 79
Joined: Tue Sep 02, 2014 5:44 pm

Re: MAGMA use for steady CFD simualtions

Postby dreamer » Mon Feb 13, 2017 12:23 pm

Dear Dr Anzt,
Thanks for your prompt answer. Yes freeing the solver solved the problem. I assumed that magma_finalize() would have done this.
For heavy matrix of few millions of nnz I see that the cost of preparing the preconditioner is prohibitive so for sure I will need the dpariluupdate.

Med
dreamer
 
Posts: 2
Joined: Sun Feb 12, 2017 10:25 am


Return to User discussion

Who is online

Users browsing this forum: No registered users and 4 guests

cron