questions/problems with magma_ssyevd and magma_ssyevd_gpu

Open discussion for MAGMA library (Matrix Algebra on GPU and Multicore Architectures)
Post Reply
ronmarc97
Posts: 9
Joined: Wed May 15, 2013 5:35 pm

questions/problems with magma_ssyevd and magma_ssyevd_gpu

Post by ronmarc97 » Fri Jul 18, 2014 4:54 pm

Hellow people.

I have some doubts about ssyevd routine in magma-1.5.0-beta2

*First of all, i am using a Fermi with 4 devices: Tesla C2070, 1147.0 MHz clock, 5375.4 MB memory, capability 2.0, each one.
In my program i create a matrix of N =1400, and everithing is ok (eigenvalues and eigenvectors).

Code: Select all

n2  = N*N;
lda = N;
// query for workspace sizes
magma_ssyevd( opts.jobz, opts.uplo,
                          N, NULL, lda, NULL,
                          aux_work,  -1,
                          aux_iwork, -1,
                          &info );
            lwork  = (magma_int_t) aux_work[0];
            liwork = aux_iwork[0];
            
            /* Allocate host memory for the matrix */
            TESTING_MALLOC_CPU( h_A,    float, N*lda );
            TESTING_MALLOC_CPU( w1,     float, N     );
            TESTING_MALLOC_CPU( w2,     float, N     );
            TESTING_MALLOC_CPU( iwork,  magma_int_t, liwork );
            
            TESTING_MALLOC_PIN( h_R,    float, N*lda  );
            TESTING_MALLOC_PIN( h_work, float, lwork  );

    magma_ssyevd( MagmaVec, MagmaUpper,
                  n, Matrix, lda, w1,
                  h_work, lwork+1,
                  iwork, liwork,
                  &info );

    if (info != 0)
       printf("magma_ssyevd returned error %d: %s.\n",
                      (int) info, magma_strerror( info ));
Also i print the lwork parameter for querying for workspace sizes ( LWORK >= max( 2*N + N*NB, 1 + 6*N + 2*N**2 )), and lwork = 3928401 (1 + 6*N + 2*N**2), Tha's OK.

* Now when i create a matrix of N =14000, show that error: "On entry to magma_ssyevd parameter number 8 had an illegal value"....that is the lwork paremeter.
When i print that parameter lwork = 392084000, and the correct value should be 392084001....why that error ???... because i must force the lwork adding +1 to get no error and get all eigenvalues and eigenvectors.
But i think this is not correct, the function magma_ssyevd not calculate the correct lwork ??? or i'm doing something wrong ??
My Ram memory of my CPU is 6 GB, i think is enough to calculate the eigenvalues and eigenvector of a matrix of 14000x14000.

* Another doubt is about magma_ssyevd_gppu: in my case each device have 2GB of RAM Memory,
The problem is the same case of magma_ssyevd for a matrix of 14000, i suposse for the size memory RAM , is not enough !!

Code: Select all

n = 14000
lda  = n;
ldda = ((n + 31)/32)*32; 
magma_ssyevd_gpu( MagmaVec, MagmaUpper,
                          n, NULL ,ldda, NULL,
                          NULL,lda,
                          aux_work,  -1,
                          aux_iwork, -1,
                          &info );
    
        lwork  = (magma_int_t) aux_work[0];
        liwork = aux_iwork[0];
        
        /* Allocate host memory for the matrix */
        TESTING_MALLOC_CPU( h_A,    float,      n*lda  );
        TESTING_MALLOC_CPU( w1,     float,      n      );
        TESTING_MALLOC_CPU( w2,     float,      n      );
        TESTING_MALLOC_CPU( iwork,  magma_int_t, liwork );
            
        TESTING_MALLOC_PIN( h_R,    float,      n*lda  );
        TESTING_MALLOC_PIN( h_work, float,      lwork  );
            
        TESTING_MALLOC_DEV( d_R,    float,      n*ldda );

     magma_ssetmatrix( n, n, nMatrix, n, d_R, ldda ); //mat->v  Transfering data to the GPU, copy test(CPU) -> d_R (GPU)
     magma_ssyevd_gpu( MagmaVec ,MagmaUpper ,
                          n, d_R, ldda, w1,
                          h_R, n,
                          h_work, lwork,
                          iwork, liwork,
                          &info );
     magma_sgetmatrix( n ,n, d_R, ldda, h_R, lda );
My doubt is when i allocate : TESTING_MALLOC_DEV( d_R, float, n*ldda );, its allocate in all the devices (4 in total) or only 1
And whe i use magma_ssyevd_gpu, that function ssyevd is distribuided in all devices.


Thanks in advances for answering my doubts.
Best regards for all you.

mgates3
Posts: 897
Joined: Fri Jan 06, 2012 2:13 pm

Re: questions/problems with magma_ssyevd and magma_ssyevd_gp

Post by mgates3 » Fri Jul 18, 2014 6:00 pm

I think this lwork query has been fixed in 1.5.0 beta 3 (just released). It happened because the correct lwork cannot be exactly represented in float, and gets rounded down. Now we round it up.

TESTING_MALLOC_DEV is a simple macro in testings.h around magma_malloc (cudaMalloc). It allocates memory on a single GPU. magma_ssyevd_gppu is a single-GPU code. Please see zheevdx_m, zheevdx_mgpu, or magma_ssyevdx_2stage_m for multi-GPU versions.

-mark

ronmarc97
Posts: 9
Joined: Wed May 15, 2013 5:35 pm

Re: questions/problems with magma_ssyevd and magma_ssyevd_gp

Post by ronmarc97 » Mon Jul 21, 2014 9:54 am

Thankss a lot mark.
I'm goingo to install MAGMA 1.5.0 beta 3, and chek the routines for multi-GPU.

ronmarc97
Posts: 9
Joined: Wed May 15, 2013 5:35 pm

Re: questions/problems with magma_ssyevd and magma_ssyevd_gp

Post by ronmarc97 » Tue Jul 29, 2014 11:17 am

Hellow

I have anohter question

I have a problem when the matrix[28000 x 28000] is allocated in magma_dsyevdx_2stage_m (for 4 GPU with 2GB of RAM Memory each one), show that error:
!!! magma_malloc_pinned failed for: h_work
The same problem happened when i use magma_ssyevd_gpu (for 1 GPU).

But when i use magma_ssyevd, there is no problem, it calcule all eigenvector and eigenvalues. Why??

* It's cause for the size of tre matrix, it can be allocated on the CPU ( 6GB of RAM).

*In case of multi_gpu, the matrix is not allocated in CPU ?? Is allocated in GPU(2GB of RAM) ? if it's true, then its not possible allocate that matrix size.
In multi_gpu, the matirx is not distribuided in all 4 device of GPU (using each memory of 2GB of RAM, total of 8GB RAM) ? it's that possible ? or i'ts allocated in a single GPU, and processed using 4 device of GPU ??

Thanks for answering my questions.
I apreciatted.

Post Reply