HPC Challenge Benchmark Record

System Information
Affiliation:   Oak Ridge National Laboratory   URL:   http://www.nccs.gov
Location:   USA, Tennessee, Oak Ridge   System Use:   Research
System Manufacturer:   Cray   System Name:   XT5
Interconnect Manufacturer:   Cray   Interconnect Type:   Seastar
Operating System:   CLE   MPI:   MPT 3.4.2
MPI Wtick:   0.000001   BLAS:   ACML 4.2.0
Language:   C   Compiler:   PathScale 3.2
Compiler Flags:   -Ofast   Processor Type:   AMD Opteron
Processor Speed:   2.6 GHz   Total Processors:   224256
Processors Entered:   196608   Processors determined:   196608
Cores per chip:   6   HPL Processes:   65536
MPI Processes:   65536   Threads Entered:   3
Threads determined:   1   FLOPs per cycle:   2
Theoretical peak:   2044.7 TFlop/s   Total memory:   299008 GiB
FFT library:   FFTW 3.2alpha3 (Cray modified)  
Explain Optimizations:

Modified HPL broadcast algorithms

Used intercept routines for fftw_create_plan, fftw_one, fftw_mpi_create_plan, fftw_mpi_local_sizes, fftw_mpi_destroy_plan, fftw_mpi to interface with Cray-modified FFTW 3.2alpha3

Modified communication phase of Sandia's AnyNodesMPIRandomAccessUpdate to coordinate updates between ranks on-node, using MPI.  No changes were made to lookahead, nor to the generation or application of updates.


HPL
HPL:   1338.67 Tflop/s   HPL time:   37341.8
HPL eps:   1.11022e-16   HPL Rnorm1:   0.000000923378
HPL Anorm1:   1055700   HPL AnormI:   1055700
HPL Xnorm1:   16312700   HPL XnormI:   25.8377
HPL N:   4216832   HPL NB:   232
HPL NProw:   256   HPL NPcol:   256
HPL depth:   1   HPL NBdiv:   2
HPL NBmin:   4   HPL CPfact:   C
HPL CRfact:   R   HPL CPtop:   1
HPL order:   R
HPL dMach EPS:   1.110223e-16   HPL sMach EPS:   0.00000005960464
HPL dMach sfMin:   2.225074e-308   HPL sMach sfMin:   1.1754939999999999e-38
HPL dMach Base:   2   HPL sMach Base:   2
HPL dMach Prec:   2.220446e-16   HPL sMach Prec:   0.0000001192093
HPL dMach mLen:   53   HPL sMach mLen:   24
HPL dMach Rnd:   1   HPL sMach Rnd:   1
HPL dMach eMin:   -1021   HPL sMach eMin:   -125
HPL dMach rMin:   2.225074e-308   HPL sMach rMin:   1.1754939999999999e-38
HPL dMach eMax:   1024   HPL sMach eMax:   128
HPL dMach rMax:   1.797693e308   HPL sMach rMax:   3.4028230000000003e38
dweps:   1.110223e-16   sweps:   0.00000005960464

PTRANS
PTRANS:   1889.16 GB/s   PTRANS time:   17.9184 seconds
PTRANS residual:   0   PTRANS N:   2108416
PTRANS NB:   232   PTRANS NProw:   256
PTRANS NPcol:   256

STREAM
S-STREAM Copy:   5.76254 GB/s   S-STREAM Scale:   5.75452 GB/s
S-STREAM Add:   6.35418 GB/s   S-STREAM Triad:   6.35478 GB/s
EP-STREAM Copy:   3.30028 GB/s   EP-STREAM Scale:   3.30789 GB/s
EP-STREAM Add:   3.69244 GB/s   EP-STREAM Triad:   3.71282 GB/s
STREAM Vector Size:   90442261   STREAM Threads:   1

RandomAccess
S-RandomAccess:   0.0143238 Gup/s   EP-RandomAccess:   0.011237 Gup/s
G-RandomAccess:   36.4275 Gup/s   G-RandomAccess N:   17592186044416
G-RandomAccess time:   1931.75 seconds   G-RandomAccess Check Time:   1404.11 seconds
G-RandomAccess Errors:   420174   G-RandomAccess Errors Fraction:   0.0000000238841
G-RandomAccess TimeBound:   -1   G-RandomAccess ExeUpdates:   70368744177664
RandomAccess N:   268435456

FFT
S-FFT:   1.21098 GFlop/s   EP-FFT:   1.24305 GFlop/s
MPIFFT:   10698.5 GFlop/s   MPIFFT N:   2199023255552
MPIFFT Max Error:   0.00000000000000238835   MPIFFT time0:   0 seconds
MPIFFT time1:   0 seconds   MPIFFT time2:   0 seconds
MPIFFT time3:   0 seconds   MPIFFT time4:   0 seconds
MPIFFT time5:   0 seconds   MPIFFT time6:   0 seconds
FFTEnblk:   16   FFTEnp:   4
FFTEl2size:   1048576

DGEMM
S-DGEMM:   28.8142 GFlop/s   EP-DGEMM:   28.3858 GFlop/s
DGEMM N:   9509

RandomRing Latency/Bandwidth
RandomRing Latency:   15.9868 usec   RandomRing Bandwidth:   0.0403873 GB/s

NaturalRing Latency/Bandwidth
NaturalRing Latency:   16.0933 usec   NaturalRing Bandwidth:   0.040266 GB/s

PingPong Latency/Bandwidth
Maximum PingPong Latency:   9.0003 usec   Maximum PingPong Bandwidth:   1.66805 GB/s
Minimum PingPong Latency:   0.586112 usec   Minimum PingPong Bandwidth:   1.52562 GB/s
Average PingPong Latency:   7.44933 usec   Average PingPong Bandwidth:   1.60932 GB/s

Size of Data Types
char:   1 byte     short:   2 bytes
int:   4 bytes   long:   8 bytes
void ptr:   8 bytes   float:   4 bytes
double:   8 bytes   size t:   8 bytes
s64Int:   8 bytes   u64Int:   8 bytes

OpenMP
M OpenMP:   -1   OpenMP Num Threads:   0
OpenMP Num Procs:   0   OpenMP Max Threads:   0

Memory
MemProc:   -1   MemSpec:   -1
MemVal:   -1

CPS
CPS_HPCC_FFT_235:   0   CPS_HPCC_FFTW_ESTIMATE:   0
CPS_HPCC_MEMALLCTR:   0   CPS_HPL_USE_GETPROCESSTIMES:   0
CPS_RA_SANDIA_NOPT:   0   CPS_RA_SANDIA_OPT2:   1


Version: 1.3.1.f - Run Type: opt - Parent ID: 380
Created: 2009-11-10 - Exported: Thu Jul 31 09:32:51 2014
HPC Challenge Benchmark Record