HPC Challenge Benchmark Record

System Information
Affiliation:   National Institute for Computational Sciences   URL:   http://www.nics.tennessee.edu/
Location:   USA, Tennessee, Oak Ridge   System Use:   Academic
System Manufacturer:   Cray, Inc.   System Name:   XT5
Interconnect Manufacturer:   Cray, Inc.   Interconnect Type:   SeaStar 2+
Operating System:   CLE 2.2   MPI:   MPT 3.4.2
MPI Wtick:   0.000001   BLAS:   ACML 4.3.0
Language:   C   Compiler:   GCC 4.4.1
Compiler Flags:   -O3 -fomit-frame-pointer -funroll-all-loops -ffast-math -fpeel-loops -ftree-vectorize -mtune=amdfam10 -fprefetch-loop-arrays   Processor Type:   AMD Opteron
Processor Speed:   2.6 GHz   Total Processors:   99072
Processors Entered:   98304   Processors determined:   98304
Cores per chip:   6   HPL Processes:   32768
MPI Processes:   32768   Threads Entered:   3
Threads determined:   3   FLOPs per cycle:   2
Theoretical peak:   1022 TFlop/s   Total memory:   132096 GiB
FFT library:   FFTW 3.2alpha3 (Cray modified)  
Explain Optimizations:
Modified HPL broadcast algorithms

Used intercept routines for fftw_create_plan, fftw_one, fftw_mpi_create_plan, fftw_mpi_local_sizes, fftw_mpi_destroy_plan, fftw_mpi to interface with Cray-modified FFTW 3.2alpha3 (enhancements to FFTW's Alltoall algorithms)

HPL
HPL:   657.625 Tflop/s   HPL time:   26460.6
HPL eps:   1.11022e-16   HPL Rnorm1:   0.00000194382
HPL Anorm1:   742848   HPL AnormI:   742778
HPL Xnorm1:   5473430   HPL XnormI:   12.182
HPL N:   2966352   HPL NB:   232
HPL NProw:   128   HPL NPcol:   256
HPL depth:   1   HPL NBdiv:   2
HPL NBmin:   4   HPL CPfact:   C
HPL CRfact:   R   HPL CPtop:   1
HPL order:   R
HPL dMach EPS:   1.110223e-16   HPL sMach EPS:   0.00000005960464
HPL dMach sfMin:   2.225074e-308   HPL sMach sfMin:   1.1754939999999999e-38
HPL dMach Base:   2   HPL sMach Base:   2
HPL dMach Prec:   2.220446e-16   HPL sMach Prec:   0.0000001192093
HPL dMach mLen:   53   HPL sMach mLen:   24
HPL dMach Rnd:   1   HPL sMach Rnd:   1
HPL dMach eMin:   -1021   HPL sMach eMin:   -125
HPL dMach rMin:   2.225074e-308   HPL sMach rMin:   1.1754939999999999e-38
HPL dMach eMax:   1024   HPL sMach eMax:   128
HPL dMach rMax:   1.797693e308   HPL sMach rMax:   3.4028230000000003e38
dweps:   1.110223e-16   sweps:   0.00000005960464

PTRANS
PTRANS:   1559.64 GB/s   PTRANS time:   11.2837 seconds
PTRANS residual:   0   PTRANS N:   1483176
PTRANS NB:   232   PTRANS NProw:   128
PTRANS NPcol:   256

STREAM
S-STREAM Copy:   7.10761 GB/s   S-STREAM Scale:   7.10645 GB/s
S-STREAM Add:   7.97324 GB/s   S-STREAM Triad:   7.99198 GB/s
EP-STREAM Copy:   3.38065 GB/s   EP-STREAM Scale:   3.38335 GB/s
EP-STREAM Add:   3.82406 GB/s   EP-STREAM Triad:   3.88189 GB/s
STREAM Vector Size:   89510540   STREAM Threads:   3

RandomAccess
S-RandomAccess:   0.0114601 Gup/s   EP-RandomAccess:   0.00887692 Gup/s
G-RandomAccess:   18.4965 Gup/s   G-RandomAccess N:   8796093022208
G-RandomAccess time:   1902.21 seconds   G-RandomAccess Check Time:   767.793 seconds
G-RandomAccess Errors:   503693   G-RandomAccess Errors Fraction:   0.0000000572633
G-RandomAccess TimeBound:   -1   G-RandomAccess ExeUpdates:   35184372088832
RandomAccess N:   268435456

FFT
S-FFT:   1.37045 GFlop/s   EP-FFT:   1.21987 GFlop/s
MPIFFT:   7529.5 GFlop/s   MPIFFT N:   1099511627776
MPIFFT Max Error:   0.00000000000000237634   MPIFFT time0:   0 seconds
MPIFFT time1:   0 seconds   MPIFFT time2:   0 seconds
MPIFFT time3:   0 seconds   MPIFFT time4:   0 seconds
MPIFFT time5:   0 seconds   MPIFFT time6:   0 seconds
FFTEnblk:   16   FFTEnp:   4
FFTEl2size:   1048576

DGEMM
S-DGEMM:   29.1413 GFlop/s   EP-DGEMM:   28.8195 GFlop/s
DGEMM N:   9460

RandomRing Latency/Bandwidth
RandomRing Latency:   15.4504 usec   RandomRing Bandwidth:   0.0558521 GB/s

NaturalRing Latency/Bandwidth
NaturalRing Latency:   15.4018 usec   NaturalRing Bandwidth:   0.055343 GB/s

PingPong Latency/Bandwidth
Maximum PingPong Latency:   8.50856 usec   Maximum PingPong Bandwidth:   1.66325 GB/s
Minimum PingPong Latency:   0.466903 usec   Minimum PingPong Bandwidth:   1.54089 GB/s
Average PingPong Latency:   7.24844 usec   Average PingPong Bandwidth:   1.60699 GB/s

Size of Data Types
char:   1 byte     short:   2 bytes
int:   4 bytes   long:   8 bytes
void ptr:   8 bytes   float:   4 bytes
double:   8 bytes   size t:   8 bytes
s64Int:   8 bytes   u64Int:   8 bytes

OpenMP
M OpenMP:   200805   OpenMP Num Threads:   3
OpenMP Num Procs:   12   OpenMP Max Threads:   3

Memory
MemProc:   -1   MemSpec:   -1
MemVal:   -1

CPS
CPS_HPCC_FFT_235:   0   CPS_HPCC_FFTW_ESTIMATE:   0
CPS_HPCC_MEMALLCTR:   0   CPS_HPL_USE_GETPROCESSTIMES:   0
CPS_RA_SANDIA_NOPT:   0   CPS_RA_SANDIA_OPT2:   1


Version: 1.3.1.f - Run Type: opt - Parent ID: 376
Created: 2009-11-02 - Exported: Sat Aug 2 04:46:49 2014
HPC Challenge Benchmark Record