Spec2K6

From Msim

Jump to: navigation, search

Spec2K6 is a standardized benchmark testing suite. Here we list all of them and any issues related to their program output.
Alpha binaries for many of the executables (and .arg files for all of them) are available here: SPEC2K6 Benchmarks

Contents

Astar

astarNS.1.arg: 0 # ./astar/astar ./astar/BigLakes2048.cfg > ./astar/BigLakes2048.out 2>> ./astar/BigLakes2048.err
Insts: 479417958886
astarNS.2.arg: 0 # ./astar/astar ./astar/rivers.cfg > ./astar/rivers.out 2>> ./astar/rivers.err
Insts: 953526885648

Bwaves

Compiled with: f90 -O4 -non_shared -DSPEC_CPU -fast *.f -o bwaves
bwavesNS.1.arg: 0 # ./bwaves/bwaves 2>> ./bwaves/bwaves.err

Bzip2

bzip2NS.1.arg: 0 # ./bzip2/bzip2 ./bzip2/input.source 280 > ./bzip2/input.source.out 2>> ./bzip2/input.source.err
Preliminary Test: Passed (656126761683 insts)
bzip2NS.2.arg: 0 # ./bzip2/bzip2 ./bzip2/chicken.jpg 30 > ./bzip2/chicken.jpg.out 2>> ./bzip2/chicken.jpg.err
Preliminary Test: Passed (247171978682 insts)
bzip2NS.3.arg: 0 # ./bzip2/bzip2 ./bzip2/liberty.jpg 30 > ./bzip2/liberty.jpg.out 2>> ./bzip2/liberty.jpg.err
Preliminary Test: Passed (454072233537 insts)
bzip2NS.4.arg: 0 # ./bzip2/bzip2 ./bzip2/input.program 280 > ./bzip2/input.program.out 2>> ./bzip2/input.program.err
Preliminary Test: Passed (791135364867 insts)
bzip2NS.5.arg: 0 # ./bzip2/bzip2 ./bzip2/text.html 280 > ./bzip2/text.html.out 2>> ./bzip2/text.html.err
Preliminary Test: Passed (1151050151694 insts)
bzip2NS.6.arg: 0 # ./bzip2/bzip2 ./bzip2/input.combined 200 > ./bzip2/input.combined.out 2>> ./bzip2/input.combined.err
Preliminary Test: Passed (517508251052 insts)

CactusADM

cactusADMNS.1.arg: 0 # ./cactusADM/cactusADM ./cactusADM/benchADM.par > ./cactusADM/benchADM.out 2>> ./cactusADM/benchADM.err
Insts: 18071243403325

Calculix

calculixNS.1.arg: 0 # ./calculix/calculix -i ./calculix/hyperviscoplastic > ./calculix/hyperviscoplastic.log 2>> ./calculix/hyperviscoplastic.err
Insts: 4326855515310

25946c25946
<   612   10 -5.5885E+03 -5.5382E+03 -5.5843E+03 -9.2052E-01  4.1954E+00  1.3958E+01
---
>   612   10 -5.5885E+03 -5.5382E+03 -5.5843E+03 -9.2051E-01  4.1954E+00  1.3958E+01
30400c30400
<   777    9 -3.3995E+00 -3.3650E+00 -3.4023E+00 -2.0614E-03  7.4870E-09 -8.4383E-03
---
>   777    9 -3.3995E+00 -3.3650E+00 -3.4023E+00 -2.0614E-03  7.4856E-09 -8.4383E-03
56305c56305
<   812   18  1.1240E-12
---
>   812   18  1.1241E-12
58279c58279
<   885   21  6.0357E-09
---
>   885   21  6.0358E-09
81797c81797
<   483    6  3.9011E+00  2.4508E+01 -5.8213E+00  1.1244E+01 -7.5171E+00 -5.6231E-01
---
>   483    6  3.9011E+00  2.4508E+01 -5.8214E+00  1.1244E+01 -7.5171E+00 -5.6231E-01
133671c133671
<   206   21 -4.7964E+02  4.1738E+02  9.2982E+01 -1.6763E+02 -1.6800E+02 -1.6575E+01
---
>   206   21 -4.7964E+02  4.1738E+02  9.2983E+01 -1.6763E+02 -1.6800E+02 -1.6575E+01
136505c136505
<   311   20  6.1358E-04  4.5584E+01  7.2399E+01 -1.5118E+02  8.7683E+01 -7.5714E+01
---
>   311   20  6.1354E-04  4.5584E+01  7.2399E+01 -1.5118E+02  8.7683E+01 -7.5714E+01
152493c152493
<   903   24 -1.7722E+03 -1.8733E+03 -1.9326E+03  3.8374E+00 -3.9815E+01  3.5267E+01
 ---
>   903   24 -1.7722E+03 -1.8733E+03 -1.9326E+03  3.8374E+00 -3.9814E+01  3.5267E+01
152819c152819
<   915   26 -1.4263E+02 -3.2478E+00 -2.6899E+01 -3.7039E+01  1.3414E+01  1.7208E+01
---
>   915   26 -1.4263E+02 -3.2477E+00 -2.6899E+01 -3.7039E+01  1.3414E+01  1.7208E+01


DealII

dealIINS.1.arg: 0 # ./dealII/dealII.peak.ev6 23 > ./dealII/dealII.log 2>> ./dealII/dealII.err
Insts: 2448509199652
dealII has some really weird behavior. Many of the output files match, many don't. The x86 results don't match the reference results. The simulator results look like they match the x86 results better but precision errors appear to be significant.


Gamess

Compiled as (-O4 claimed memory issues): f90 -DSPEC_CPU -O3 -i8 -non_shared *.o -o gamess
gamessNS.1.arg: 0 # ./gamess/gamess < ./gamess/cytosine.2.config > ./gamess/cytosine.2.out 2>> ./gamess/cytosine.2.err
gamessNS.2.arg: 0 # ./gamess/gamess < ./gamess/h2ocu2+.gradient.config > ./gamess/h2ocu2+.gradient.out 2>> ./gamess/h2ocu2+.gradient.err
gamessNS.3.arg: 0 # ./gamess/gamess < ./gamess/triazolium.config > ./gamess/triazolium.out 2>> ./gamess/triazolium.err

Gcc

Compiling as: cc -non_shared -DSPEC_CPU -O4 *.c -o gcc -DNDEBUG -DSPEC_CPU_LP64 -lm

gccNS.1.arg: 0 # ./gcc/gcc ./gcc/166.i -o ./gcc/166.s > ./gcc/166.out 2>> ./gcc/166.err
Insts: 144963741833
gccNS.2.arg: 0 # ./gcc/gcc ./gcc/200.i -o ./gcc/200.s > ./gcc/200.out 2>> ./gcc/200.err
Insts: 261480319773
gccNS.3.arg: 0 # ./gcc/gcc ./gcc/c-typeck.i -o ./gcc/c-typeck.s > ./gcc/c-typeck.out 2>> ./gcc/c-typeck.err
Insts: 283377731557
gccNS.4.arg: 0 # ./gcc/gcc ./gcc/cp-decl.i -o ./gcc/cp-decl.s > ./gcc/cp-decl.out 2>> ./gcc/cp-decl.err
Insts: 193035943330
gccNS.5.arg: 0 # ./gcc/gcc ./gcc/expr.i -o ./gcc/expr.s > ./gcc/expr.out 2>> ./gcc/expr.err
Insts: 225497004132
gccNS.6.arg: 0 # ./gcc/gcc ./gcc/expr2.i -o ./gcc/expr2.s > ./gcc/expr2.out 2>> ./gcc/expr2.err
Insts: 309592634996
gccNS.7.arg: 0 # ./gcc/gcc ./gcc/g23.i -o ./gcc/g23.s > ./gcc/g23.out 2>> ./gcc/g23.err
Insts: 374417030098
gccNS.8.arg: 0 # ./gcc/gcc ./gcc/s04.i -o ./gcc/s04.s > ./gcc/s04.out 2>> ./gcc/s04.err
Insts: 347682725406
gccNS.9.arg: 0 # ./gcc/gcc ./gcc/scilab.i -o ./gcc/scilab.s > ./gcc/scilab.out 2>> ./gcc/scilab.err
Insts: 97315785943

GemsFDTD

GemsFDTDNS.1.arg: 0 # ./GemsFDTD/GemsFDTD > ./GemsFDTD/ref.log 2>> ./GemsFDTD/ref.err

Gobmk

gobmkNS.1.arg: 0 # ./gobmk/gobmk --quiet --mode gtp < ./gobmk/13x13.tst > ./gobmk/13x13.out 2>> ./gobmk/13x13.err
Insts: 375426999910
gobmkNS.2.arg: 0 # ./gobmk/gobmk --quiet --mode gtp < ./gobmk/nngs.tst > ./gobmk/nngs.out 2>> ./gobmk/nngs.err
Insts: 992469737687
gobmkNS.3.arg: 0 # ./gobmk/gobmk --quiet --mode gtp < ./gobmk/score2.tst > ./gobmk/score2.out 2>> ./gobmk/score2.err
Insts: 532130093711
gobmkNS.4.arg: 0 # ./gobmk/gobmk --quiet --mode gtp < ./gobmk/trevorc.tst > ./gobmk/trevorc.out 2>> ./gobmk/trevorc.err
Insts: 375182042979
gobmkNS.5.arg: 0 # ./gobmk/gobmk --quiet --mode gtp < ./gobmk/trevord.tst > ./gobmk/trevord.out 2>> ./gobmk/trevord.err
Insts: 531856246663

Gromacs

gromacsNS.1.arg: 0 # ./gromacs/gromacs -silent -deffnm ./gromacs/gromacs -nice 0 2>> ./gromacs/gromacs.errQ

H264ref

h264refNS.1.arg: 0 # ./h264ref/h264ref -d ./h264ref/foreman_ref_encoder_baseline.cfg > ./h264ref/foreman_ref_baseline_encodelog.out 2>> ./h264ref/foreman_ref_baseline_encodelog.err
Insts: 723410410661

137,138c137
<  Leaky BucketRateFile does not have valid entries;
<  using rate calculated from avg. rate 
---
>  LeakyBucketRate File does not exist; using rate calculated from avg. rate 

h264refNS.2.arg: 0 # ./h264ref/h264ref -d ./h264ref/foreman_ref_encoder_main.cfg > ./h264ref/foreman_ref_main_encodelog.out 2>> ./h264ref/foreman_ref_main_encodelog.err
Insts: 654470462092

140,141c140
<  Leaky BucketRateFile does not have valid entries;
<  using rate calculated from avg. rate 
---
>  LeakyBucketRate File does not exist; using rate calculated from avg. rate 

h264refNS.3.arg: 0 # ./h264ref/h264ref -d ./h264ref/sss_encoder_main.cfg > ./h264ref/sss_main_encodelog.out 2>> ./h264ref/sss_main_encodelog.err
Insts: 5955427373236

2c2
< Parsing Configfile sss_encoder_main.cfg.....................................................................................................
---
> Parsing Configfile ./h264ref/sss_encoder_main.cfg.....................................................................................................
188,189c188
<  Leaky BucketRateFile does not have valid entries;
<  using rate calculated from avg. rate
---
>  LeakyBucketRate File does not exist; using rate calculated from avg. rate

Hmmer

Minors errors in case 2 only. hmmerNS.1.arg: 0 # ./hmmer/hmmer ./hmmer/nph3.hmm ./hmmer/swiss41 > ./hmmer/nph3.out 2>> ./hmmer/nph3.err
Insts: 1153321346094
hmmerNS.2.arg: 0 # ./hmmer/hmmer --fixed 0 --mean 500 --num 500000 --sd 350 --seed 0 ./hmmer/retro.hmm > ./hmmer/retro.out 2>> ./hmmer/retro.err
Insts: 2447074118685

15,16c15,16
< mu     :   -48.936745
< lambda :     0.235556
---
> mu     :   -48.936687
> lambda :     0.235582

Lbm

Compiling as: cc -non_shared -DSPEC_CPU -O4 *.c -o lbm -lm lbmNS.1.arg: 0 # ./lbm/lbm 3000 ./lbm/reference.dat 0 0 ./lbm/100_100_130_ldc.of > ./lbm/lbm.out 2>> ./lbm/lbm.err
Preliminary Test: Passed (1326844953820 insts)

Leslie3d

leslie3dNS.1.arg: 0 # ./leslie3d/leslie3d < ./leslie3d/leslie3d.in > ./leslie3d/leslie3d.stdout 2>> ./leslie3d/leslie3d.err
Preliminary Test: Passed (1605335976415 insts)

Libquantum

Compiled with: cc -I/usr/include/cxx -non_shared -DSPEC_CPU -DSPEC_CPU_HPUX -O4 *.c -o libquantum -lm
libquantumNS.1.arg: 0 # ./libquantum/libquantum 1397 8 > ./libquantum/ref.out 2>> ./libquantum/ref.err

Mcf

mcfNS.1.arg: 0 # ./mcf/mcf inp.in > ./mcf/inp.out 2>> ./mcf/inp.err
Preliminary Test: Passed (421217771893 insts)

Milc

Minor difference. milcNS.1.arg: 0 # ./milc/milc < ./milc/su3imp.in > ./milc/su3imp.out 2>> ./milc/su3imp.err
Insts: 1102242006115

108c108
< Unitarity checked.  Max deviation 6.66e-16
---
> Unitarity checked.  Max deviation 4.44e-16

Namd

namdNS.1.arg: 0 # ./namd/namd --input ./namd/namd.input --iterations 38 --output ./namd/namd.out > ./namd/namd.stdout 2>> ./namd/namd.err
Insts: 2369133747901
A lot of differences, the majority (if not all) look like 0 vs -0.

Omnetpp

omnetppNS.1.arg: 0 # ./omnetpp/omnetpp ./omnetpp/omnetpp.ini > ./omnetpp/omnetpp.log 2>> ./omnetpp/omnetpp.err
Insts: 830819114560
omnetpp.sca is correct.
omnetpp.log:

13780c13780
<      Messages:  created: 124887970
---
>      Messages:  created: 124887978
13782c13782
<      Messages:  created: 128781731
---
>      Messages:  created: 128781733
13790c13790
<      Messages:  created: 144400982
---
>      Messages:  created: 144400984
13797c13797
< ** Event #205000000   T=2.9610707  ( 2.96s)
---
> ** Event #205000000   T=2.9610709  ( 2.96s)
13810c13810
<      Messages:  created: 183435417
---
>      Messages:  created: 183435419
13812c13812
<      Messages:  created: 187342901
---
>      Messages:  created: 187342903
13815c13815
< ** Event #250000000   T=3.5761254  ( 3.57s)
---
> ** Event #250000000   T=3.5761255  ( 3.57s)
13818c13818
<      Messages:  created: 199064760
---
>      Messages:  created: 199064762
13820c13820
<      Messages:  created: 202962178
---
>      Messages:  created: 202962180
13822c13822
<      Messages:  created: 206859752
---
>      Messages:  created: 206859754

Perlbench

perlbenchNS.1.arg: 0 # ./perlbench/perlbench -I./perlbench/lib ./perlbench/checkspam.pl 2500 5 25 11 150 1 1 1 1 > ./perlbench/checkspam.2500.5.25.11.150.1.1.1.1.out 2>> ./perlbench/checkspam.2500.5.25.11.150.1.1.1.1.err
perlbenchNS.2.arg: 0 # ./perlbench/perlbench -I./perlbench/lib ./perlbench/diffmail.pl 4 800 10 17 19 300 > ./perlbench/diffmail.4.800.10.17.19.300.out 2>> ./perlbench/diffmail.4.800.10.17.19.300.err
perlbenchNS.3.arg: 0 # ./perlbench/perlbench -I./perlbench/lib ./perlbench/splitmail.pl 1600 12 26 16 4500 > ./perlbench/splitmail.1600.12.26.16.4500.out 2>> ./perlbench/splitmail.1600.12.26.16.4500.err

Povray

povrayNS.1.arg: 0 # ./povray/povray ./povray/SPEC-benchmark-ref.ini > ./povray/SPEC-benchmark-ref.stdout 2>> ./povray/SPEC-benchmark-ref.stderr
Insts: 1250666008507
Huge number of differences in the log output. The tga output differs from the reference output but is clearly a chess board.

Sjeng

sjengNS.1.arg: 0 # ./sjeng/sjeng.peak.ev6 ./sjeng/ref.txt > ./sjeng/ref.out 2>> ./sjeng/ref.err
Preliminary Test: 3687846212529

Soplex

Compiled as: g++ -non_shared -DSPEC_CPU -DNDEBUG -O4 *.cc -o soplex -lm
soplexNS.1.arg: 0 # ./soplex/soplex -s1 -e -m45000 ./soplex/pds-50.mps > ./soplex/pds-50.mps.out 2>> ./soplex/pds-50.mps.stderr
soplexNS.2.arg: 0 # ./soplex/soplex -m3500 ./soplex/ref.mps > ./soplex/ref.out 2>> ./soplex/ref.stderr

SpecrandF

specrandFNS.1.arg: 0 # ./specrandF/specrand 1255432124 234923 > ./specrandF/rand.234923.out 2>> ./specrandF/rand.234923.err
Preliminary Test: Passed (626925130 insts)

Specrand

specrandNS.1.arg: 0 # ./specrand/specrand 1255432124 234923 > ./specrand/rand.234923.out 2>> ./specrand/rand.234923.err
Preliminary Test: Passed (626925130 insts)

Sphinx3

Compiled as: cc -non_shared -DSPEC_CPU -O4 *.c -o sphinx3 -lm
sphinx3NS.1.arg: 0 # ./sphinx3/sphinx_livepretend ./sphinx3/ctlfile ./sphinx3/ ./sphinx3/args.an4 > ./sphinx3/an4.log 2>> ./sphinx3/an4.err

Tonto

tontoNS.1.arg: 0 # ./tonto/tonto > ./tonto/tonto.out 2>> ./tonto/tonto.err

Wrf

wrfNS.1.arg: 0 # ./wrf/wrf > ./wrf/rsl.out.0000 2>> ./wrf/wrf.err

Xalancbmk

xalancbmkNS.1.arg: 0 # ./xalancbmk/Xalan -v ./xalancbmk/t5.xml ./xalancbmk/xalanc.xsl > ./xalancbmk/ref.out 2>> ./xalancbmk/ref.err

Zeusmp

zeusmpNS.1.arg: 0 # ./zeusmp/zeusmp > ./zeusmp/zeusmp.stdout 2>> ./zeusmp/zeusmp.err
Preliminary Test: Output passed, execution incomplete.

Personal tools