Surfin' Safari

Announcing SunSpider 0.9

Posted by Maciej Stachowiak on Tuesday, December 18th, 2007 at 2:06 pm

A New JavaScript Benchmark From the WebKit Team

Here in WebKit-land, we strive for great performance and we love to make things faster. When doing performance work, it’s critical to be able to measure how fast you are at any given time and track changes. And it’s important to make sure your metric reflects realistic use cases. None of the existing JavaScript benchmarks were exactly suited to our needs (more on that below), so we decided to make our own.

We’re widely announcing this benchmark today because we’d like to share with other browser-hosted and standalone JavaScript engines. And we’d like to have input from JavaScript experts on the usefulness and validity of the tests. In that spirit of cooperation, we’re not going to post competitive numbers or even comparisons to older Safari/WebKit versions for now (although I can tell you that the shipping version of Safari 3 is not the fastest JS engine we tested).

Try It Out!

But people like playing with these benchmarks. So I’m sure a lot of you would like to try your luck and test your browser. If you’d like to post your results in the comments, please mention what browser, what version, and what hardware and OS you’re running on. Make sure to quit extraneous apps and avoid jiggling your mouse for the minute or two that it runs. Have fun!

The Gory Details

There are a lot of JavaScript benchmarks out there already: Celtic Kane’s JavaScript speed test, the JavaScript processing test from iBench, Jason Orendorff’s JavaScript speed tests, John Resig’s tests, and others. But none of these were quite what we needed to measure improve real-world JavaScript performance.

So we made our own. Why bother? Well, we think our benchmark has a combination of useful properties that we didn’t see in any existing test:

  • It’s based on real code that does interesting things; both things that the web apps of today are doing, and more advanced code of the sorts we can expect as web apps become more advanced. Very few of the tests could be classed as microbenchmarks.
  • It’s balanced between different aspects of the JavaScript language — not dominated by just a small handful of different things. In fact, we collected test cases from all over the web, including from other benchmarks. But at the same time, we avoided DOM tests and stuck to the core JavaScript language itself.
  • It’s super easy to run in the browser or from the command line, so you can test both pure engine performance, and the results you actually get in the browser.
  • We included statistical analysis so you can see how stable the results you’re getting really are.

Incidentally, we’ve been doing a lot of work on JavaScript performance lately. If you’re interested in helping out, stop by the webkit-dev@webkit.org mailing list, or the #webkit IRC channel on chat.freenode.net and say hi. Our engine is highly hackable so it’s easier than you think to get started.

62 Responses to “Announcing SunSpider 0.9”

  1. gloubibou Says:

    Suggestion: The test should print out the user agent with the results

    I ran SunSpider 0.9 on a MacBookPro 2.33 GHz, Mac OS X 10.5.1, stock Safari 3:

    ============================================
    RESULTS (means and 95% confidence intervals)
    --------------------------------------------
    Total:                  9206.2ms +/- 0.4%
    --------------------------------------------
    
      3d:                   1333.8ms +/- 1.0%
        cube:                446.6ms +/- 0.6%
        morph:               525.6ms +/- 2.2%
        raytrace:            361.6ms +/- 1.0%
    
      access:               1896.0ms +/- 0.4%
        binary-trees:        142.0ms +/- 0.6%
        fannkuch:            823.2ms +/- 0.5%
        nbody:               424.4ms +/- 0.9%
        nsieve:              506.4ms +/- 1.7%
    
      bitops:               1641.2ms +/- 0.3%
        3bit-bits-in-byte:   315.2ms +/- 0.9%
        bits-in-byte:        437.2ms +/- 1.2%
        bitwise-and:         427.2ms +/- 0.5%
        nsieve-bits:         461.6ms +/- 0.6%
    
      controlflow:           224.6ms +/- 11.6%
        recursive:           224.6ms +/- 11.6%
    
      crypto:                770.8ms +/- 0.6%
        aes:                 219.2ms +/- 0.6%
        md5:                 277.8ms +/- 0.9%
        sha1:                273.8ms +/- 0.9%
    
      date:                  501.8ms +/- 0.4%
        format-tofte:        255.0ms +/- 0.5%
        format-xparb:        246.8ms +/- 0.9%
    
      math:                 1178.6ms +/- 0.5%
        cordic:              621.6ms +/- 0.5%
        partial-sums:        296.0ms +/- 1.2%
        spectral-norm:       261.0ms +/- 1.0%
    
      regexp:                344.0ms +/- 0.3%
        dna:                 344.0ms +/- 0.3%
    
      string:               1315.4ms +/- 0.8%
        base64:              302.4ms +/- 0.4%
        fasta:               351.6ms +/- 1.1%
        tagcloud:            232.8ms +/- 1.6%
        unpack-code:         210.0ms +/- 3.6%
        validate-input:      218.6ms +/- 2.0%
    
  2. crschmidt Says:

    System: OS X 10.4.11 on a macbook with… well, whatever early macbooks had :) and 2GB of RAM.

    Each browser was closed and reopened immediately prior to running the test.

    Safari 3.0.4: http://tinyurl.com/3288j5
    Latest OmniWeb: http://tinyurl.com/2ed259
    FF 3b1: http://tinyurl.com/ypluck
    FF2.0.0.11: http://tinyurl.com/yv3z3l
    Opera 9.23 gave me problems with tests failing: http://tinyurl.com/254kxe

  3. Alexander Strange Says:

    Nice results URL. Can’t we get some kind of base64+zlib for those things?

    (Without stopping the rather long compile going on, Safari 3 is better than the other two release browsers I have at 10088ms +/- .7% on an MBP Core 2 2.4Ghz)

  4. Joachim Says:

    Opera 9.5/4579: http://tinyurl.com/2bfl4o
    Safari 3.0.4: http://tinyurl.com/ywaez2

    Are recent WebKit nightlies significantly faster than 3.0.4?

  5. Harry Says:

    New iMac 2.4 GHz w/1GB RAM.

    Safari: 8760

    Camino: 10696

  6. RichB Says:

    Nokia N95 v20/series60 browser/webkit doesn’t begin the test run.

  7. icefox Says:

    Macbook 2Ghz

    webkit depo built with debug from this week

    ============================================
    RESULTS (means and 95% confidence intervals)
    --------------------------------------------
    Total:                 11847.6ms +/- 0.2%
    --------------------------------------------
    
      3d:                   1623.4ms +/- 0.7%
        cube:                531.0ms +/- 1.0%
        morph:               643.6ms +/- 1.1%
        raytrace:            448.8ms +/- 0.7%
    
      access:               2352.6ms +/- 0.3%
        binary-trees:        216.2ms +/- 0.3%
        fannkuch:            945.4ms +/- 0.7%
        nbody:               489.4ms +/- 0.5%
        nsieve:              701.6ms +/- 1.2%
    
      bitops:               1987.0ms +/- 0.5%
        3bit-bits-in-byte:   417.0ms +/- 0.6%
        bits-in-byte:        534.0ms +/- 0.5%
        bitwise-and:         488.6ms +/- 1.1%
        nsieve-bits:         547.4ms +/- 0.6%
    
      controlflow:           313.2ms +/- 0.4%
        recursive:           313.2ms +/- 0.4%
    
      crypto:               1011.8ms +/- 0.2%
        aes:                 279.6ms +/- 0.4%
        md5:                 374.0ms +/- 0.4%
        sha1:                358.2ms +/- 0.3%
    
      date:                  737.4ms +/- 0.3%
        format-tofte:        361.2ms +/- 0.7%
        format-xparb:        376.2ms +/- 0.3%
    
      math:                 1438.4ms +/- 0.3%
        cordic:              758.8ms +/- 0.4%
        partial-sums:        340.2ms +/- 0.9%
        spectral-norm:       339.4ms +/- 0.6%
    
      regexp:                436.0ms +/- 0.6%
        dna:                 436.0ms +/- 0.6%
    
      string:               1947.8ms +/- 0.5%
        base64:              507.0ms +/- 1.4%
        fasta:               457.0ms +/- 0.5%
        tagcloud:            335.2ms +/- 0.6%
        unpack-code:         320.6ms +/- 0.9%
        validate-input:      328.0ms +/- 0.5%
    
  8. David Smith Says:

    icefox: definitely try retesting with a nondebug build. You should get around a factor of 2 better than that.

  9. Mark Rowe Says:

    icefox: There is no sense in running performance tests on debug builds.

  10. brainsik Says:

    MacBook Pro 2.33 GHz Intel Core 2 Duo

    Closed everything down and then ran each browser individually.

    Safari 3.0.4 (5523.10.6) aka stock v3.0.4

    Camino 2007112812 (1.5.4) aka stock v1.5.4

    Firefox 2.0.0.9 aka stock 2.0.0.9

  11. geoff Says:

    Nice work guys.

    On a 2.4 GHz MacBook Pro:

    Firefox 2: 13982.8ms
    Firefox 3.0b: 10127.4ms

    Opera 9.2: 12553.6ms
    Opera 9.5b: 7295.0ms

    Safari 3: 8999.2ms
    Webkit – recent nightly build (r28767): 3861.2ms !!!

  12. crschmidt Says:

    From earlier:
    Safari 3.0.4: http://tinyurl.com/3288j5 , 11914.2ms

    Add to that:
    Webkit nightly from last night: http://tinyurl.com/2g2wyo, 5133.6ms

  13. Pingback from Benchmarking And Testing Browsers | Robert Accettura’s Fun With Wordage:

    [...] WebKit team has now released SunSpider a javascript benchmarking tool. Something tells me this will lead to some performance improvements [...]

  14. Darth Ed Says:

    FYI, there’s a typo in the text at http://webkit.org/perf/sunspider-0.9/sunspider.html
    “triest” should be “tries”.

  15. bluea Says:

    Powerbook G4 867 “Minime”: Safari 3.0.4 (5523.10) on Mac OS X 10.5.1: 47683.0

    Powerbook G4 867 “Minime”: Safari Nightly r28843 on Mac OS X 10.5.1: 25275.2

    Firefox 2.0.0.11 on the same machine bogs down in one of the bit-swap tests and (presumably) messes up the results by opening a dialog sheet and halting the script.

    47k – I think I’m the winner! Where’s the booby prize department?

  16. Mountain/\Ash Says:

    iMac 2.8 GHz Intel Core 2 Duo – GB Ram

    8079.8ms +/- 0.3% – Safari 3.0.4
    13630.4ms +/- 6.9% – Firefox 2.0.0.9

  17. Maciej Stachowiak Says:

    No one has posted results for any Windows or Linux browsers yet, it would be neat to see those.

  18. leeg Says:

    http://tinyurl.com/2zwbyd

    Operating System:
    Linux mabinogi 2.6.22-14-powerpc #1 Sun Oct 14 21:42:06 GMT 2007 ppc GNU/Linux
    Browser:
    Mozilla Firefox 2.0.0.11
    Hardware:
    processor : 0
    cpu : 7447A, altivec supported
    clock : 666.666000MHz
    revision : 0.2 (pvr 8003 0102)
    bogomips : 36.73
    timebase : 18432000
    platform : PowerMac
    machine : PowerBook6,7
    motherboard : PowerBook6,7 MacRISC3 Power Macintosh
    detected as : 287 (iBook G4)
    pmac flags : 0000001a
    L2 cache : 512K unified
    pmac-generation : NewWorld

    MemTotal: 1033564 kB

  19. Rijk Says:

    Windows XP SP2, AMD Athlon XP 3000+ (2.1Ghz), 1 GB RAM

    13525.8ms +/- 3.4% – Opera 9.50 latest internal build
    16442.6ms +/- 0.7% – Safari 3.0.4 (523.13)
    19683.4ms +/- 0.9% – Firefox 3 beta 1
    26424.4ms +/- 23.8% – Firefox 3 beta 2
    32672.4ms +/- 1.0% – Firefox 2.0.11
    82053.8ms +/- 3.9% – IE 7
    NaNms +/- NaN% – Opera 9.25 (some test return NaN…)

  20. Mark Rowe Says:

    Rijk, if you’re getting a variance > 5% I would strongly recommend running the test again. I would also be very interested to see how the latest WebKit nightly build compares to the other browsers on your system.

  21. Rijk Says:

    Mark: good idea, here’s a second run:
    19587.2ms +/- 1.1% – Firefox 3 beta 2

    About the nightly, I’ve no idea how to run that. I’m not geek enough :)

  22. Mark Rowe Says:

    Rijk: That figure is much closer to what I would expect for Firefox 3b2.

    Testing the Safari nightly build should be quite easy. Grab the latest Windows build from http://nightly.webkit.org/, and launch it by double-clicking the “run-webkit-nightly.cmd” script. Voilà!

  23. Thinine Says:

    Here are the results for IE7 on a P42.8HT, Windows XP. I know the percentages are high for some tests, but they always are.

    ============================================RESULTS (means and 95% confidence intervals)
    --------------------------------------------
    Total:                  60827.8ms +/- 32.2%
    --------------------------------------------
    3d:                    2197.0ms +/- 0.5%
    cube:                 597.2ms +/- 1.5%
    morph:                653.0ms +/- 1.4%
    raytrace:             946.8ms +/- 1.2%
    access:                2825.0ms +/- 0.6%
    binary-trees:         740.6ms +/- 1.4%
    fannkuch:            1015.8ms +/- 0.0%
    nbody:                578.0ms +/- 0.0%
    nsieve:               490.6ms +/- 2.2%
    bitops:                2415.8ms +/- 1.1%
    3bit-bits-in-byte:    644.0ms +/- 1.3%
    bits-in-byte:         584.4ms +/- 1.9%
    bitwise-and:          643.6ms +/- 3.3%
    nsieve-bits:          543.8ms +/- 1.6%
    controlflow:            796.8ms +/- 1.8%
    recursive:            796.8ms +/- 1.8%
    crypto:                1621.6ms +/- 1.4%
    aes:                  575.0ms +/- 1.5%
    md5:                  521.6ms +/- 2.2%
    sha1:                 525.0ms +/- 2.1%
    date:                  1499.8ms +/- 0.9%
    format-tofte:         703.0ms +/- 2.0%
    format-xparb:         796.8ms +/- 0.0%
    math:                  1943.8ms +/- 0.5%
    cordic:               734.2ms +/- 1.9%
    partial-sums:         453.2ms +/- 0.1%
    spectral-norm:        756.4ms +/- 1.4%
    regexp:                 428.2ms +/- 2.6%
    dna:                  428.2ms +/- 2.6%
    string:               47099.8ms +/- 41.6%
    base64:             28634.4ms +/- 67.4%
    fasta:                884.4ms +/- 1.2%
    tagcloud:            4931.2ms +/- 2.6%
    unpack-code:          968.6ms +/- 5.7%
    validate-input:     11681.2ms +/- 4.7%
    

    Needless to say, WebKit is quite a bit faster.

  24. Synchro Says:

    I did a bunch of tests and graphed them, for some reason my trackback didn’t work:

    http://marcus.bointon.com/archives/50-SunSpider-Benchmarks-WebKit-Rocks.html

    Main surprise is how amazingly fast Safari Win is!

  25. Pingback from Announcing SunSpider 0.9 (A New JavaScript Benchmark From the WebKit Team) « turnings:

    [...] SunSpider 0.9 (A New JavaScript Benchmark From the WebKit Team) 19 12 2007 Announcing SunSpider 0.9: Incidentally, we’ve been doing a lot of work on JavaScript performance lately. If you’re [...]

  26. johanbostrom Says:

    MacBook Pro 2.4GHz (standard config)
    Safari 3.0.4 (5523.10.6): 8601.4s +/- 0.1%
    Firefox 2.0.0.11: 11.577.4ms +/- 0.4%

    iPhone 1.1.1:
    3d-cube: 19927ms
    3d-morph: 39906ms
    3d-raytrace: 22377ms
    access-binary-trees: 13276ms
    access-fannkuch: 5018ms
    access-nbody: 18639ms
    access-nsieve: 10047ms
    bitops-3bit-bits-in-byte: 5104ms
    bitops-bits-in-byte: 10026ms
    (then it stops the execution)

  27. paulsmith Says:

    http://tinyurl.com/yptvpu

    Ubuntu Gutsy Gibbon on a Lenovo ThinkPad T61. Firefox 2.0.0.11. 2.6.22-14-generic GNU/Linux kernel. 3GB RAM. 2 x Core Duo 2 @ 2.0 GHz

  28. Rijk Says:

    How can I be sure the nightly build is running? ‘Help > About’ still says 3.0.4… Test run gives me 9887.4ms +/- 1.0% though, so it seems this is something else.

  29. Thinine Says:

    Hmm. Should file the Win/Mac performance difference as a bug. Or several. Or it may just be compiler differences between GCC and MS’s.

  30. Maciej Stachowiak Says:

    Rijk: javascript:alert(navigator.userAgent) should be able to tell for sure if you are running a nightly. The About dialog won’t show anything different.

  31. pyang Says:

    MacBook C2D 2.16 GHz, 2 GB RAM

    Camino Trunk Version 2007121601 (2.0a1pre): 8283.8ms +/- 0.6%; http://tinyurl.com/35dnyw
    Safari Version 3.0.4 (5523.10.6): 9711.6ms +/- 0.1%; http://tinyurl.com/2kd6co
    Firefox 2.0.0.11: 13286.6ms +/- 0.5%; http://tinyurl.com/35vp76

  32. codinghorror Says:

    I posted some Windows summary results here, along with the complete results as a text file:

    http://www.codinghorror.com/blog/archives/001023.html

  33. ideum Says:

    iMac 2.8 GHz Extreme, 2GB RAM, OS X 10.5.1, Safari 3.0.4 (5523.10.6)

    http://tinyurl.com/272etn

    ============================================
    RESULTS (means and 95% confidence intervals)
    ——————————————–
    Total: 7320.6ms +/- 0.2%
    ——————————————–

  34. mikerundle Says:

    Hey guys, here are my tests:

    Camino 1.5.2 running on iMac 2.4 Ghz Core 2 Duo 10.4.11, 1GB RAM (yeah I sure need more!):

    ============================================
    RESULTS (means and 95% confidence intervals)
    ——————————————–
    Total: 11148.8ms +/- 0.3%
    ——————————————–

    Safari 3.0.4 running on same iMac:

    ============================================
    RESULTS (means and 95% confidence intervals)
    ——————————————–
    Total: 8838.6ms +/- 0.6%
    ——————————————–

    Nice!!!!

  35. mikerundle Says:

    Same iMac from up above, but with Webkit Nightly:

    ===========================================
    RESULTS (means and 95% confidence intervals)
    ——————————————–
    Total: 3784.4ms +/- 1.3%
    ——————————————–

  36. Grant Hutchins Says:

    PowerMac G5 dual 2.7 GHz, 2.5 GB RAM
    Safari Version 3.0.4 (5523.10.6)

  37. neeks Says:

    MacBook C2D 2.0 GHz, 2 GB RAM
    Safari 3.0.4 (5523.10.6): 10329.8ms +/- 0.1%

  38. Synchro Says:

    The benchmark results output is a bit fiddly to deal with – it would be much more useful for benchmark automation if the tests, timings and confidence levels were generated in some CSV format for easy parsing. Mixing in the test group results is also a bit annoying – perhaps display them together first and indicate which group each test is part of? Perhaps a bit like this (tab delimited, so may not render well):

    ============================================
    RESULTS (means in ms and 95% confidence intervals in %)
    --------------------------------------------
    Total	2483.2	6.6
    --------------------------------------------
    
    group	3d	414.6	36.0
    group	access	398.8	16.2
    group	bitops	314.8	26.0
    group	controlflow	46.0	41.0
    group	crypto	272.8	22.9
    group	date	180.0	41.2
    group	math	256.2	41.7
    group	regexp	112.0	54.0
    group	string	488.0	25.2
    3d	cube	122.0	58.7
    3d	morph	212.4	48.2
    3d	raytrace	80.2	57.1
    access	binary-trees	42.0	56.9
    access	fannkuch	136.0	17.8
    access	nbody	74.2	19.3
    access	nsieve	146.6	48.4
    bitops	3bit-bits-in-byte	74.0	63.5
    bitops	bits-in-byte	78.4	28.7
    bitops	bitwise-and	60.2	26.1
    bitops	nsieve-bits	102.2	9.7
    controlflow	recursive	46.0	41.0
    crypto	aes	74.2	53.5
    crypto	md5	100.4	52.3
    crypto	sha1	98.2	65.8
    date	format-tofte	76.0	52.5
    date	format-xparb	104.0	67.9
    math	cordic	122.2	78.5
    math	partial-sums	60.0	38.8
    math	spectral-norm	74.0	48.4
    regexp	dna	112.0	54.0
    string	base64	74.0	36.8
    string	fasta	74.0	60.1
    string	tagcloud	140.0	38.2
    string	unpack-code	76.0	55.0
    string	validate-input	124.0	36.6
    

    Alternatively it might be nice to have different output formats switchable by a param.

  39. Mark Rowe Says:

    The command-line version of SunSpider generates the result files as JavaScript. They look something like the following:

    var output = [
    {
        "3d-cube": 149,
        "3d-morph": 128,
        "3d-raytrace": 138,
    ...
    },
    {
        "3d-cube": 149,
        "3d-morph": 128,
        "3d-raytrace": 138,
    ...
    }];
    

    This makes it nice and trivial for the SunSpider test harness, which is also written in JS, to parse the results. The URL generated by the browser-hosted version gets unpacked into a similar structure in-memory by the results page that generates the human-readable output. This output isn’t really intended to be parsed by a script.

  40. EdwinSmith Says:

    bitops-bitwise-and doesn’t declare bitwiseAndValue as a local variable. It’s a valid test either way but it doesn’t seem like this test wants to be measuring global variable access, as much as it wants to measure bitwise operator performance.

    Could it be a bug? if so, there’s your report :-) Goes without saying, some engines are tuned to have faster access to local variables than globals.

  41. Maciej Stachowiak Says:

    EdwinSmith:

    This was originally reported as a problem with bitwise ops in various browsers, but you’re right that it stresses global access more than bitops. We’re considering moving this test to the “access” category as a global property access test.

  42. ggaren Says:

    Edwin: To clarify, it’s intentional that bitops-bitwise-and uses globals. Lots of scripts on the web use globals, and we wanted SunSpider to be representative.

    You’re right, though — an engine that lacks fast access to globals will do poorly on the test, regardless of its bitops implementation.

  43. aehiilrs Says:

    12″ Powerbook G4, 1.5GHz + 768 megs of ram, Mac OS 10.5.1, Safari 3.0.4.

    I apparently need to upgrade.

    ============================================
    RESULTS (means and 95% confidence intervals)
    ——————————————–
    Total: 31682.0ms +/- 0.9%
    ——————————————–

    3d: 4752.2ms +/- 2.2%
    cube: 1470.2ms +/- 5.1%
    morph: 2002.8ms +/- 6.1%
    raytrace: 1279.2ms +/- 8.5%

    access: 6065.2ms +/- 2.8%
    binary-trees: 684.6ms +/- 14.9%
    fannkuch: 2246.8ms +/- 6.3%
    nbody: 1478.2ms +/- 6.8%
    nsieve: 1655.6ms +/- 7.4%

    bitops: 5436.0ms +/- 2.1%
    3bit-bits-in-byte: 1201.6ms +/- 5.7%
    bits-in-byte: 1476.2ms +/- 7.7%
    bitwise-and: 1141.6ms +/- 4.9%
    nsieve-bits: 1616.6ms +/- 10.5%

    controlflow: 987.6ms +/- 10.5%
    recursive: 987.6ms +/- 10.5%

    crypto: 2727.6ms +/- 2.9%
    aes: 689.4ms +/- 7.9%
    md5: 1031.6ms +/- 5.5%
    sha1: 1006.6ms +/- 7.0%

    date: 2067.0ms +/- 5.4%
    format-tofte: 915.6ms +/- 9.2%
    format-xparb: 1151.4ms +/- 5.2%

    math: 4008.6ms +/- 4.7%
    cordic: 1974.4ms +/- 8.7%
    partial-sums: 941.0ms +/- 10.3%
    spectral-norm: 1093.2ms +/- 14.9%

    regexp: 961.0ms +/- 9.0%
    dna: 961.0ms +/- 9.0%

    string: 4676.8ms +/- 2.2%
    base64: 1090.4ms +/- 7.6%
    fasta: 1122.2ms +/- 6.9%
    tagcloud: 841.6ms +/- 6.7%
    unpack-code: 763.8ms +/- 10.9%
    validate-input: 858.8ms +/- 8.6%

  44. name99 Says:

    I tried running it on iPhone and I don’t know if it crashed or was just running extremely slowly, but some text came up then nothing much happened for five minutes. Not a promising start. One more bug to fix for iPhone 1.2.

  45. timofonic Says:

    My system:
    CPU: AMD Athlon XP 1700+ (1463 MHz)
    RAM: 512MB SDRAM
    Graphics card: ATI Radeon 9200SE
    Operating System: Linux kernel 2.6.22 (Gentoo 2007.0)

    Here are the test results:
    Firefox/2.0.0.11

    9.50 Beta 2

  46. James W Cready Says:

    My system:
    CPU: 2 x 2.66 GHz Dual-Core Xeon
    RAM: 2 GB 677 MHz DDR2
    OS : Mac OS X Leopard
    Browser: WebKit nightly (5523.10)

    ============================================
    RESULTS (means and 95% confidence intervals)
    --------------------------------------------
    Total:                 3342.8ms +/- 0.4%
    --------------------------------------------
    
      3d:                   399.2ms +/- 1.6%
        cube:               131.6ms +/- 3.4%
        morph:              124.6ms +/- 1.1%
        raytrace:           143.0ms +/- 0.6%
    
      access:               535.2ms +/- 0.9%
        binary-trees:        83.2ms +/- 1.3%
        fannkuch:           254.4ms +/- 1.0%
        nbody:              145.4ms +/- 0.8%
        nsieve:              52.2ms +/- 2.6%
    
      bitops:               458.4ms +/- 0.9%
        3bit-bits-in-byte:   87.4ms +/- 1.3%
        bits-in-byte:       112.2ms +/- 0.9%
        bitwise-and:        159.4ms +/- 1.4%
        nsieve-bits:         99.4ms +/- 2.1%
    
      controlflow:          115.8ms +/- 0.5%
        recursive:          115.8ms +/- 0.5%
    
      crypto:               271.2ms +/- 0.4%
        aes:                 75.2ms +/- 0.7%
        md5:                 99.2ms +/- 1.0%
        sha1:                96.8ms +/- 0.6%
    
      date:                 275.6ms +/- 0.4%
        format-tofte:       123.2ms +/- 0.5%
        format-xparb:       152.4ms +/- 0.4%
    
      math:                 443.2ms +/- 0.6%
        cordic:             173.4ms +/- 0.8%
        partial-sums:       182.6ms +/- 0.9%
        spectral-norm:       87.2ms +/- 0.6%
    
      regexp:               165.8ms +/- 0.3%
        dna:                165.8ms +/- 0.3%
    
      string:               678.4ms +/- 0.5%
        base64:              97.0ms +/- 0.9%
        fasta:              171.0ms +/- 1.1%
        tagcloud:           144.2ms +/- 0.9%
        unpack-code:        144.0ms +/- 1.1%
        validate-input:     122.2ms +/- 1.8%
    
  47. Simon Says:

    Hi Webkit

    Makes me very proud, that you are using my javascript code for the 3d cube benchmark [http://www.speich.net/computer/moztesting/3d.htm]. That’s really great.
    But (why’s there always a but?) it would have been nice, if you had at least notified me before using it.

    Happy testing Simon

  48. Synchro Says:

    What is it with the little sun logo in the top left of the benchmark page? It seems to appear differently in every browser (empty yellow box, thick yellow line, question mark, or a sun image)! Is it some kind of secret CSS validation test to wind up IE users?

  49. EdwinSmith Says:

    In math-partial-sums, partial() also contains a bunch of global references: a2 – a9, k3, sk, ck. Just wanted to point it out to see if that also is intentional. (if so, comments to that effect would be cool).

    Cheers, and cool benchmarks!

  50. Pingback from Announcing SunSpider 0.9 « outaTiME:

    [...] (Via Surfin’ Safari.) [...]

  51. Pingback from iPhone Updates - Apple iPhone Software, iPhone Hardware, iPhone cables, iPhone cases, iPhone News:

    [...] that powers Safari) wanted to measure how fast Safari’s JavaScript Engine really was. They looked at the existing JavaScript benchmarks and decided none of them did what they needed, so they built [...]

  52. Pingback from iPhone Software » Blog Archive » SunSpider 0.9:

    [...] that powers Safari) wanted to measure how fast Safari’s JavaScript Engine really was. They looked at the existing JavaScript benchmarks and decided none of them did what they needed, so they built [...]

  53. Maciej Stachowiak Says:

    @Simon

    Sorry for the lack of notice, you are credited in the test file however: http ://webkit.org/perf/sunspider-0.9/3d-cube.html.

  54. Maciej Stachowiak Says:

    @EdwinSmith:

    That test file comes straight from the computer language shootout, and we certainly didn’t modify it intentionally, but it does look like the person who oriiginally wrote the code made a mistake.

  55. Spot Says:

    OSX 10.4.11
    1.42 GHz G4
    512 MB DDR SDRAM

    Webkit (523.12.2) 15040.6ms +/- 1.9%
    Opera 9.50.4506 19597.4ms +/- 0.9%
    Shiira 2.2 (071003) 29371.0ms +/- 5.8%
    Sunrise 1.5.6 28998.0ms +/- 0.4%
    Demeter 1.0.7 34182.6ms +/- 6.6%
    iCab 4.0.0 beta19 34793.6ms +/- 2.8%
    Camino 1.6a1 37203.4ms +/- 1.4%

  56. jonknee Says:

    Here are results for the OLPC XO laptop. Not the fastest, but it works.

    ============================================
    RESULTS (means and 95% confidence intervals)
    --------------------------------------------
    Total:                 176167.2ms +/- 0.5%
    --------------------------------------------
    
      3d:                   34310.6ms +/- 1.3%
        cube:                6620.2ms +/- 1.4%
        morph:              22864.4ms +/- 2.2%
        raytrace:            4826.0ms +/- 1.4%
    
      access:               18614.4ms +/- 1.4%
        binary-trees:        2479.0ms +/- 7.0%
        fannkuch:            6243.8ms +/- 1.0%
        nbody:               6607.4ms +/- 1.0%
        nsieve:              3284.2ms +/- 0.8%
    
      bitops:               30300.2ms +/- 2.6%
        3bit-bits-in-byte:   6556.2ms +/- 4.1%
        bits-in-byte:        5627.2ms +/- 0.8%
        bitwise-and:        13951.8ms +/- 5.6%
        nsieve-bits:         4165.0ms +/- 0.5%
    
      controlflow:           1844.6ms +/- 7.1%
        recursive:           1844.6ms +/- 7.1%
    
      crypto:               10088.4ms +/- 2.0%
        aes:                 4469.4ms +/- 2.4%
        md5:                 2782.6ms +/- 4.3%
        sha1:                2836.4ms +/- 0.6%
    
      date:                 11062.4ms +/- 2.6%
        format-tofte:        7142.2ms +/- 4.7%
        format-xparb:        3920.2ms +/- 1.4%
    
      math:                 23868.8ms +/- 1.4%
        cordic:             11430.6ms +/- 2.2%
        partial-sums:        8856.6ms +/- 2.6%
        spectral-norm:       3581.6ms +/- 2.0%
    
      regexp:                6929.0ms +/- 1.6%
        dna:                 6929.0ms +/- 1.6%
    
      string:               39148.8ms +/- 1.0%
        base64:              7545.0ms +/- 3.3%
        fasta:               7969.4ms +/- 2.9%
        tagcloud:            6121.4ms +/- 3.0%
        unpack-code:        12942.6ms +/- 1.1%
        validate-input:      4570.4ms +/- 1.3%
    
  57. Pingback from Alp Toker » Blog Archive » HTML5 canvas enhancements, Acid2 support and more:

    [...] further ahead of the stock browser shipped on ARM devices like the N800/N810. Try one of the freely available JS/AJAX benchmarks and see for [...]

  58. Pingback from Alp Toker: HTML5 canvas enhancements, Acid2 support and more - 工程師的雞排攤:

    [...] further ahead of the stock browser shipped on ARM devices like the N800/N810. Try one of the freely available JS/AJAX benchmarks and see for [...]

  59. mcroft Says:

    I also can’t get it to run to completion on mobile Safari on an iPhone. Is this on the list for a future iPhone update?

  60. Pingback from afongen » links for 2007-12-28:

    [...] Announcing SunSpider 0.9 JavaScript performance benchmark (tags: javascript performance) [...]

  61. mcroft Says:

    Just for shits and grins, I ran it with the new iCab 4.0 on my G5 dual 2.7 (4GB Ram) 10.5.1
    Safari 3 (stock) was 15452.4ms total (or 1.1 times slower than iCab) A recent nightly build of Safari was 1.86 times faster (7521.6ms)

     ============================================
    RESULTS (means and 95% confidence intervals)
    --------------------------------------------
    Total:                 13957.6ms +/- 1.0%
    --------------------------------------------
    
      3d:                   2135.6ms +/- 0.6%
        cube:                635.6ms +/- 1.7%
        morph:               955.2ms +/- 2.0%
        raytrace:            544.8ms +/- 1.5%
    
      access:               2767.8ms +/- 1.5%
        binary-trees:        259.2ms +/- 0.7%
        fannkuch:           1082.0ms +/- 2.7%
        nbody:               502.2ms +/- 0.7%
        nsieve:              924.4ms +/- 1.6%
    
      bitops:               2494.8ms +/- 1.5%
        3bit-bits-in-byte:   540.4ms +/- 1.9%
        bits-in-byte:        670.0ms +/- 1.9%
        bitwise-and:         577.0ms +/- 3.2%
        nsieve-bits:         707.4ms +/- 1.7%
    
      controlflow:           355.2ms +/- 0.8%
        recursive:           355.2ms +/- 0.8%
    
      crypto:               1192.4ms +/- 0.6%
        aes:                 311.2ms +/- 1.0%
        md5:                 445.6ms +/- 1.5%
        sha1:                435.6ms +/- 0.5%
    
      date:                  860.2ms +/- 5.9%
        format-tofte:        395.2ms +/- 13.3%
        format-xparb:        465.0ms +/- 0.6%
    
      math:                 1756.8ms +/- 2.0%
        cordic:              936.8ms +/- 1.9%
        partial-sums:        381.4ms +/- 1.9%
        spectral-norm:       438.6ms +/- 3.7%
    
      regexp:                422.8ms +/- 0.1%
        dna:                 422.8ms +/- 0.1%
    
      string:               1972.0ms +/- 0.8%
        base64:              434.2ms +/- 1.5%
        fasta:               518.2ms +/- 1.7%
        tagcloud:            376.4ms +/- 0.6%
        unpack-code:         309.8ms +/- 0.5%
        validate-input:      333.4ms +/- 1.4%
    
  62. scottn Says:

    I created an account just so I could post this as there doesn’t seem to be much Windows results. Out of curiousity I downloaded Safari the other day and played with it (I am a big Opera Fan).

    {Windows Vista32, Core2Duo 2.6ghz, 1gb RAM}
    
    "From" = Webkit Nightly r29785
    "To" = Safari 3.0.4 Beta
    

    WOW, What an improvement! Safari went from losing to Opera 9.50 Weekly (posted last) to destroying it. I won’t even post IE7 numbers ast they were a little worse than Safari 3.0.4. Webkit for the win. And Open Source as well? Sheesh..

    TEST                   COMPARISON            FROM                 TO             DETAILS
    =============================================================================
    
    ** TOTAL **:           *2.10x as slow*   4784.8ms +/- 2.2%   10068.0ms +/- 3.8%     significant
    
    =============================================================================
    
      3d:                  *2.34x as slow*    578.4ms +/- 4.9%    1353.6ms +/- 11.7%     significant
        cube:              *2.35x as slow*    186.0ms +/- 6.1%     436.6ms +/- 16.9%     significant
        morph:             *2.83x as slow*    193.2ms +/- 8.9%     545.8ms +/- 21.2%     significant
        raytrace:          *1.86x as slow*    199.2ms +/- 4.3%     371.2ms +/- 19.0%     significant
    
      access:              *2.45x as slow*    738.6ms +/- 8.2%    1809.6ms +/- 9.9%     significant
        binary-trees:      *1.86x as slow*     97.0ms +/- 9.3%     180.8ms +/- 14.6%     significant
        fannkuch:          *2.20x as slow*    358.6ms +/- 12.7%     789.6ms +/- 20.4%     significant
        nbody:             *2.01x as slow*    189.4ms +/- 12.0%     380.6ms +/- 18.6%     significant
        nsieve:            *4.90x as slow*     93.6ms +/- 0.7%     458.6ms +/- 3.7%     significant
    
      bitops:              *2.88x as slow*    599.2ms +/- 2.9%    1725.4ms +/- 13.5%     significant
        3bit-bits-in-byte: *3.73x as slow*     93.6ms +/- 0.7%     349.2ms +/- 18.6%     significant
        bits-in-byte:      *3.38x as slow*    137.4ms +/- 6.3%     464.8ms +/- 18.9%     significant
        bitwise-and:       *1.73x as slow*    221.6ms +/- 4.2%     384.0ms +/- 13.2%     significant
        nsieve-bits:       *3.60x as slow*    146.6ms +/- 7.3%     527.4ms +/- 9.9%     significant
    
      controlflow:         *2.15x as slow*    127.8ms +/- 6.6%     274.6ms +/- 12.8%     significant
        recursive:         *2.15x as slow*    127.8ms +/- 6.6%     274.6ms +/- 12.8%     significant
    
      crypto:              *2.76x as slow*    367.0ms +/- 7.3%    1013.6ms +/- 9.7%     significant
        aes:               *2.25x as slow*    124.8ms +/- 10.9%     280.8ms +/- 13.0%     significant
        md5:               *3.21x as slow*    117.4ms +/- 19.3%     377.2ms +/- 9.1%     significant
        sha1:              *2.85x as slow*    124.8ms +/- 0.4%     355.6ms +/- 21.6%     significant
    
      date:                *1.59x as slow*    451.0ms +/- 5.0%     717.8ms +/- 16.6%     significant
        format-tofte:      *1.68x as slow*    189.0ms +/- 2.6%     318.4ms +/- 15.9%     significant
        format-xparb:      *1.52x as slow*    262.0ms +/- 8.1%     399.4ms +/- 18.3%     significant
    
      math:                *2.04x as slow*    621.0ms +/- 5.6%    1269.6ms +/- 13.8%     significant
        cordic:            *2.81x as slow*    237.2ms +/- 3.8%     667.4ms +/- 23.1%     significant
        partial-sums:      *1.24x as slow*    262.2ms +/- 13.4%     324.4ms +/- 16.7%     significant
        spectral-norm:     *2.28x as slow*    121.6ms +/- 6.6%     277.8ms +/- 15.9%     significant
    
      regexp:              *1.24x as slow*    332.4ms +/- 16.5%     412.0ms +/- 26.0%     significant
        dna:               *1.24x as slow*    332.4ms +/- 16.5%     412.0ms +/- 26.0%     significant
    
      string:              *1.54x as slow*    969.4ms +/- 2.1%    1491.8ms +/- 5.2%     significant
        base64:            *2.19x as slow*    152.8ms +/- 5.8%     334.0ms +/- 5.2%     significant
        fasta:             *1.46x as slow*    246.4ms +/- 3.5%     358.8ms +/- 17.8%     significant
        tagcloud:          *1.42x as slow*    183.0ms +/- 13.1%     259.0ms +/- 15.5%     significant
        unpack-code:       *1.41x as slow*    184.2ms +/- 4.6%     259.0ms +/- 6.7%     significant
        validate-input:    *1.38x as slow*    203.0ms +/- 6.9%     281.0ms +/- 20.1%     significant
    
    Opera 9.50 Weekly 9745:
    ============================================
    RESULTS (means and 95% confidence intervals)
    --------------------------------------------
    Total:                     NaNms +/- NaN%
    --------------------------------------------
    
      3d:                   1038.4ms +/- 27.2%
        cube:                308.8ms +/- 40.7%
        morph:               324.2ms +/- 11.4%
        raytrace:            405.4ms +/- 61.9%
    
      access:               1301.6ms +/- 19.0%
        binary-trees:        274.8ms +/- 72.6%
        fannkuch:            446.2ms +/- 19.0%
        nbody:               268.4ms +/- 13.1%
        nsieve:              312.2ms +/- 59.0%
    
      bitops:                986.2ms +/- 18.1%
        3bit-bits-in-byte:    90.6ms +/- 9.7%
        bits-in-byte:        137.2ms +/- 6.7%
        bitwise-and:         474.4ms +/- 16.5%
        nsieve-bits:         284.0ms +/- 35.8%
    
      controlflow:            90.4ms +/- 9.6%
        recursive:            90.4ms +/- 9.6%
    
      crypto:                  NaNms +/- NaN%
        aes:                 365.0ms +/- 64.4%
        md5:                   NaNms +/- NaN%
        sha1:                171.6ms +/- 47.2%
    
      date:                    NaNms +/- NaN%
        format-tofte:          NaNms +/- NaN%
        format-xparb:        686.4ms +/- 26.7%
    
      math:                    NaNms +/- NaN%
        cordic:              265.4ms +/- 20.8%
        partial-sums:          NaNms +/- NaN%
        spectral-norm:       124.6ms +/- 0.5%
    
      regexp:                951.6ms +/- 17.6%
        dna:                 951.6ms +/- 17.6%
    
      string:               8261.4ms +/- 13.6%
        base64:              889.0ms +/- 10.5%
        fasta:              1978.0ms +/- 11.9%
        tagcloud:            764.4ms +/- 25.2%
        unpack-code:        4031.2ms +/- 19.6%
        validate-input:      598.8ms +/- 54.1%