Short version
On my machine given my unscientific measurements in Q2 2009:
V8 is really, really fast.
SquirrelFish is really fast, and often close to really, really fast.
TraceMonkey is fast, but often far behind really fast.
Longer version
If you haven't been living under a rock the last year then you know that there has been substantial progress in the JavaScript VM camps. I wanted to get some performance numbers from recent versions of V8, TraceMonkey and SquirrelFish.
Here's what I used (I'm on Mac OS X). It was the most recent versions at the time of writing (May 22nd 2009).
V8: Chromium 3.0.182 (182.0)
TraceMonkey: Firefox 3.5b4
SquirrelFish (extreme): Safari 4 Public Beta (5528.17)
To get somewhat fair performance figures I ran three benchmark suites: Dromaeo, V8 Benchmark Suite and SunSpider. I ran the JavaScript Tests part of Dromaeo since I intended to measure JavaScript, not DOM, performance.
Here are the very unscientific results from my Core 2 Duo laptop. I re-ran some of the tests a few times, best result posted. The browser was restarted between each test run.
- Dromaeo JavaScript Tests (higher is better):
- V8: 176 runs/s
- SquirrelFish: 138 runs/s
- TraceMonkey: 46 runs/s
- V8: 176 runs/s
- V8 Benchmark Suite version 4 (higher is better):
- V8: 2325
- SquirrelFish: 1440
- TraceMonkey: 189
- SunSpider 0.9 (lower is better):
- V8: 865ms
- SquirrelFish: 1022ms
- TraceMonkey:1480ms
For the SunSpider bars lower is better, for the others higher is better. I may update this blog post with new numbers if I get better results (with the same versions). V8 is the fastest, SquirrelFish is second and TraceMonkey comes in last in all benchmarks. edit: These numbers are unfair since V8 was a nightly build (no stable Chrome released yet for Mac OS X) while the others were a few months old. Check out the less unfair numbers in the newer "V8 and JavaScriptCore are really, really fast" post instead.
All VM's have their own merits. Nothing beats getting real-world performance characteristics by throwing your own programs at them so do that if you want to do a serious evaluation of them. You may also be interested in evaluating code size and memory consumption, especially if you're into embedding. Oh and in that case, make sure that the JIT has a backend for your target.
To me the V8 source code was the most beautiful and TraceMonkey the least. This is very subjective so take a look for yourself. And while you're checking out the code run the log command so that you get a feeling of how fast the project is moving. And what they're changing. That may be a good or bad thing to you.
If you just want to run a fast browser then all three should be plenty fast for most (current) use cases. I run Mozilla Firefox myself since I'm addicted to extensions such as LastPass and Tree Style Tab.
THE POST IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE POST OR THE USE OR OTHER DEALINGS IN THE POST.
7 comments:
If you're going to testing a bleeding edge version of V8 then its a little strange to use such an old version of JavaScriptCore (one from February). In my testing on Mac OS X, a recent nightly build of JavaScriptCore beats V8 on SunSpider, Dromaeo, and v3 of the V8 benchmark. V8 is slightly ahead on v4 of their benchmark, but that's not too surprising since it contains a brand new sub-test that they've clearly targeted in their optimization work before making the benchmark available. I'm sure that gap will disappear in the near future.
As long as we're posting unscientific benchmarks, here's one I ran on my computer and found *very* interesting. I basically took the code from http://www.smallshire.org.uk/sufficientlysmall/2009/05/22/ironpython-2-0-and-jython-2-5-performance-compared-to-python-2-5/ and wrote a Clojure version. Slava Pestov, author of Factor, a very fast and interesting dynamic language, posted a Factor version and I found a JavaScript implementation that uses K7 (which is based on V8). Here are the results:
/tmp $ time python silly-bench.py 22
4194303
524.912 secs
/tmp $ time clj silly-bench.clj 22
4194303
13.947 secs
/tmp $ time ~/src/factor/factor silly-bench.factor 22
4194303
4.627 secs
/tmp $ time ~/src/k7/k7 silly-bench.js 22
4194303
1.874 secs
As you can see, Python is *way* behind every other implementation; Clojure performed better than I initially thought it would, though I guess there's a lot of room for improvement; Factor is compiled to native code and Slava is very attentive to performance, so it wasn't a surprise to see it perform that well. The real shocker for me was JavaScript with V8: it's almost 300x faster than the Python version! If JavaScript were to gain a standard library similar to Python's, it would probably take the world by storm outside the browser!
For a fair comparison, you should compare to a WebKit nightly. You may also want to try v3 of the v8 benchmark and wonder why the results change so much in the v8 engine's favor with v4 of the benchmark...
Thanks for the feedback Mark. I've put this information in the post and published unscientific benchmark numbers from nightly builds in a new one.
I'm sure that you'll be able to catch up on the Splay benchmark as well. For now I think it suffices to say that both V8 and JavaScriptCore are really, really fast. V8 is faster in some tests and JavaScriptCore in others. Keep up the awesome work and keep the friendly competition!
If you're going to use the latest nightly Chromium, why didn't you use the latest nightlies for Firefox or WebKit? These results are very biased.
Eli - check out the edit: section on the top of this post.
@Vincent: there's actually work afoot to give JS a standard library akin to Python's, and we're making good progress:
https://wiki.mozilla.org/ServerJS
Post a Comment