Tuesday, July 13, 2010

Browser Benchmarks - July 8, 2010 - Ready. Set. Fight!

Update: More recent benchmark

Yesterday my Opera browser at work auto-updated to 10.60. Opera's auto update is finally coming in line with Firefox...still not quite as nice, but it's getting there. More interesting was the dev blog about 10.60, and I saw elsewhere that Firefox 4.0 beta 1 was out, and also checked up on IE9 to see how things were progressing. Somewhere along the way, I also found the Peacekeeper benchmark and decided I wanted to benchmark some browsers to see what the current state-of-play is. I will be using that for overall (HTML5 / DOM / Javascript) performance, and Sunspider for pure Javascript performance. I'm also throwing ACID3 in there for a sense of where each browser is standards-wise. 

System: Intel Core i5 750 @ 3.36GHz, 4GB RAM, 80GB Intel SSD (G1), ATI Radeon HD 4850
OS: Windows 7 64-bit Home Premium
Fresh boot. No other apps or system tray programs running aside from Microsoft Security Essentials.
Each browser was run by itself, with only one tab for the benchmark itself.

First off, I'll start with Sunspider. These are the final numbers, but I have linked the full results for each. I don't have any fancy graphs, so I will order them fastest to slowest.

  1. Chrome 5.0.375.99 - 224.0ms +/- 2.0% [Full Result]
  2. Opera 10.60 final - 231.4ms +/- 1.5% [Full Result]
  3. Safari - 273.4ms +/- 2.3% [Full Result]
  4. IE 9 Preview 3 (1.9.7874.6000) -  293.6ms +/- 0.7% [Full Result]
  5. Firefox 4.0 Beta 1 - 406.8ms +/- 1.9% [Full Result]
  6. Firefox 3.6.6 - 575.6ms +/- 1.1% [Full Result]
  7. IE 8 - 3555.8ms +/- 0.6% [Full Result]
Chrome has the lead on pure Javascript performance with its V8 Javascript engine, but Opera is not far behind. It's only trailing by 3.3%, barely more than the margin of error. Next up is Safari, taking 22% longer. IE9 Preview is showing great promise, at 31% behind Chrome. 

Firefox 4 beta is not in the same ballpark currently, at just more than half the speed of Chrome. It is a good increase over Firefox 3.6.6, which would've been almost a 3x difference, but it is a ways off. The big takeaway is that the IE9 team appears to now be ahead of the Firefox 4 team on Javascript performance...times are certainly changing.

IE8 is included as a baseline, and because a lot of users are still on IE.

Next up is Peacekeeper, for this I have screenshots and the full results here.

Opera 10.60 leads here overall. Chrome is close, at 16% lower score. It's worth noting just how far ahead Chrome and Opera are over every other browser in the overall category. Safari 5 is next, at less than half the score. Firefox 4b1 is on the heels of Safari, and then there are the rest.

What's interesting to note here is that even though Safari & Chrome are both webkit based browsers, clearly the Chrome team is going the extra mile on performance.

I want to drill into what makes up these scores, as this benchmark is new to me, but I think that will be in a follow-up post as this post is already almost a week late if you look at the date..

Finally, lets look at ACID 3.

  • Chrome 5.0.375.99 - 100/100
  • Opera 10.60 - 100/100
  • Safari 5  - 100/100
  • Firefox 4 Beta 1 -  97/100
  • Firefox 3.6.6 - 94/100
  • IE 9 Preview 3 -  83/100
  • IE8 20/100 (FAIL)

Chrome, Opera, and Safari all receive full marks. I did not compare pixel to pixel, but they have had good track records with ACID tests. Firefox 4 Beta 1 is getting close to passing, slightly better than Firefox 3.6.6. IE9 has come a long way from IE8 but still has a ways to go in order to pass ACID 3.

Regardless of your preferred browser, there is some intense competition now in Javascript engines and general rendering performance, resulting in the experience improving for everyone. It is an exciting time, and I don't think you can go wrong with Chrome, Firefox, Opera, or Safari. Rendering and performance are more than good enough in all of them, so it comes down to other usability / features. Once IE9 is closer to final, I may even be able to recommend that.

Update: More Recent Benchmark 

No comments:

Post a Comment