Web Browser Performance Comparison And Database
Web Benchmarks Explained And Results
The following tests are quite standard. SunSpider, Kraken and Octane cover the various javascript rendering engines. Acid and HTML5Test are compliance based metrics for standard web development practices. Finally, Peacekeeper throws various multimedia tasks at the browsers, including video playback, 2D and 3D rendering, as well as complex DOM handling.
SunSpider is currently at version 1.0.2, Kraken is version 1.1, and both results are measured in milliseconds. Octane is version 2.0 and comprises 17 tests, with a total score generated at the end. HTML5Test is a score based on the total number of supported elements found within the HTML5 specification, and is currently marked out of a possible 555.
Peacekeeper is now a discontinued benchmark from Futuremark, but still has many useful metrics as it includes multimedia testing, which the other benchmarks do not check.
Acid was mentioned as a test, but it is not recorded in the results. This is simply because it's a compliance test that's measured out of a total score of 100. Every single browser tested passed without issue, scoring the full 100 points.
Well, we can definitely say now that Microsoft has come a long way since the dreaded days of IE6. Ignoring Edge for a minute, it's clear that some of the underlining changes that Microsoft started to make a few months ago to Internet Explorer's core, are showing through. We touched on this briefly back when Edge was known as Spartan, so it's good to know that it retained some of these optimizations.
Both IE and Edge dominate SunSpider in the benchmarks, completing it in half the time compared to the other browsers. However, it's clear that Internet Explorer is a one-trick pony when it comes to performance, as every other test, it lags behind and badly. Regardless, Javascript processing goes to Microsoft.
What hasn't changed is compliance - Microsoft still has a long way to go in both IE and Edge when it comes to HTML5. It's improving, and most people won't notice any detrimental effects, but it still gives developers a headache.
Microsoft's Edge is certainly off to strong start with the Javascript benchmarks, however, its lack of open-standards support is what really lets it down. The reason for the poor Peacekeeper and HTML5 results, are linked to the same problem: multimedia. With no support for Theora and WebM (VP8 and VP9 codecs), Peacekeeper can't perform two of the video tests. With HTML5, it's the lack of supported video and 2D graphics rendering that knock off a fair few points, although there are plenty more issues as well.
Edge 20 (left), Internet Explorer 11 (right)
A closer look at the performance metrics, one thing that will be clear is that for CPU time, neither IE or Edge fully exploit the CPU during the tests. The main reason is because the spikes that can be seen with the other browsers are due to the extra video tests from Peacekeeper. Also, looking at the memory use for IE (green line), you'll notice that it has rather poor garbage collection too - it's not letting go of resources from memory.
Blink browsers Chrome, Opera and Vivaldi, all score extremely well with HTML5, with their only notable issues being related to audio/video codec support, microdata, and a few other odds and ends. They all did reasonably well with the peacekeeper test, losing out to Firefox due to the video render tests.
Vivaldi performs well with Octane, compared to the other Blink browsers, but at the same time, scores the second lowest score in Kraken. It's current alpha state is likely the main cause, which should be patched away in the coming months.
Left to right: Chrome 45, Opera 32 and Vivaldi Preview 4
Both Chrome and Opera have near identical performance charts, showing the fundamental similarities in implementation of the Blink engine. It's only Vivaldi that stands out, showing its lack of maturity with significantly higher process times than the other Blink browsers. It's still early days for the new browser though, so expect this to come down in time.
Firefox is not so much an underdog, as it is mediocre. Scoring very average results all around, although it holds the upper-hand when it comes to multimedia with Peacekeeper. FF Developer shows some performance issues with JS, but this could be as much to do with it being 'unstable' rather than the effects of the new core that's being built. Mozilla has stated that their new multiprocess system should not affect web pages, and is more for plugins. The good news from this is that you shouldn't see any performance hits as the new core is rolled out in the coming month.
Firefox 41 (Left) and Firefox Developer 43 (right)
In terms of performance characteristics, nothing really sticks out for the two Firefox browsers. The memory delta is a little bit higher compared to the other browsers, but is still within an acceptable range. It's only with more tabs and maybe a couple plugins enabled that a real comparison could be made.
Final Thoughts And Recommendations
If you are in a business environment that's rolling out Windows 10, and the only browsers you have access to are Edge and IE - go with Edge. It's the better browser of the two by far (security not withstanding at this point). If you do have a choice, then there are perhaps better options to consider, depending on your use case.
The performance differences between browser types are quite small currently. If you exclude IE, all the other browsers are within 10-20% of each other, in a give-or-take fashion. There is no dominant browser generally speaking. What it will come down to is features. As e10s rolls out with Firefox, even Chrome extensions may soon be compatible with FF.
For web standards like HTML5, Blink browsers (Chrome, Opera and Vivaldi) still have the upper-hand, even beating the rather vocal and former web-standards champion, Mozilla.
Moving forward in the coming months, we will be revisiting these tests, providing a health-check for each browser. Will Microsoft start putting some effort into supporting web standards? Who knows. At least Redmond on the right track with performance.
SunSpider is currently at version 1.0.2, Kraken is version 1.1, and both results are measured in milliseconds. Octane is version 2.0 and comprises 17 tests, with a total score generated at the end. HTML5Test is a score based on the total number of supported elements found within the HTML5 specification, and is currently marked out of a possible 555.
Peacekeeper is now a discontinued benchmark from Futuremark, but still has many useful metrics as it includes multimedia testing, which the other benchmarks do not check.
Acid was mentioned as a test, but it is not recorded in the results. This is simply because it's a compliance test that's measured out of a total score of 100. Every single browser tested passed without issue, scoring the full 100 points.
Both IE and Edge dominate SunSpider in the benchmarks, completing it in half the time compared to the other browsers. However, it's clear that Internet Explorer is a one-trick pony when it comes to performance, as every other test, it lags behind and badly. Regardless, Javascript processing goes to Microsoft.
What hasn't changed is compliance - Microsoft still has a long way to go in both IE and Edge when it comes to HTML5. It's improving, and most people won't notice any detrimental effects, but it still gives developers a headache.
Microsoft's Edge is certainly off to strong start with the Javascript benchmarks, however, its lack of open-standards support is what really lets it down. The reason for the poor Peacekeeper and HTML5 results, are linked to the same problem: multimedia. With no support for Theora and WebM (VP8 and VP9 codecs), Peacekeeper can't perform two of the video tests. With HTML5, it's the lack of supported video and 2D graphics rendering that knock off a fair few points, although there are plenty more issues as well.
Edge 20 (left), Internet Explorer 11 (right)
A closer look at the performance metrics, one thing that will be clear is that for CPU time, neither IE or Edge fully exploit the CPU during the tests. The main reason is because the spikes that can be seen with the other browsers are due to the extra video tests from Peacekeeper. Also, looking at the memory use for IE (green line), you'll notice that it has rather poor garbage collection too - it's not letting go of resources from memory.
Blink browsers Chrome, Opera and Vivaldi, all score extremely well with HTML5, with their only notable issues being related to audio/video codec support, microdata, and a few other odds and ends. They all did reasonably well with the peacekeeper test, losing out to Firefox due to the video render tests.
Vivaldi performs well with Octane, compared to the other Blink browsers, but at the same time, scores the second lowest score in Kraken. It's current alpha state is likely the main cause, which should be patched away in the coming months.
Left to right: Chrome 45, Opera 32 and Vivaldi Preview 4
Both Chrome and Opera have near identical performance charts, showing the fundamental similarities in implementation of the Blink engine. It's only Vivaldi that stands out, showing its lack of maturity with significantly higher process times than the other Blink browsers. It's still early days for the new browser though, so expect this to come down in time.
Firefox is not so much an underdog, as it is mediocre. Scoring very average results all around, although it holds the upper-hand when it comes to multimedia with Peacekeeper. FF Developer shows some performance issues with JS, but this could be as much to do with it being 'unstable' rather than the effects of the new core that's being built. Mozilla has stated that their new multiprocess system should not affect web pages, and is more for plugins. The good news from this is that you shouldn't see any performance hits as the new core is rolled out in the coming month.
Firefox 41 (Left) and Firefox Developer 43 (right)
In terms of performance characteristics, nothing really sticks out for the two Firefox browsers. The memory delta is a little bit higher compared to the other browsers, but is still within an acceptable range. It's only with more tabs and maybe a couple plugins enabled that a real comparison could be made.
Final Thoughts And Recommendations
If you are in a business environment that's rolling out Windows 10, and the only browsers you have access to are Edge and IE - go with Edge. It's the better browser of the two by far (security not withstanding at this point). If you do have a choice, then there are perhaps better options to consider, depending on your use case.
The performance differences between browser types are quite small currently. If you exclude IE, all the other browsers are within 10-20% of each other, in a give-or-take fashion. There is no dominant browser generally speaking. What it will come down to is features. As e10s rolls out with Firefox, even Chrome extensions may soon be compatible with FF.
For web standards like HTML5, Blink browsers (Chrome, Opera and Vivaldi) still have the upper-hand, even beating the rather vocal and former web-standards champion, Mozilla.
Moving forward in the coming months, we will be revisiting these tests, providing a health-check for each browser. Will Microsoft start putting some effort into supporting web standards? Who knows. At least Redmond on the right track with performance.
