Windows 10 on ARM extensively benchmarked natively and with x86 emulation

Microsoft is currently in the process of launching a new platform, Always Connected PCs, which rely largely on devices running Windows 10 on ARM-based Snapdragon processors from Qualcomm.

With Windows 10 on ARM Microsoft is trying to negate the main disadvantages of Windows RT by offering emulation of x86 processors for regular Windows apps, thereby freeing users from the confines of the Microsoft Store.

Concerns, however, remain around running desktop apps and operating systems on what is still a mobile processor, especially around compatibility and performance.

Laptop site Techspot has now posted extensive benchmarks for the HP Envy X2, which runs the Snapdragon 835 processor and SOC, like all the other new ARM Windows 10 devices, for both emulated x86 code and native ARM code.

Firstly, however, the issue of compatibility with benchmarking software came up, which is somewhat of a warning sign.

Techspot writes:

With Windows on ARM, the limitations of the platform mean a large chunk of these benchmarks don’t work. Some didn’t work as they were 64-bit only, and others didn’t work for unknown reasons. Several benchmarks that I normally run as a 64-bit app I needed to redownload as 32-bit to get working, and sometimes those still didn’t work.

PCMark 8 works but the Creative test crashes and the Work test takes so long it’s pointless to run. PCMark 10 launches but the standard test is not supported. Cinebench R15 is 64-bit only and doesn’t run. Premiere is also a 64-bit only app these days. Blender has a 32-bit version but requires OpenGL 2.1 so it doesn’t work. MATLAB stopped providing a 32-bit version recently, but older x86 versions don’t work. And Sandra doesn’t work as I believe it needs to use an x86 driver. That’s eight benchmarks that don’t work, while nine did work, so about half.

When it came to the tests itself, for emulated x86 benchmarks the $1000 HP Envy X2 did not cover itself in glory.

As can be seen from the results above, performance for emulated code is not great. Techspot writes:

In fact doing any sort of rendering is very slow on the Snapdragon 835. The (Atom-based 2016 Celeron) N3450 slaughters Windows on ARM emulation in x264 and x265 rendering, and again the N3450 is one of the slowest Intel chips you can get.

The results are repeated with a number of other benchmarks with similar results e.g.

Starting off with PCMark 8’s Home test, and immediately it’s not good news for Windows on ARM’s emulated performance. One of the cheapest and slowest Intel chips you can get in a mobile form factor, the Goldmont-based Celeron N3450, is 51 percent faster here. 51 percent! The Core i5-5200U from three years ago is 182 percent faster, while the i7-7Y75, Intel’s current low-power Core offering, is 160 percent faster.

There were some tests where the Snapdragon 835 processor was competitive with the Atom-based processor but was still handily beaten by more modern Intel processors.

Techspot writes:

There are some benchmarks where Windows on ARM performance isn’t as embarrassing. Compression and decompression is solid provided there’s multi-threading, with the Snapdragon 835 beating the N3450 handily in WinRAR, and in 7-Zip decompression. Intel’s other CPUs are much faster but hey, at least Windows on ARM gets a small win here.

When it comes to native code the story is somewhat better, with the HP Envy X2 at least on par with the Celeron N3450.

Browser benchmarks when running on Edge (but not Chrome) give similar or slightly better results than the Celeron processor.

Techspot writes:

The results really highlight the difference between emulation and native code. While not a like-for-like as Edge and Chrome use different rendering engines, Chrome performance is terrible on the Envy x2 compared to Edge. In contrast, Chrome and Edge are roughly equivalent on the N3450, while Chrome is much faster on the Core i5-5200U. And yes, I’m using the old i5-5200U here as we know the latest Kaby Lake processors will destroy the Snapdragon 835.

Techspot found the battery life advantages were real, offering up to 50% better battery life when browsing compared to the best Intel laptop they tested, and in many cases much, much longer.

They however conclude that at the price point Microsoft and their partners are aiming for (between $600 and $1000) Windows 10 on ARM currently just does not offer enough to make it worthwhile.

They write:

My final thoughts on Windows on ARM are largely negative. Yeah, you get fantastic battery life, but in my opinion it’s not worth it when the performance you get ranges from disappointing to outright terrible. There aren’t enough ARM-optimized apps for not just enthusiasts, but typical users wanting to run Chrome and Office, and even in ARM-optimized apps, you’re not going to match or get close to the performance Intel offers with their low-power Core processors.

The final nail in the coffin is pricing. The HP Envy x2 is a $1,000 tablet. Battery life is fantastic at this price point, but the Snapdragon 835 struggles to outperform an Intel Celeron CPU typically found in sub-$300 notebooks, so it’s way too expensive. Other Windows on ARM devices are expected to sell for as low as $600, which is a more appropriate price point, but even then I don’t think it offers enough value over existing Intel options.

For example, right now you can purchase the entry-level Surface Pro with a Core m3 processor for $599, down from $799. Even with just a Core m3 processor, the Surface Pro is a much more capable device that’s way cheaper than the Envy x2 or similarly priced to other Windows on ARM devices. If you’re more after a laptop, a quick search on Amazon brings up mid-range laptops with a significantly more powerful Core i5-8250U for less than $600.

Read Techspot’s full report with all the benchmarks here.