Download complete report

Why is it that the arrival of iOS 7 is necessarily a momentous event for the smartphone market? Simple: Unlike any other operating system out there, it will be in the hands of millions or tens of millions of users within a few days after its launch.

And that will make it a force to be reckoned with.

The question is, of course:

How good is it really?

Introducing: The 2013 Mobile OS User Experience Benchmarks

Whether we like it or not, smartphones have become a software game. Take any recent top-of-the-line smartphone, and you are likely to get a well-designed, fast, pleasant to use bit of hardware: fluid operation, responsive interaction, fast graphics. The difference of user experience, therefore, stems almost exclusively from the operating system, the user interface design, the application integration, the overall coherence.

This research project compares the five major mobile operating systems in use today: iOS 7, iOS 6, Android, Windows Phone 8, and Blackberry 10, and rates them in terms of user experience.

We do not look at features, we do not compare cutting-edge options and gadgets, we only look at aspects that have a direct impact on the day-to-day user experience of an average, non-technical user.

The aspects we have surveyed and rated are the following:
cognitive load, efficiency, customization, as  well as user experience friction.Based on the results from these benchmarks we have then established an overall Mobile Operating System User Experience Index presented at the end of this document.

The benchmarks are based on the Pfeiffer Consulting Methodology for User Experience Quantification. You can find detailed information on the subject by following this link.

Key Results

The combined results of the four different benchmarks and evaluations give iOS 7 a clear advantage in terms of of overall user experience, taking into account the context defined for these benchmarks: 
day-to-day user experience of an average, non-technical user. For details and methodology please refer to the complete report.

Individual Benchmark Results — Cognitive Load

Cognitive load—the sum of elements you need to get familiar with in order to use a device spontaneously and intuitively—is one of the key aspects of user experience for a non-technical user. For this benchmark we counted the number of apps/widgets as well as other icons and user interface elements a default installation of the operating system contains. Differences between operating systems are significant. For a detailed  comparison of cognitive load elements of mobile operating systems please check the full report.

Individual Benchmark Results — Efficiency and Integration

Smartphone user experience depends in large part on the easy and efficient integration of different key features and services.

For this benchmark we analyzed access to key settings, integration with notifications, multitasking, and camera access, among others. While there is a core set of integration functionality all mobile operating systems share, there are significant and surprising differences between the five major players in the market.

Individual Benchmark Results — Customization

Consumer-level customization is one of the key user experience aspects of connected digital devices. Nevertheless, the current crop of mobile operating systems differs quite significantly in terms of customization, ranging from the almost dizzying granularity of Android’s options, to the starkness of Windows Phone 8, which allows hardly any user-level customization.

Individual Benchmark Results — User Experience Friction (UXF)

UXF is the bad stuff, the aspects of a device that can annoy you in a niggling way, or, in extreme cases, drive you crazy. Basically, UXF occurs whenever a device does not do what you expect it to do – or lacks a key feature that should be available. For this survey we took only the most obvious UXF factors into account, and rated them. For a detailed account of the UXF factors, as well as the rating method, see the full report.

Download complete report