Press releases from October 2016


New Android benchmark tests available now

October 13, 2016


PCMark for Android benchmark for smart phones and tablets

Today we are releasing a major update for our PCMark for Android benchmarking app. The update adds two new benchmark tests based on real-world activities and an exciting new way to compare the performance of devices across Android OS version updates.

The best way to benchmark smartphones and tablets is to look at the performance of the device as a whole. PCMark for Android uses tests based on everyday activities. By using real applications and native APIs, scores from PCMark for Android better reflect real-world performance and battery life.

Alex Voica, Senior Marketing Specialist at Imagination Technologies, says, "PCMark is actually a solid example of mobile benchmarking done right."

NEW - Work 2.0 performance benchmark and battery life test

PCMark for Android Work 2.0 benchmark and battery life test

See how your device handles common productivity tasks - browsing the web, editing videos, working with documents and data, and editing photos.

PCMark for Android Work 2.0 benchmark test

Work 2.0 is an improved version of the older Work benchmark. It updates the Web Browsing, Writing, and Photo Editing tests and adds two all-new tests for Video Editing and Data Manipulation. You'll find more details in our earlier post and in the PCMark for Android technical guide.

Use the Work 2.0 benchmark to test the performance and battery life of the latest Android devices. And while we now recommend using Work 2.0, the original Work test is still available in the app. Scores from Work 2.0 are not comparable with the older Work test.

NEW - Computer Vision benchmark

PCMark for Android Computer Vision benchmark

Computer Vision is an exciting field with opportunities for a range of innovative apps and services. The PCMark for Android Computer Vision benchmark measures device performance for a range of image recognition tasks using a number of popular open-source libraries.

TensorFlow is an open-source machine learning library developed by Google. The test uses a pre-trained neural network to recognize different objects in a set of pictures.

ZXing, commonly known as Zebra Crossing, is a multi-format barcode image processing library. The test uses ZXing to read a set of barcodes and QR codes.

Tesseract is an open-source optical character recognition library. The test recognizes and extracts English text from a set of images.

You'll find more information for each test in our previous post and in the PCMark for Android technical guide.

PCMark for Android Work 2.0 benchmark test

NEW - Compare performance across OS versions

With this update we are making it easier to see how major Android OS updates affect a device’s performance.

The in-app Best Devices list is the ideal way to compare your device with the latest smartphones and tablets. The update adds the ability to filter the scores by major Android OS version number. For more details, read our post on comparing smartphone performance across Android OS versions.

PCMark for Android comparing smartphone performane across Android OS versions

A benefit of using a benchmark based on real-world tasks is that you can see the areas where the OS update has the biggest impact. There could be big improvements in some types of activities due to new drivers, OS optimisations or changes to APIs. In other areas, performance might stay the same or even degrade slightly.

What’s new in PCMark for Android v2.0.3705

  • Test performance and battery life with the Work 2.0 benchmark.
  • Measure performance with the Computer Vision benchmark.
  • Compare device performance across Android OS versions.

Download

PCMark for Android is a free benchmark app available from Google Play.

Download PCMark for Android benchmark from Google Play



Comparing smartphone performance across Android OS versions

October 12, 2016


PCMark for Android benchmark for smart phones and tablets

PCMark for Android is getting a major update tomorrow. We've already taken a closer look at the new Work 2.0 and Computer Vision benchmarks. In this post, we’ll present another new feature, the ability to compare scores across Android OS versions.

Android OS updates can breathe new life into older devices. New features and interface design changes can give a device a fresh, modern feel. While beneath the surface, driver optimizations and changes to the operating system can improve performance, stability, and battery life.

With this update, we're making it easier to see how Android OS updates affect a device's performance.

PCMark for Android comparing smartphone performance across Android OS versions

LG Nexus 5 and Samsung Galaxy S5 performance across three Android OS generations

In this post, we'll be digging into our database to take a detailed look at how the performance of two of the most popular smartphones has changed across three Android generations.

We looked into the PCMark for Android Work benchmark scores for the LG Nexus 5 and Samsung Galaxy S5 for Android versions 4.x, 5.x and 6.x.

LG Nexus 5 PCMark for Android Work performance by Android OS version

Samsung Galaxy S5 PCMark for Android Work performance by Android OS version

As you can see, while the general trend is for performance to increase with each new OS version, the change varies by device and activity. In fact, for some tests you can see that performance decreased slightly after the update. It's worth remembering that some Android updates focus more on adding new features and functionality than performance.

In these charts, each score is the average of all the benchmarks results received for that model, with obvious outliers excluded. For a popular device, the average comes from thousands of individual benchmark results from users around the world.

For some devices, these individual benchmark scores can vary a lot. People often run benchmarks under less than ideal conditions, for example, with other apps running in the background, with a device that is already running hot or on a device that is poorly configured.

The charts below show the distribution of individual benchmark results for each model. The peak shows the most common score range.

LG Nexus 5 PCMark for Android Work performance distribution by Android OS version

For the LG Nexus 5, you can see that each Android OS update pushes the curve towards higher scores and better performance.

Samsung Galaxy S5 PCMark for Android Work performance distribution by Android OS version

With the Galaxy S5, we see a much larger jump in performance from Android 4.x to 5.x. There is a slight drop in scores moving to Android 6.x, though this is based on limited data since the Android 6.0 update was only rolled out to Samsung S5 devices very recently.

Next, we look at the performance for specific scenarios in the Work benchmark test, such as web browsing, video playback, writing and photo editing.


LG Nexus 5 PCMark for Android Web Browsing performance distribution by Android OS version

LG Nexus 5 PCMark for Android Writing performance distribution by Android OS version

LG Nexus 5 PCMark for Android Video Playback performance distribution by Android OS version

LG Nexus 5 PCMark for Android Photo Editing performance distribution by Android OS version

Samsung Galaxy S5 PCMark for Android Web Browsing performance distribution by Android OS version

Samsung Galaxy S5 PCMark for Android Writing performance distribution by Android OS version

Samsung Galaxy S5 PCMark for Android Video Playback performance distribution by Android OS version

Samsung Galaxy S5 PCMark for Android Photo Editing performance distribution by Android OS version

It's clear from these charts that one should not generalize when talking about the performance benefits of Android OS updates. The effects can vary greatly by device and by activity. 

OS updates can make wide-reaching changes that affect the whole device. People assume that these updates will bring better performance, but we can see that this is not always the case for every activity. It is easy to misinterpret the benefits of an update if you rely on synthetic tests that only measure CPU performance in isolation.

By using a benchmark based on real-world tasks, you can see the areas where an OS update has the biggest impact. For some activities, there might be big improvements as a result of new drivers, OS optimizations or changes to APIs. In other areas, the performance might stay the same or even degrade.

PCMark for Android is the only benchmark app that can give you these insights into a device's real-world performance. And it will get even better tomorrow when we introduce our new Work 2.0 and Computer Vision benchmark tests.

Compare performance across Android OS versions

The in-app Best Devices screen is the ideal way to compare the latest smartphones and tablets. See which devices top the rankings for performance and battery life. This update gives you the option to filter the Best Devices list by a specific Android OS version. By default, the list shows scores from devices with the same Android OS version as your own device.

  1. Swipe left to the Best Devices screen.
  2. Tap the FILTERS button.
  3. Make a selection from the Test dropdown.
  4. Make a selection from the Android OS version dropdown.

PCMark for Android performance comparison by Android OS version in the Best Devices List

It's easy to compare scores across Android OS versions for your own device.

  1. Swipe left to the Best Devices screen.
  2. Tap the FILTERS button.
  3. Use the Search box to find your device or another model you're considering.
  4. Tap on the device to open the Device Details view. Landscape mode works best for this view.
  5. Scroll down, then select a benchmark test and Android version to compare scores.

PCMark for Android comparing smartphone performane across Android OS versions

Use this feature to see how scores have changed with each OS version upgrade, assess the difference in performance, and identify the areas with significant and modest performance changes.

You can choose any model from the Best Devices list and see a side-by-side comparison with your own device. Areas where the device is superior are highlighted green. Areas where it is inferior are shaded red.

Coming soon

PCMark for Android is a free benchmark app available now from Google Play. You'll be able to compare scores across Android versions after the update is released at 10:00 UTC on Thursday, October 13.

Download PCMark for Android benchmark from Google Play



New Android benchmark tests - Computer Vision in focus

October 10, 2016


PCMark for Android benchmark for smart phones and tablets

Our popular smartphone benchmarking app, PCMark for Android, is getting a major update this week. On October 13, we'll add two new benchmark tests and a neat new way to compare devices across Android OS versions. In this post, we'll be focusing on the new Computer Vision benchmark test. (Make sure you check out our post on the new Work 2.0 benchmark as well.)

Computer Vision is an exciting field at the cutting edge of computer science. Recent advances mean that some computer vision techniques are now possible on mobile devices, opening the door for a host of innovative new apps and services.

We couldn't be more proud to introduce the PCMark for Android Computer Vision benchmark for smartphones and tablets. This new test measures the performance of your device using tests based on popular open-source image recognition and machine learning libraries.

PCMark for Android Computer Vision test

Image recognition with TensorFlow

In this test, a pre-trained neural network tries to identify the objects in a set of photos. This technique has many practical uses in mobile applications such as identifying and classifying images in photography apps, tagging people and places in social networking apps, and helping visually impaired people understand the world around them.

The test uses a TensorFlow Inception model that has been trained with the ImageNet database. There's a lot of jargon here, so let's break it down. TensorFlow is an open-source machine learning library developed and supported by Google. Inception is a deep convolutional neural network architecture developed by Google for TensorFlow that is part of the current state of the art for computer image recognition. ImageNet is an academic data set containing thousands of images that is commonly used for training image recognition systems.

The model is loaded into the memory of your device and tested with a set of images. The test measures the time the model takes to classify each image.

Graphical barcode scanning with ZXing

There are dozens of barcode and QR code scanning apps. These helpful apps cut out the need to type long serial numbers and website addresses.

This test uses ZXing, an open-source, multi-format barcode image processing library, to read a set of barcodes and QR codes. To make the test a better representation of real-world use, the codes have been photographed under poor lighting condition and some have simulated tearing damage. The test measures the time taken to recognize each barcode and QR code.

Optical character recognition with Tesseract

Optical character recognition (OCR) is a technique for recognizing and extracting text from images. OCR has many practical and useful applications on mobile devices, from augmented reality translation apps like Google Translate, to document scanners and business card readers.

Tesseract is an open-source optical character recognition library. This test uses a fork of Tesseract Tools for Android called tess-two. The test uses tess-two to recognize and extract English text from a set of images. The images tested have been photographed in poor lighting conditions to simulate the challenges of OCR in the real-world. The test measures the time taken to recognize and extract the text from each image.

Computer Vision benchmark score

PCMark for Android Computer Vision test result

After running the Computer Vision benchmark, you get an overall score and a result for each subtest. You will also see a detailed hardware monitoring chart that shows how your device's CPU clock speed, temperature and battery level changed during the test.

See how your device compares with the latest smartphones and tablets in the in-app Best Devices List. Tap any device to compare it side-by-side with your own device. You can also filter the list by searching for a specific model, brand, CPU, GPU or SoC.

Coming soon

PCMark for Android is a free benchmark app available now from Google Play. The Computer Vision test will be available at 10:00 UTC on Thursday, October 13.

Download PCMark for Android benchmark from Google Play



What's new in the Work 2.0 Android benchmark

October 7, 2016


PCMark for Android benchmark for smart phones and tablets

Next week, on October 13, we will release a major update for our PCMark for Android benchmarking app. The update will add two new benchmark tests and an exciting new way to compare devices across Android OS versions. In this post, we'll be looking at the new and improved Work 2.0 benchmark and battery life test. 

Each new generation of mobile devices is more powerful than the last. With the latest smartphones and tablets we can perform more tasks at home and work without needing a notebook or desktop PC. 

The new Work 2.0 benchmark shows you how well your device handles common productivity tasks. It benchmarks performance and battery life with tests based on real applications using native APIs. Work 2.0 is an improved version of the original Work benchmark. It updates the Web Browsing, Writing, and Photo Editing tests and adds two all-new tests for Video Editing and Data Manipulation.

Video Editing test

PCMark for Android Work 2.0 Video Editing test

Combine a good-quality camera with decent processing power and your smartphone can act as a mobile movie studio. Mobile video editing apps help you trim and reorder clips, apply filters, and tweak the contrast and tone, before sharing your videos online.

The Video Editing test measures your device’s performance when playing, editing and saving video. The first part of the test measures how well your device performs when applying real-time effects to a number of video clips covering a range of common resolutions and frame rates. The second part of the test measures performance when decoding, editing, encoding and muxing a video.

The test uses OpenGL ES 2.0, the native Android MediaCodec API, and ExoPlayer, a Google-developed, open-source media player that improves on the native Android MediaPlayer API.

Data Manipulation test

PCMark for Android Work 2.0 Data Manipulation test

There are now hundreds of apps that help us quantify our personal lives and work performance. We turn to fitness apps to analyse our workouts and monitor our health. We use personal finance apps to track our bank accounts, credit cards, and stock markets.

The new Data Manipulation test measures the time taken to parse data from various file formats, then records the frame rate while interacting with dynamic charts in real-time. Each test uses 10,000 tuples of data to create bar, line and pie charts, which are then animated in response to common gestures such as swipes and zoom.

Web Browsing 2.0 test

Did you know that more people now go online with smartphones than with PCs? Web Browsing 2.0 measures how well your device performs when displaying and interacting with web content. It tests performance when rendering a web page, scrolling, zooming, searching for content, and re-rendering the page after editing and adding an item.

Web Browsing 2.0 improves on the original Web Browsing test by measuring frame rate as well as the time taken to complete the tasks.

Writing 2.0 test

Tablets and smartphones with larger screens let you carry on working even when you're away from the office. The Writing 2.0 test measures how well your device performs when working with documents and text.

The Writing 2.0 test measures performance while adding text and images to a document and then cutting, copying, and pasting text within the document. Writing 2.0 expands on the original test by additionally measuring performance while creating, displaying, encrypting, and decrypting a PDF document.

Photo Editing 2.0 test

"The best camera is the one you have with you," says award-winning photographer, Chase Jarvis. No wonder more photos are taken on smartphones than cameras these days. Photography apps help us crop, fix, and filter our photos before we share them with friends.

The Photo Editing 2.0 test measures how well your device performs when applying various filters and effects to images. It measures the time taken to open, edit and save a set of images. The measurements include the time taken to move graphics data to and from the CPU and GPU, decode from and encode to JPEG format, and to read and write from the device’s storage.

Photo Editing 2.0 improves on the original test by using the latest android.renderscript API in place of the older android.support.v8.renderscript API that was used previously.

Work 2.0 battery life test

For many people, knowing that a phone will last through the day is the deciding factor when choosing a new model. After all, what use is great performance if it means you're always tethered to a charger?

Unfortunately, some manufacturers set battery life expectations too high by quoting figures based on unrealistic use cases. Measuring battery life with the real-world applications and activities in the Work 2.0 benchmark gives a much more realistic view of how long a device will last in everyday use.

"[PCMark is] one of my favorite battery benchmarks because it mixes together various different tasks, stresses different parts of the hardware, and in general gives a very good idea of how battery life will be when the user is doing many things with their smartphone." - Brandon Chester, Smartphone Editor at AnandTech

Work 2.0 benchmark score

PCMark for Android Work 2.0 benchmark results

After running the Work 2.0 benchmark, you get an overall score and results from each of the subtests. Use these scores to compare your device with other smartphones or tablets. Detailed hardware monitoring charts show you how the CPU clock speed, temperature and battery charge level changed during the benchmark run. You can see how CPU scaling and thermal management affect your device's performance and battery life.

After running the Work 2.0 battery life test, you get the battery life result, an overall Work 2.0 performance score, and a chart showing the performance in each loop of the test. The chart makes it easy to see if the performance changed over time due to thermal issues or other factors.

The in-app Best Devices list is the ideal way to compare the performance, popularity and battery life of the latest Android smartphones and tablets. Tap any device to see a side-by-side comparison with your own device, or search for a specific model, brand, CPU, GPU or SoC.

Coming soon

PCMark for Android is a free benchmark app available now from Google Play. The Work 2.0 update will be available at 10:00 UTC on Thursday, October 13.

Download PCMark for Android benchmark from Google Play



New Android benchmark tests coming on October 13

October 5, 2016


PCMark for Android benchmark app

When we released PCMark for Android two years ago, we introduced a fresh approach to benchmarking smartphones and tablets. We focussed on testing the whole device with tests based on everyday activities like browsing websites, watching videos, working with text, and editing photos. 

A modern smartphone is a complex system of interconnected parts. Everyday activities like catching up on social media, writing an email or taking a photo place different demands on the CPU, GPU, memory, and storage. Narrow synthetic tests that only stress a single component in isolation cannot provide a realistic view of these real-world demands. Only by looking at the device as a complete system can you measure its real-world performance.

"[PCMark] tends to test every aspect of a mobile device, unlike microbenchmarks that can often miss aspects of the system that can affect performance." - Joshua Ho, AnandTech Senior Editor

On October 13, we'll release a major update for PCMark for Android. The update will add two new benchmark tests and an exciting new way to compare devices across Android OS versions. 

We'll be sharing more details about these new tests and features over the next few days. For now, here's an introduction to what's coming.

Work 2.0 performance benchmark and battery life test

PCMark for Android Work 2.0 benchmark and battery life test

Use the Work 2.0 benchmark to see how your device handles common productivity tasks  - browsing the web, editing videos, working with documents and data, and editing photos. Work 2.0 measures performance and battery life with tests based on real applications using native APIs.

Work 2.0 is an improved version of the original Work benchmark. It updates the Web Browsing, Writing, and Photo Editing tests and adds two all-new tests for Video Editing and Data Manipulation.

Computer Vision benchmark

PCMark for Android Computer Vision benchmark

Computer Vision is an exciting field with opportunities for a range of innovative mobile apps and services. The new Computer Vision benchmark uses popular open-source libraries to test your device's performance for image recognition, graphical barcode scanning, and optical character recognition tasks.

Compare performance across Android OS versions

The in-app Best Devices list is a great way to compare your device with the latest smartphones and tablets. The update will add the ability to filter the scores by Android OS version number. It's a neat way to see how software updates affect the performance of a device.

Download

PCMark for Android is a free benchmark app available now from Google Play. The update will be available at 10:00 UTC on Thursday, October 13.

Download PCMark for Android benchmark from Google Play



News from October 2016

Older News

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998