Benchmark Rules and Guidelines

People rely on Futuremark benchmarks to produce accurate and unbiased results. We safeguard that trust by having clear rules for hardware manufacturers and software developers that specify how a platform must interact with our benchmark software. In simple terms:

A platform must run the benchmark without modification as if it were any other application.


Optimization policy

Benchmark specific optimizations are not allowed. Additionally, all generic optimizations that change the work specified by the benchmark are prohibited.

  • The platform may not change the quality level of the work.
  • The platform may not use an alternative technique to that requested by the workload.
  • The platform may not replace or remove any portion of the requested work even if the change would result in the same output.
  • Optimizations based on empirical data of benchmark workloads are not allowed.
  • Optimizations that change the output of the work are not allowed.

Application detection

The platform may not detect the launch of the benchmark executable. The platform must not alter, replace or override any parameters or parts of the test, nor modify the usual functioning of the platform based on the detection of the benchmark.

Exceptions
  • The platform may detect the benchmark application if it is the general approach for enabling specific platform features, such as multi-GPU use.
  • The platform may detect the benchmark application if there is a known conflict between the benchmark application and the platform that would otherwise prevent the benchmark from functioning correctly. This must be pre-approved with Futuremark.

Delisted devices


The purpose of the 3DMark Device Channel is to help you compare the performance of different devices. When a manufacturer breaks the rules to boost the ranking of a device it is unfair and misleading.

We delist devices that do not comply with our rules. We do this to encourage all manufactures and operators to be fair and honest with their customers.

Delisted models appear unranked, and without scores, at the bottom of the 3DMark Device Channel and the Best Mobile Devices list on our website. Scores from delisted devices should not be used to compare devices.

Report a device

Press reviewers can report models with suspect scores by emailing us.
Please include evidence to back up your claim.

Right to appeal

Manufacturers of delisted devices are invited to contact us if they believe a model has been unfairly delisted, or if they have updated their device software to comply with our benchmark rules.


  • Which models have been delisted?

    Delisted devices appear without scores at the bottom of our Best Smartphone and Tablets list.

  • Why have you delisted these models?

    When a device modifies its behavior as a result of detecting 3DMark, its benchmark scores are no longer accurate and cannot be used to make fair comparisons between devices. Delisting is a last resort when we have evidence that a manufacturer is breaking our rules.

  • But Company X says it is not cheating?

    Whatever their intentions, manufacturers must follow our rules to get a valid benchmark score. Break the rules, no score. It's as simple as that. Manufacturers can contact us if they believe a device has been unfairly delisted.

  • Isn't the point of benchmarking to measure maximum performance?

    Outside of industry, there is little value in measuring theoretical maximum performance. Wouldn't you rather know how a device performs while running apps and games? 3DMark measures real-world performance, which includes the effects of CPU scaling, thermal throttling, and how the device chooses to balance power and battery life.

  • I'm not cheating, why delist my score?

    Delisting does not mean that you have done anything wrong. The fault usually lies with the operator or manufacturer of your device.

  • My phone is stock, why is it delisted?

    We are focused on identifying and delisting stock device models with factory settings that break our rules. All results from those specific models, modded or stock, are affected by the delisting.

  • So overclocking isn't allowed now?

    Overclocking by manufacturers is allowed provided that it applies equally to all apps, all of the time. Overclocking optimizations that are selectively applied to our benchmarks are forbidden.

  • If I root my phone, will it be delisted?

    No. Rooting, modifying, and tweaking your device is part of the fun of benchmarking. We have no plans to delist devices based on results from rooted, modded or jail-broken devices.

  • How can I get a valid score?

    Delisted devices will require updates from the manufacturer or operator before they can get valid scores. You should contact the manufacturer and operator of your device and show them this page.

  • Why are HTC One (M7), HTC One Mini, Samsung Galaxy Note 10.1, and
    HP Slatebook 10 x2 delisted?

    These devices detect 3DMark, then set their CPU clocks to max, even when 3DMark is idle in the UI. They do not do this with a renamed, but otherwise identical version of the app, which means 3DMark cannot measure their real-world peformance.

  • Why is HTC One (M8) delisted?

    This device automatically enables a special high performance mode when it detects 3DMark. It does not do this with a renamed, but otherwise identical version of the app, which means 3DMark cannot measure its real-world peformance.

  • Why are some versions of Samsung's Galaxy S IV and Galaxy Note III delisted?

    With Android 4.3 these devices detect 3DMark, then set their CPU clocks to max. They do not do this with a renamed, but otherwise identical version of the app, which means 3DMark cannot measure their real-world peformance. Use the official Android 4.4.2 update or later to get a valid 3DMark score for these devices.

Driver approval policy

All currently supported versions of 3DMark on Windows are covered by our Driver Approval Policy, which dictates which drivers can be used to obtain valid results.

We test each vendor's drivers to ensure they do not unfairly influence or manipulate benchmark scores. You should only compare 3DMark scores when using the approved Windows drivers listed on this page. Older drivers may also have been approved but in general we recommend using the latest approved drivers to get the best results from your system.

Guidelines

Drivers must meet the following conditions before being considered for approval testing:

  • Only a vendor's official, public and production quality driver releases will be considered.
  • The driver must be WHQL certified for all supported graphics chipsets.
  • The driver-package must be publicly available for download from the vendor's website before the approval will be published.
  • OEM specific drivers based on an approved reference driver can be approved if the fulfil the guidelines.
  • We aim to process driver approval requests within seven working days.

Optimization Policy

Benchmark specific driver optimizations are not allowed. Additionally, all generic driver optimizations that change the rendering quality or rendering technique requested or specified by the benchmark are prohibited:

  • The driver may not change the rendering quality level, rendering technique or replace or alter any shaders or other parts of the benchmark test.
  • Driver optimizations based on empirical data of benchmark workloads are not allowed.
  • Generic driver optimizations that do not violate the above rules and benefit applications and games in general, are acceptable only if the rendering output is mathematically consistent with that of the Microsoft DirectX reference rasterizer.

Application detection

With the exception of setting mandatory parameters specific to systems with multiple GPUs, such as AMD CrossFire or NVIDIA SLI, drivers may not detect the launch of the benchmark executable and alter, replace or override any parameters or parts of the test based on the detection.

Special cases

In special cases, such as a major product launch, we will consider approving a new driver with the following exceptions to our standard policy:

  • The driver may be pre-release and is not required to be WHQL certified.
  • The driver must be delivered to Futuremark at least seven days before the desired approval date.
  • The driver-package must be publicly available for download from the vendor's website within seven working days of the approval.

Cancellation

We reserve the right to cancel the approval of any driver if it is later found not to comply with the conditions defined in this, or later, versions of this policy. Such changes will be shown on the approved drivers list on this page.

Policy changes

We reserve the right to revise this document. This version is 1.1 dated April 23, 2012.