Benchmark Rules and Guidelines
People rely on Futuremark benchmarks to produce accurate and unbiased results. We safeguard that trust by having clear rules for hardware manufacturers and software developers that specify how a platform must interact with our benchmark software. In simple terms:
A platform must run the benchmark without modification as if it were any other application.
Benchmark specific optimizations are not allowed. Additionally, all generic optimizations that change the work specified by the benchmark are prohibited.
- The platform may not change the quality level of the work.
- The platform may not use an alternative technique to that requested by the workload.
- The platform may not replace or remove any portion of the requested work even if the change would result in the same output.
- Optimizations based on empirical data of benchmark workloads are not allowed.
- Optimizations that change the output of the work are not allowed.
The platform may not detect the launch of the benchmark executable. The platform must not alter, replace or override any parameters or parts of the test, nor modify the usual functioning of the platform based on the detection of the benchmark.
- The platform may detect the benchmark application if it is the general approach for enabling specific platform features, such as multi-GPU use.
- The platform may detect the benchmark application if there is a known conflict between the benchmark application and the platform that would otherwise prevent the benchmark from functioning correctly. This must be pre-approved with Futuremark.
The purpose of the 3DMark Device Channel is to help you compare the performance of different devices. When a manufacturer breaks the rules to boost the ranking of a device it is unfair and misleading.
We delist devices that do not comply with our rules. We do this to encourage all manufactures and operators to be fair and honest with their customers.
Delisted models appear unranked, and without scores, at the bottom of the 3DMark Device Channel and the Best Mobile Devices list on our website. Scores from delisted devices should not be used to compare devices.
Report a device
Press reviewers can report models with suspect scores by emailing us. Please include evidence to back up your claim.
Right to appeal
Manufacturers of delisted devices are invited to contact us if they believe a model has been unfairly delisted, or if they have updated their device software to comply with our benchmark rules.
Which models have been delisted?
Delisted devices appear without scores at the bottom of our Best Mobile Devices list.
Why have you delisted these models?
You can only compare benchmark scores when the results are fair and accurate. That why we have clear rules that say what is and isn't allowed. Delisting is a last resort when we have evidence that a manufacturer is breaking the rules.
How do you know these devices are breaking the rules?
We use a variety of tools and techniques to check each model against our rules. It is often easy to show how scores are being manipulated. Members of the press can contact us for testing advice.
But Company X says it is not cheating?
Whatever their intentions, manufacturers must follow our rules to get a valid benchmark score. Break the rules, no score. It's as simple as that. Manufacturers can contact us if they believe a device has been unfairly delisted.
I'm not cheating, why delist my score?
Delisting does not mean that you have done anything wrong. The fault usually lies with the operator or manufacturer of your device.
My phone is a stock model, why is it delisted?
We are focused on identifying and delisting stock device models with factory settings that break our rules. All results from those specific models, modded or stock, are affected by the delisting.
If I root my phone, will it be delisted?
No. Rooting, modifying, and tweaking your device to improve its performance is part of the fun of benchmarking. We have no plans to delist devices based on results from rooted, modded or jail-broken devices.
How can I get a valid score?
Delisted devices will require updates from the manufacturer or operator before they can get valid scores. You should contact the manufacturer and operator of your device and show them this page.
Driver approval policy
All currently supported versions of 3DMark on Windows are covered by our Driver Approval Policy, which dictates which drivers can be used to obtain valid results.
We test each vendor's drivers to ensure they do not unfairly influence or manipulate benchmark scores. You should only compare 3DMark scores when using the approved Windows drivers listed on this page. Older drivers may also have been approved but in general we recommend using the latest approved drivers to get the best results from your system.
Drivers must meet the following conditions before being considered for approval testing:
- Only a vendor's official, public and production quality driver releases will be considered.
- The driver must be WHQL certified for all supported graphics chipsets.
- The driver-package must be publicly available for download from the vendor's website before the approval will be published.
- OEM specific drivers based on an approved reference driver can be approved if the fulfil the guidelines.
- We aim to process driver approval requests within seven working days.
Benchmark specific driver optimizations are not allowed. Additionally, all generic driver optimizations that change the rendering quality or rendering technique requested or specified by the benchmark are prohibited:
- The driver may not change the rendering quality level, rendering technique or replace or alter any shaders or other parts of the benchmark test.
- Driver optimizations based on empirical data of benchmark workloads are not allowed.
- Generic driver optimizations that do not violate the above rules and benefit applications and games in general, are acceptable only if the rendering output is mathematically consistent with that of the Microsoft DirectX reference rasterizer.
With the exception of setting mandatory parameters specific to systems with multiple GPUs, such as AMD CrossFire or NVIDIA SLI, drivers may not detect the launch of the benchmark executable and alter, replace or override any parameters or parts of the test based on the detection.
In special cases, such as a major product launch, we will consider approving a new driver with the following exceptions to our standard policy:
- The driver may be pre-release and is not required to be WHQL certified.
- The driver must be delivered to Futuremark at least seven days before the desired approval date.
- The driver-package must be publicly available for download from the vendor's website within seven working days of the approval.
We reserve the right to cancel the approval of any driver if it is later found not to comply with the conditions defined in this, or later, versions of this policy. Such changes will be shown on the approved drivers list on this page.
We reserve the right to revise this document. This version is 1.1 dated April 23, 2012.