MLPerf Training v0.5 Results
December 12th, 2018
You may wish to read the Training Overview to better understand the results.
To see the current MLPerf Training v0.6 results go here.
MLPerf v0.5 Results Table Explanation
The MLPerf results table is organized first by Division and then by Category. MLPerf has two divisions. The Closed division is intended to compare hardware platforms or software frameworks “apples-to-apples” and requires using the same model and optimizer as the reference implementation. The Open division is intended to foster faster models and optimizers and allows any ML approach that can reach the target quality. MLPerf divides benchmark results into four Categories based on availability.
Available In Cloud systems are available for rent in the cloud.
Available On Premise systems contain only components that are available for purchase.
Preview systems must be submittable as Available In Cloud or Available on Premise in the next submission round.
Research systems either contain experimental hardware or software or available components at experimentally large scale.
Each row in the results table is a set of results produced by a single submitter
using the same software stack and hardware platform. Each row contains the following information:
Submitter: The organization that submitted the results.
Hardware: The type of ML hardware used, e.g. accelerators or high-performance CPUs.
Chip Count and Type: The number of ML hardware chips used, and if they are accelerators (a) or CPUs (c).
Software: The ML framework and primary ML hardware library used.
Benchmark Results: The benchmark results as described above. By default, benchmark results are presented as speedups relative to a Pascal P100. The results page enables switching to absolute times.
Cloud Scale: (only for Available Cloud systems): Cloud scale is derived from on-demand pricing by several major cloud providers and gives a rough indicator of relative system size/cost. The reference single Pascal P100 system has a cloud scale of 1. A system with a cloud scale of 4 would be roughly four times as big/expensive.
Power (only for non-Available Cloud systems): Information for Available On-premise systems. Due to the complexity of standardizing power measurement, this version of MLPerf simply allows voluntary reporting of arbitrary unofficial power information.
Details: link to metadata for submission.