The MLPerf results table is organized first by System Type, then by Division, and then by Category.
MLPerf training supports two system types:
Regular, a broad category of on-premise and cloud systems
HPC, large super computing systems
MLPerf has two divisions.
The Closed division is intended to compare hardware platforms or software frameworks “apples-to-apples” and
requires using the same model and optimizer as the reference implementation.
The Open division is intended to foster faster models and optimizers and
allows any ML approach that can reach the target quality.
MLPerf divides benchmark results into four Categories based on availability.
Available In Cloud systems are available for rent in the cloud.
Available On Premise systems contain only components that are available for purchase.
Preview systems must be submittable as Available In Cloud or Available on Premise in the next submission round.
Research, Development, or Internal (RDI) contain experimental, in development, or internal-use hardware or software.
Each row in the results table is a set of results produced by a single submitter
using the same software stack and hardware platform. Each row contains the following information:
Submitter: The organization that submitted the results.
System: General system description.
Processor and count: The type and number of CPUs used, if CPUs perform the majority of ML compute.
Accelerator and count: The type and number of accelerators used, if accelerators perform the majority of ML compute.
Software: The ML framework and primary ML hardware library used.
Benchmark Results: Training time to reach a specified target quality, measured in minutes.
Details: link to metadata for submission.
Code: link to code for submission.
Notes: arbitrary notes from submitter.