[ Back to MLPerf inference benchmarks index ]
GPT-J has two variants - gptj-99 and gptj-99.9 where the 99 and 99.9 specifies the required accuracy constraint
with respect to the reference floating point model. gptj-99.9 model is applicable only on a datacenter system.
In the edge category, gptj-99 has Offline and SingleStream scenarios and in the datacenter category, both gptj-99 and gptj-99.9 have Offline and Server scenarios.
Please check MLPerf inference GitHub for more details.
Run using the MLCommons CM framework
From Feb 2024, we suggest you to use this GUI to configure MLPerf inference benchmark, generate CM commands to run it across different implementations, models, data sets, software and hardware, and prepare your submissions.
Install MLCommons CM automation framework with automation recipes for MLPerf as described here.
The following guides explain how to run different implementations of this benchmark via CM:
Check the MLCommons Task Force on Automation and Reproducibility and get in touch via public Discord server.