MLCommons

All

103 repositories

modelbench
Public
Run safety benchmarks against AI models and view detailed reports showing how well they performed.
Python
•
Apache License 2.0
•12•73•193•4•Updated Jan 18, 2025Jan 18, 2025
mlperf-automations
Public
Python
•
Apache License 2.0
•5•1•46•0•Updated Jan 18, 2025Jan 18, 2025
dynabench
Public
Python
•
MIT License
•16•21•7•5•Updated Jan 17, 2025Jan 17, 2025
mlperf_inference_unofficial_submissions_v5.0
Public
These are automated test submissions for validating the MLPerf inference workflows
mlperf mlperf-inference mlperf-inference-results
Mermaid
•1•0•0•0•Updated Jan 17, 2025Jan 17, 2025
mlcflow
Public
MLCFlow: Simplifying MLPerf Automations
automation-framework mlperf mlcflow mlperf-automations
Python
•
Apache License 2.0
•3•2•5•0•Updated Jan 17, 2025Jan 17, 2025
inference
Public
Reference implementations of MLPerf™ inference benchmarks
benchmark machine-learning
Python
•
Apache License 2.0
•538•1.3k•170•14•Updated Jan 17, 2025Jan 17, 2025
mlperf_inference_test_submissions_v5.0
Public
HTML
•3•0•0•0•Updated Jan 17, 2025Jan 17, 2025
algorithmic-efficiency
Public
MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.
Python
•
Apache License 2.0
•69•350•30•12•Updated Jan 16, 2025Jan 16, 2025
mobile_app_open
Public
Mobile App Open
C++
•
Apache License 2.0
•24•51•29•2•Updated Jan 16, 2025Jan 16, 2025
training
Public
Reference implementations of MLPerf™ training benchmarks
benchmark machine-learning
Python
•
Apache License 2.0
•562•1.6k•16•42•Updated Jan 15, 2025Jan 15, 2025
ck
Public
Collective Knowledge (CK) and Collective Minds (CM): community-driven projects to learn how to run AI, ML and other emerging workloads in a more efficient and cost-effective way across diverse models, datasets, software and hardware using CK, CM/CMX and MLPerf automations
education automation optimization best-practices collaboration cm modularity workflows portability reusability
Python
•
Apache License 2.0
•120•613•13•0•Updated Jan 15, 2025Jan 15, 2025
cm4mlops
Public
A collection of portable, reusable and cross-platform CM automations for MLOps and MLPerf to simplify the process of building, benchmarking and optimizing AI systems across diverse models, data sets, software and hardware
workflow devops machine-learning automation cross-platform reusable workflow-automation mlops mlperf
Python
•
Apache License 2.0
•24•16•7•0•Updated Jan 15, 2025Jan 15, 2025
power-dev
Public
Dev repo for power measurement for the MLPerf™ benchmarks
Python
•
Apache License 2.0
•25•20•10•4•Updated Jan 14, 2025Jan 14, 2025
training_results_v3.1
Public
This repository contains the results and code for the MLPerf™ Training v3.1 benchmark.
Python
•
Apache License 2.0
•11•17•7•4•Updated Jan 14, 2025Jan 14, 2025
inference_results_visualization_template
Public template
JavaScript
•
Apache License 2.0
•6•0•1•0•Updated Jan 8, 2025Jan 8, 2025
logging
Public
MLPerf™ logging library
Python
•
Apache License 2.0
•46•32•41•2•Updated Jan 8, 2025Jan 8, 2025
mlperf_inference_submissions
Public template
Shell
•0•0•0•1•Updated Jan 8, 2025Jan 8, 2025
inference_policies
Public
Issues related to MLPerf™ Inference policies, including rules and suggested changes
Apache License 2.0
•52•58•18•3•Updated Jan 8, 2025Jan 8, 2025
mobile_open
Public
MLPerf Mobile benchmarks
Python
•
Apache License 2.0
•2•10•6•2•Updated Jan 7, 2025Jan 7, 2025
ailuminate
Public
Apache License 2.0
•1•1•1•0•Updated Jan 4, 2025Jan 4, 2025
GaNDLF-Synth
Public
Extension for GaNDLF [gandlf.org] to enable synthesis.
medical-imaging synthesis
Python
•
Apache License 2.0
•2•4•6•0•Updated Dec 30, 2024Dec 30, 2024
tiny
Public
MLPerf™ Tiny is an ML benchmark suite for extremely low-power systems such as microcontrollers
C
•
Apache License 2.0
•93•368•13•6•Updated Dec 24, 2024Dec 24, 2024
medperf
Public
An open benchmarking platform for medical artificial intelligence using Federated Evaluation.
Python
•
Apache License 2.0
•33•150•52•23•Updated Dec 21, 2024Dec 21, 2024
GaNDLF
Public
A generalizable application framework for segmentation, regression, and classification using PyTorch
machine-learning framework deep-learning regression pytorch medical-imaging classification segmentation data-augmentation biomedical-image-processing
Python
•
Apache License 2.0
•80•170•15•3•Updated Dec 20, 2024Dec 20, 2024
policies
Public
General policies for MLPerf™ including submission rules, coding standards, etc.
Python
•
Apache License 2.0
•55•28•35•39•Updated Dec 18, 2024Dec 18, 2024
training_results_v4.1
Public
This repository contains the results and code for the MLPerf™ Training v4.1 benchmark.
Python
•6•4•1•0•Updated Dec 11, 2024Dec 11, 2024
mlperf-automations_archived
Public archive
This repository contains the automations and scripts used to automate MLPerf benchmarks (mainly MLPerf inference for now)
automation mlperf
Python
•
Apache License 2.0
•24•0•46•0•Updated Dec 11, 2024Dec 11, 2024
mlperf_client
Public
MLPerf Client is a benchmark for Windows and macOS, focusing on client form factors in ML inference scenarios.
benchmarking machine-learning ml
C++
•
Apache License 2.0
•0•8•0•0•Updated Dec 10, 2024Dec 10, 2024
ck-mlops
Public
A collection of portable workflows, automation recipes and components for MLOps in a unified CK format. Note that this repository is outdated - please check the 2nd generation of the CK workflow automation meta-framework with portable MLOps and DevOps components here:
machine-learning automation optimization artificial-intelligence open-science systems reproducibility workflows portability co-design
Python
•
Apache License 2.0
•7•18•0•0•Updated Dec 10, 2024Dec 10, 2024
chakra
Public
Repository for MLCommons Chakra schema and tools
Python
•
Apache License 2.0
•41•79•28•11•Updated Dec 6, 2024Dec 6, 2024