Skip to content

Latest commit

 

History

History
91 lines (70 loc) · 6.87 KB

HISTORY.CM.md

File metadata and controls

91 lines (70 loc) · 6.87 KB
Raw

This document narrates the history of the creation and design of CM and CM4MLOps (also known as CK2) by Grigori Fursin. It also highlights the donation of this open-source technology to MLCommons, aimed at benefiting the broader community and fostering its ongoing development as a collaborative, community-driven initiative:

  • Jan 28, 2021: After delivering an invited ACM TechTalk'21 about the Collective Knowledge framework (CK1) and reproducibility initiatives for conferences, as well as CK-MLOps and MLPerf automations, Grigori received lots of positive feedback and suggestions for improvements to workflow automations: https://learning.acm.org/techtalks/reproducibility.

    Following this, Grigori began prototyping CK2 (later CM) to streamline CK1, CK-MLOps and MLPerf benchmarking. The goal was to dramatically simplify CK1 workflows by introducing just a few core and portable automations, which eventually evolved into CM script and CM cache.

    At that time, the cTuning foundation hosted CK1 and all the prototypes for the CM framework at https://github.com/ctuning/ck: ref1, ref2, ref3.

  • Sep 23, 2021: donated CK1, CK-MLOps, MLPerf automations and early prototypes of CM from the cTuning repository to MLCommons: ref1, ref2, ref3, ref4, ref5.

  • Mar 1, 2022: started developing cm-mlops: ref.

  • Mar 24, 2022: after successfully stabilizing the initial prototype of CM, donated it to MLCommons to benefit the entire community: ref1, ref2, ref3, ref4.

  • Apr 6, 2022: started transitioning previous MLOps and MLPerf automations from the mlcommons/ck-mlops format to the new CM format using the cm-mlops repository (will be later renamed to cm4mlops): ref1, ref2.

  • Apr 22, 2022: began architecting "Intelligent Components" in the CM-MLOps repository, which will be renamed to CM Script at a later stage: ref1, ref2, ref3.

    At the same time, prototyped other core CM automations, including IC, Docker, and Experiment: ref1, ref2.

  • Apr 28, 2022: donated CM-MLOps to MLCommons, which was later renamed to CM4MLOps: ref.

  • May 9, 2022: developed the initial set of core IC automations for MLOps (aka CM scripts): ref1, ref2, ref3.

  • May 11, 2022: After successfully prototyping CM and CM-MLOps, deprecated the CK1 framework in favor of CM. Welcomed Arjun as a maintainer and tester for CM and CM-MLOps: ref.

    Created a file to document and track our public developments at MLCommons.

  • Jun 8, 2022: renamed the 'IC' automation to the more intuitive 'CM script' automation. ref1, ref2, ref3.

  • Jun 16, 2022: prototyped the CM cache automation to facilitate caching and reuse of the outputs from CM scripts: ref1, ref2, ref3.

  • Sep 6, 2022: delivered CM demo to run MLPerf while deprecating CK1 automations for MLPerf: ref1, ref2.

    Welcomed Arjun Suresh as a contributor to CM: ref.

  • From September 2022: coordinated community development of CM and CM4MLOps to modularize and automate MLPerf and support reproducibility initiatives at ML and Systems conferences through the MLCommons Task Force on Automation and Reproducibility.

  • Starting in April 2024, began the gradual transfer of ongoing maintenance and enhancement responsibilities for CM and CM4MLOps, including MLPerf automations, to MLCommons. Welcomed Anandhu Sooraj as a maintainer and contributor to CM4MLOps with MLPerf automations.

For more details, please refer to this white paper and ACM REP'23 keynote.