[ Back to index ]
Click here to see the table of contents.
We've created CM automation to help users set up multiple virtual Python environments without altering current Python installation. Please follow this guide to set up Python virtual environment on your system.
We use the following definitions adopted by ACM and NISO:
-
Repeatability (Same team, same experimental setup)
The measurement can be obtained with stated precision by the same team using the same measurement procedure, the same measuring system, under the same operating conditions, in the same location on multiple trials. For computational experiments, this means that a researcher can reliably repeat her own computation.
-
Reproducibility (Different team, different experimental setup)
The measurement can be obtained with stated precision by a different team using the same measurement procedure, the same measuring system, under the same operating conditions, in the same or a different location on multiple trials. For computational experiments, this means that an independent group can obtain the same result using the author's own artifacts.
-
Replicability (Different team, same experimental setup)
The measurement can be obtained with stated precision by a different team, a different measuring system, in a different location on multiple trials. For computational experiments, this means that an independent group can obtain the same result using artifacts which they develop completely independently.
Feel free to ask your questions at our public Discord server. Join our MLCommons taskforce on automation and reproducibility to participate in collaborative developments and optimization challenges.