diff --git a/evaluation/artefacts.qmd b/evaluation/artefacts.qmd index 9281b95..810ef2f 100644 --- a/evaluation/artefacts.qmd +++ b/evaluation/artefacts.qmd @@ -7,8 +7,8 @@ This page evaluates the extent to which the original study meets the recommendat Of the **8** essential STARS components: -* **2** were met fully (✅) -* **6** were not met (❌) +* **3** were met fully (✅) +* **5** were not met (❌) Of the **5** optional STARS components: @@ -24,7 +24,7 @@ Of the **5** optional STARS components: | Minimum documentation | Minimal instructions (e.g. in README) that overview (a) what model does, (b) how to install and run model to obtain results, and (c) how to vary parameters to run new experiments | ❌ Not met | No documentation provided | | ORCID | ORCID for each study author | ❌ Not met | - | | Citation information | Instructions on how to cite the research artefact (e.g. CITATION.cff file) | ❌ Not met | - | -| Remote code repository | Code available in a remote code repository (e.g. GitHub, GitLab, BitBucket) | ❌ Not met | - | +| Remote code repository | Code available in a remote code repository (e.g. GitHub, GitLab, BitBucket) | ✅ Fully | | | Open science archive | Code stored in an open science archive with FORCE11 compliant citation and guaranteed persistance of digital artefacts (e.g. Figshare, Zenodo, the Open Science Framework (OSF), and the Computational Modeling in the Social and Ecological Sciences Network (CoMSES Net)) | ❌ Not met | - | | **Optional components** | | Enhanced documentation | Open and high quality documentation on how the model is implemented and works (e.g. via notebooks and markdown files, brought together using software like Quarto and Jupyter Book). Suggested content includes:
• Plain english summary of project and model
• Clarifying license
• Citation instructions
• Contribution instructions
• Model installation instructions
• Structured code walk through of model
• Documentation of modelling cycle using TRACE
• Annotated simulation reporting guidelines
• Clear description of model validation including its intended purpose | ❌ Not met | - | diff --git a/evaluation/reproduction_report.qmd b/evaluation/reproduction_report.qmd index 1b81599..a5729ae 100644 --- a/evaluation/reproduction_report.qmd +++ b/evaluation/reproduction_report.qmd @@ -116,7 +116,7 @@ import plotly.express as px # Create dataframe of results col = ['fully', 'partially', 'not', 'na'] eval_dict = { - 'STARS (essential)': [2, 0, 6, 0], + 'STARS (essential)': [3, 0, 5, 0], 'STARS (optional)': [2, 0, 3, 0], 'Badges (criteria)': [3, 0, 9, 0], 'Badges (badges)': [0, 0, 12, 0], diff --git a/logbook/posts/2024_09_03/index.qmd b/logbook/posts/2024_09_03/index.qmd new file mode 100644 index 0000000..95b3fa2 --- /dev/null +++ b/logbook/posts/2024_09_03/index.qmd @@ -0,0 +1,19 @@ +--- +title: "Day 16" +author: "Amy Heather" +date: "2024-09-03" +categories: [evaluation] +bibliography: ../../../quarto_site/references.bib +--- + +## Untimed: Amendment to evaluation + +I noticed a mistake in my evaluation of the repository against the STARS framework + +**Criteria:** Code available in a remote code repository (e.g. GitHub, GitLab, BitBucket) + +**Original decision:** ❌ Not met + +**New decision:** ✅ Fully - + +I amended `artefacts.qmd`, `reproduction_report.qmd` and [stars_wp1_summary](https://github.com/pythonhealthdatascience/stars_wp1_summary) accordingly.