Skip to content

Commit

Permalink
fix(evaluation): corrected stars remote code repository evaluation
Browse files Browse the repository at this point in the history
  • Loading branch information
amyheather committed Sep 3, 2024
1 parent b405fe0 commit 57c5ec0
Show file tree
Hide file tree
Showing 3 changed files with 23 additions and 4 deletions.
6 changes: 3 additions & 3 deletions evaluation/artefacts.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ This page evaluates the extent to which the original study meets the recommendat

Of the **8** essential STARS components:

* **2** were met fully (✅)
* **6** were not met (❌)
* **3** were met fully (✅)
* **5** were not met (❌)

Of the **5** optional STARS components:

Expand All @@ -24,7 +24,7 @@ Of the **5** optional STARS components:
| Minimum documentation | Minimal instructions (e.g. in README) that overview (a) what model does, (b) how to install and run model to obtain results, and (c) how to vary parameters to run new experiments | ❌ Not met | No documentation provided |
| ORCID | ORCID for each study author | ❌ Not met | - |
| Citation information | Instructions on how to cite the research artefact (e.g. CITATION.cff file) | ❌ Not met | - |
| Remote code repository | Code available in a remote code repository (e.g. GitHub, GitLab, BitBucket) | ❌ Not met | - |
| Remote code repository | Code available in a remote code repository (e.g. GitHub, GitLab, BitBucket) | ✅ Fully | <https://github.com/shiweih/desECR> |
| Open science archive | Code stored in an open science archive with FORCE11 compliant citation and guaranteed persistance of digital artefacts (e.g. Figshare, Zenodo, the Open Science Framework (OSF), and the Computational Modeling in the Social and Ecological Sciences Network (CoMSES Net)) | ❌ Not met | - |
| **Optional components** |
| Enhanced documentation | Open and high quality documentation on how the model is implemented and works (e.g. via notebooks and markdown files, brought together using software like Quarto and Jupyter Book). Suggested content includes:<br>• Plain english summary of project and model<br>• Clarifying license<br>• Citation instructions<br>• Contribution instructions<br>• Model installation instructions<br>• Structured code walk through of model<br>• Documentation of modelling cycle using TRACE<br>• Annotated simulation reporting guidelines<br>• Clear description of model validation including its intended purpose | ❌ Not met | - |
Expand Down
2 changes: 1 addition & 1 deletion evaluation/reproduction_report.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ import plotly.express as px
# Create dataframe of results
col = ['fully', 'partially', 'not', 'na']
eval_dict = {
'STARS (essential)': [2, 0, 6, 0],
'STARS (essential)': [3, 0, 5, 0],
'STARS (optional)': [2, 0, 3, 0],
'Badges (criteria)': [3, 0, 9, 0],
'Badges (badges)': [0, 0, 12, 0],
Expand Down
19 changes: 19 additions & 0 deletions logbook/posts/2024_09_03/index.qmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
title: "Day 16"
author: "Amy Heather"
date: "2024-09-03"
categories: [evaluation]
bibliography: ../../../quarto_site/references.bib
---

## Untimed: Amendment to evaluation

I noticed a mistake in my evaluation of the repository against the STARS framework

**Criteria:** Code available in a remote code repository (e.g. GitHub, GitLab, BitBucket)

**Original decision:** ❌ Not met

**New decision:** ✅ Fully - <https://github.com/shiweih/desECR>

I amended `artefacts.qmd`, `reproduction_report.qmd` and [stars_wp1_summary](https://github.com/pythonhealthdatascience/stars_wp1_summary) accordingly.

0 comments on commit 57c5ec0

Please sign in to comment.