Skip to content

Commit

Permalink
finalize DFE development docs; closes #1085 (#1625)
Browse files Browse the repository at this point in the history
* finalize DFE development docs; closes #1085

* dfe qc issue template
  • Loading branch information
petrelharp authored Dec 23, 2024
1 parent ff79d8d commit b845b62
Show file tree
Hide file tree
Showing 2 changed files with 37 additions and 13 deletions.
17 changes: 17 additions & 0 deletions .github/ISSUE_TEMPLATE/dfe-qc-issue-template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
name: DFE QC issue template
about: Quality control process for DFE addition
title: QC for {dfe_id} ({species})
labels: DFE QC
assignees: ''

---

**PR for new model:** {link to Pull Request}
**Original paper:** {ink to original paper}
**Parameter values:** {reference to where parameter values can be found}

**Potential issues that might lead to differences between the model implementations:**
{description of any potential issues}

**QC'er requests:** {tag potential developers to QC the model}
33 changes: 20 additions & 13 deletions docs/development.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1652,7 +1652,7 @@ Testing your DFE model and submitting a PR

After you finished your implementation, and specified all the
necessary citations,
we recommend that you run some basic local tests to see that
we recommend that you run some basic local checks to see that
the model was successfully loaded to ``stdpopsim``.
You may follow the process outlined for `Testing your demographic model and submitting a PR`_.

Expand All @@ -1667,24 +1667,31 @@ of the development team before it is fully incorporated into ``stdpopsim``.
This will likely require additional feedback from you,
so, stay tuned for discussion during the review process.

To facilitate this, there is one more step: please open a
`new issue <https://github.com/popsim-consortium/stdpopsim/issues/new>`__,
using the "DFE QC" template.
The template asks for the basic information that someone will need
to independently verify the implemented DFE.

---------------------
Reviewing a DFE model
---------------------

The review process for DFE models is currently being developed.
For now, we suggest that you
`open a new blank issue <https://github.com/popsim-consortium/stdpopsim/issues/new>`__
and specify the following information:
The process for reviewing a DFE is essentially the same
as for reviewing a demographic model (see `Overview of the stdpopsim review process`_).
Briefly, you will re-implement the DFE "blind", i.e., without looking at the DFE
implementation already added to the code.
Then, the unit tests check whether the implementations are equivalent.
To do this, you add your implementation to ``stdpopsim/qc/<SPECIES_ID>.py``,
followed by a call like

.. code-block:: python
1. **PR for new model:**
2. **Original paper:**
3. **Parameter values:**
4. **Potential issues:**
5. **QC'er requests:**
_species.get_dfe(_MODEL_ID_).register_qc(_your_review_function())
A reviewer will be assigned to check your implementation and approve it.
All discussion about the review can be conducted in the **QC issue**
mentioned above.
where ``_MODEL_ID_`` is the string specified as the ID for the original DFE,
and ``_your_review_function()`` is the function you've added to the QC file
that returns a DFE object.

****************
Coding standards
Expand Down

0 comments on commit b845b62

Please sign in to comment.