Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

finalize DFE development docs; closes #1085 #1625

Merged
merged 2 commits into from
Dec 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .github/ISSUE_TEMPLATE/dfe-qc-issue-template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
name: DFE QC issue template
about: Quality control process for DFE addition
title: QC for {dfe_id} ({species})
labels: DFE QC
assignees: ''

---

**PR for new model:** {link to Pull Request}
**Original paper:** {ink to original paper}
**Parameter values:** {reference to where parameter values can be found}

**Potential issues that might lead to differences between the model implementations:**
{description of any potential issues}

**QC'er requests:** {tag potential developers to QC the model}
33 changes: 20 additions & 13 deletions docs/development.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1652,7 +1652,7 @@ Testing your DFE model and submitting a PR

After you finished your implementation, and specified all the
necessary citations,
we recommend that you run some basic local tests to see that
we recommend that you run some basic local checks to see that
the model was successfully loaded to ``stdpopsim``.
You may follow the process outlined for `Testing your demographic model and submitting a PR`_.

Expand All @@ -1667,24 +1667,31 @@ of the development team before it is fully incorporated into ``stdpopsim``.
This will likely require additional feedback from you,
so, stay tuned for discussion during the review process.

To facilitate this, there is one more step: please open a
`new issue <https://github.com/popsim-consortium/stdpopsim/issues/new>`__,
using the "DFE QC" template.
The template asks for the basic information that someone will need
to independently verify the implemented DFE.

---------------------
Reviewing a DFE model
---------------------

The review process for DFE models is currently being developed.
For now, we suggest that you
`open a new blank issue <https://github.com/popsim-consortium/stdpopsim/issues/new>`__
and specify the following information:
The process for reviewing a DFE is essentially the same
as for reviewing a demographic model (see `Overview of the stdpopsim review process`_).
Briefly, you will re-implement the DFE "blind", i.e., without looking at the DFE
implementation already added to the code.
Then, the unit tests check whether the implementations are equivalent.
To do this, you add your implementation to ``stdpopsim/qc/<SPECIES_ID>.py``,
followed by a call like

.. code-block:: python

1. **PR for new model:**
2. **Original paper:**
3. **Parameter values:**
4. **Potential issues:**
5. **QC'er requests:**
_species.get_dfe(_MODEL_ID_).register_qc(_your_review_function())

A reviewer will be assigned to check your implementation and approve it.
All discussion about the review can be conducted in the **QC issue**
mentioned above.
where ``_MODEL_ID_`` is the string specified as the ID for the original DFE,
and ``_your_review_function()`` is the function you've added to the QC file
that returns a DFE object.

****************
Coding standards
Expand Down
Loading