diff --git a/.github/ISSUE_TEMPLATE/dfe-qc-issue-template.md b/.github/ISSUE_TEMPLATE/dfe-qc-issue-template.md new file mode 100644 index 000000000..47e42e33e --- /dev/null +++ b/.github/ISSUE_TEMPLATE/dfe-qc-issue-template.md @@ -0,0 +1,17 @@ +--- +name: DFE QC issue template +about: Quality control process for DFE addition +title: QC for {dfe_id} ({species}) +labels: DFE QC +assignees: '' + +--- + +**PR for new model:** {link to Pull Request} +**Original paper:** {ink to original paper} +**Parameter values:** {reference to where parameter values can be found} + +**Potential issues that might lead to differences between the model implementations:** +{description of any potential issues} + +**QC'er requests:** {tag potential developers to QC the model} diff --git a/docs/development.rst b/docs/development.rst index e8234a1a8..92fb82b58 100644 --- a/docs/development.rst +++ b/docs/development.rst @@ -1652,7 +1652,7 @@ Testing your DFE model and submitting a PR After you finished your implementation, and specified all the necessary citations, -we recommend that you run some basic local tests to see that +we recommend that you run some basic local checks to see that the model was successfully loaded to ``stdpopsim``. You may follow the process outlined for `Testing your demographic model and submitting a PR`_. @@ -1667,24 +1667,31 @@ of the development team before it is fully incorporated into ``stdpopsim``. This will likely require additional feedback from you, so, stay tuned for discussion during the review process. +To facilitate this, there is one more step: please open a +`new issue `__, +using the "DFE QC" template. +The template asks for the basic information that someone will need +to independently verify the implemented DFE. + --------------------- Reviewing a DFE model --------------------- -The review process for DFE models is currently being developed. -For now, we suggest that you -`open a new blank issue `__ -and specify the following information: +The process for reviewing a DFE is essentially the same +as for reviewing a demographic model (see `Overview of the stdpopsim review process`_). +Briefly, you will re-implement the DFE "blind", i.e., without looking at the DFE +implementation already added to the code. +Then, the unit tests check whether the implementations are equivalent. +To do this, you add your implementation to ``stdpopsim/qc/.py``, +followed by a call like + +.. code-block:: python -1. **PR for new model:** -2. **Original paper:** -3. **Parameter values:** -4. **Potential issues:** -5. **QC'er requests:** + _species.get_dfe(_MODEL_ID_).register_qc(_your_review_function()) -A reviewer will be assigned to check your implementation and approve it. -All discussion about the review can be conducted in the **QC issue** -mentioned above. +where ``_MODEL_ID_`` is the string specified as the ID for the original DFE, +and ``_your_review_function()`` is the function you've added to the QC file +that returns a DFE object. **************** Coding standards