Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New QC Tracking #2

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

New QC Tracking #2

wants to merge 2 commits into from

Conversation

NickCEBM
Copy link
Contributor

This adjusts our pipeline to now include all the details about the QC process directly in the CSV that runs the website. This will allow use to run the entire website off of the CSV and not rely on externally scraping QC info.

There are also a few minor changes and corrections throughout for both future use and convenience:

  1. All completion dates are now explicitly in the CSV rather than just the ones we use. These are used on ocassion and had to be retrieved from the intermittent steps of the old SQL query. Now they are just included here for convenience.
  2. Created some new variables both for easier usage with the website and easier use in the code. This included covered, and variables that should, when converted to bools, add up to the numerator and denominator used on the website.
  3. Fixed a bug in the defaulted_date section

In order to implement the csv fields into the QC pipeline:

  1. No longer need to scrape to look for results of due trials. inc_in_num now should calculate the numerator directly
  2. For trials in QC, we can calculate the days late (if late) from first_submission_qc
  3. cancelled_now tells you if the results are currently cancelled. If results_due is true and cancelled_now is true, the trial should enter the overdue - cancelled status on the website (this is accounted for in the inc_in_num calculation)
  4. In order to stay consistent with the way we currently handle cancelled trials, when a trial is cancelled and then resubmitted, we calculate days overdue as of the next submission after cancellation. new_submission_date gives you this date. However, once a trial moves from having pending results to having full results, then we can revert to just using results_submitted_date as we currently do to calculate days late (if any).

This adjusts our pipeline to now include all the details about the QC process directly in the CSV that runs the website. This will allow use to run the entire website off of the CSV and not rely on externally scraping QC info.

There are also a few minor changes and corrections throughout for both future use and convenience:
1. All completion dates are now explicitly in the CSV rather than just the ones we use. These are used on ocassion and had to be retrieved from the intermittent steps of the old SQL query. Now they are just included here for convenience.
2. Created some new variables both for easier usage with the website and easier use in the code. This included `covered`, and variables that should, when converted to bools, add up to the numerator and denominator used on the website.
3. Fixed a bug in the `defaulted_date` section

In order to implement the csv fields into the QC pipeline:

1. No longer need to scrape to look for results of due trials. `inc_in_num` now should calculate the numerator directly
2. For trials in QC, we can calculate the days late (if late) from `first_submission_qc`
3. `cancelled_now` tells you if the results are currently cancelled. If `results_due` is true and `cancelled_now` is true, the trial should enter the `overdue - cancelled` status on the website (this is accounted for in the `inc_in_num` calculation)
4. In order to stay consistent with the way we currently handle cancelled trials, when a trial is cancelled and then resubmitted, we calculate days overdue as of the next submission after cancellation. `new_submission_date` gives you this date. However, once a trial moves from having pending results to having full results, then we can revert to just using `results_submitted_date` as we currently do to calculate days late (if any).
@NickCEBM NickCEBM requested a review from sebbacon May 21, 2019 17:05
@NickCEBM
Copy link
Contributor Author

NickCEBM commented May 23, 2019

Just want to point out that this should ideally be pretty thoroughly vetted before implementing live. Lots of moving parts here and working with the QC data array is not the most straightforward thing.

small change of `type() ==` to `isinstance()`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant