Update to when derived variables are set and preventing overwriting (Biospecimen)

[erincschwartz]

It appears that the blood, urine, mouthwash collected derived variables are being set when Collection Data Entry screen is Saved (not upon Finalization), if Sample Collected checkmark is entered for blood, urine and/or mouthwash are checked and Full Specimen ID is scanned in. The SMDB shows that these samples have been collected. See Dev test case Connect ID 8668097914 These derived variables should only be set when Finalization is complete on the Collection Data Entry screen.

During testing on 12/9, it was confirmed that current code for setting derived variables operates as an "every save" operation and should be a "finalization" operation.

Also, it appears that even after derived variables are set, they are overwritten in some cases when another biospecimen collection is entered. See example of this for mouthwash collected status, which had originally been set to "Collected" after Research collection was entered but was updated to 'Not collected' after a subsequent Clinical colleciton was entered: https://nih.app.box.com/file/1593167641811 (slide 1). Once set, derived variables should not be overwritten.

[erincschwartz]

We have encountered another issue that may or may not be related to the issues above.

Connect ID: 2499626470
Specimens have been received and finalized in the dashboard for blood and urine and BioRec_CollectFinal_v1r0= yes; however, various derived variables did not derive as expected. Here is a list of what we have noticed, there may be more:

BioFin_BaseBloodCol_v1r0= No

Also, the following variables are not populated:
BioFin_BaseBloodCol_v1r0
BioFin_BaseUrineCol_v1r0
BioSpm_BloodSettingBL_v1r0
BioSpm_UrineSettingBL_v1r0
BioClin_DBBloodRRLBL_v1r0
BioClin_DBBloodRRLDtBL_v1r0
BioClin_DBUrineRRLBL_v1r0
BioClin_DBUrineRRLDtBL_v1r0

Similar issue exists for Connect ID 224012264, except BioSpm_BloodSettingBL_v1r0 and BioSpm_UrineSettingBL_v1r0 did derive as expected.

[amber-emmes]

I'll look into this. Are you able to give more precise replication steps?

[brotzmanmj]

@amber-emmes for the latter occurrence that Erin entered late this afternoon, you would enter a clinical collection into the dashboard, mark at least one blood tube as collection, save and finalize the collection. Then look at derived variable BioFin_BaseBloodCol_v1r0 in the participants table. It should be 'Yes'. BioSpm_BloodSettingBL_v1r0 should be 'Clinical'. The variable BioClin_DBBloodRRLBL_v1r0 should be Yes and BioClin_DBBloodRRLDtBL_v1r0 should be the same date/time stamp as the date/time you made the specimen link in the dashboard when you entered the samples.

Same thing for urine but with the urine related variables above.

We're not sure why sometimes this is not occurring. Are these part of 'check derived variables' function? Is there anything that would cause 'check derived variables' sometimes not to run?

[amber-emmes]

I'm not seeing these variable names when I look them up in the CID dictionary. Would you be able to either provide the concept IDs or point me to where else I could view these?

The checkDerivedVariables function itself was unchanged in the recent work, but it is possible that it's not being run at some relevant appropriate time. The other possibility is that this is an existing bug which was only caught due to this testing, in which case it could be either that the checkDerivedVariables function is not running at all necessary points or that it's running but not always calculating the necessary variables.

[brotzmanmj]

Hi @amber-emmes please check with Madhuri for variables.

Since this item is for January release and not December release, it can wait for investigation until after the December release work is done. It is important but not 'drop everything' urgent. Thanks.

[erincschwartz]

When working on this issue, be sure to test #1062 to make sure the derived variables in that scenario populate as expected.

[amber-emmes]

Is there a bit more information available about what should be saved on page save vs. on finalization? Right now I'm thinking that all information on the page about the specimens should be saved, but should any participant information be updated? Any information not displayed on the page?