Option to convert all MDCs to NA even on numeric data

[emoryn]

Feature Request Description

I notice on categorical and logical data types it converts all MDCs to NA, I would like this also for numeric data types. Or if I could set an option somewhere to include or exclude MDCs would be the best.

Proposed Solution

I'm not sure exactly how this works in the background to propose a more specific solution

Describe alternatives you've considered
running code on my own to replace the MDCs with NA

[ezraporter]

Thanks for opening the issue!

We had some discussion of this on #181. What we concluded at the time was that implementing this for general field types would require us to add an additional API call to retrieve the MDCs from the REDCap project info and we didn't think the benefit justified the extra API call.

For you use case, how cumbersome is manually replacing the MDCs? @skadauke did say at the time we might reconsider if someone opened an issue 😊

[emoryn]

Hello, What I've been doing is data[data=="MDC"] <- NA for ever data code. It's kind of cumbersome and on the date variables I'm getting an error about string not being in an unambiguous format. I think the biggest issue is that it's converting numeric variables to character strings when there's an MDC in the column, so I also have to go through and mutate(variable=as.numeric(variable)) to each one which is quite a task when there are hundreds of variables. Best, Emory Neer, B.S. Research Support Analyst bit.ly/emoryumzoom<http://bit.ly/emoryumzoom> | (305) 355-9081 | bit.ly/emoryneer<http://bit.ly/emoryneer> ***@***.***?anonymous&ep=signature> Book time to meet with ***@***.***?anonymous&ep=signature> [cid:75a44a12-7cb6-4646-bc0d-67bd4528f2ad] The information contained in this transmission may contain privileged and/or confidential information, including patient information protected by federal and state privacy laws. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.

…

________________________________ From: Ezra Porter ***@***.***> Sent: Thursday, July 11, 2024 2:59 PM To: CHOP-CGTInformatics/REDCapTidieR ***@***.***> Cc: Neer, Emory ***@***.***>; Author ***@***.***> Subject: [EXTERNAL] Re: [CHOP-CGTInformatics/REDCapTidieR] Option to convert all MDCs to NA even on numeric data (Issue #195) CAUTION: This email originated from outside the organization. DO NOT CLICK ON LINKS or OPEN ATTACHMENTS unless you know and trust the sender. Thanks for opening the issue! We had some discussion of this on #181<https://urldefense.com/v3/__https://github.com/CHOP-CGTInformatics/REDCapTidieR/issues/181__;!!KVu0SnhVq1hAFvslES2Y!IiyY03wUkS2Vab5XUfdGlLPiEgDNlsRJOOmwe9YiEuqoOaXjRQ-LijjkN705U-xfRvOyU9rC4ZGzf7ekFGQBxu12PDBzCA$>. What we concluded at the time was that implementing this for general field types would require us to add an additional API call to retrieve the MDCs from the REDCap project info and we didn't think the benefit justified the extra API call. For you use case, how cumbersome is manually replacing the MDCs? @skadauke<https://urldefense.com/v3/__https://github.com/skadauke__;!!KVu0SnhVq1hAFvslES2Y!IiyY03wUkS2Vab5XUfdGlLPiEgDNlsRJOOmwe9YiEuqoOaXjRQ-LijjkN705U-xfRvOyU9rC4ZGzf7ekFGQBxu3JB5lpLA$> did say at the time we might reconsider if someone opened an issue 😊 — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https://github.com/CHOP-CGTInformatics/REDCapTidieR/issues/195*issuecomment-2223676802__;Iw!!KVu0SnhVq1hAFvslES2Y!IiyY03wUkS2Vab5XUfdGlLPiEgDNlsRJOOmwe9YiEuqoOaXjRQ-LijjkN705U-xfRvOyU9rC4ZGzf7ekFGQBxu3BINxeRg$>, or unsubscribe<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AQYIHL4IUCU7OZ2PTV7OEULZL3IXPAVCNFSM6AAAAABKVBYQC6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRTGY3TMOBQGI__;!!KVu0SnhVq1hAFvslES2Y!IiyY03wUkS2Vab5XUfdGlLPiEgDNlsRJOOmwe9YiEuqoOaXjRQ-LijjkN705U-xfRvOyU9rC4ZGzf7ekFGQBxu3yUBRyMg$>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[emoryn]

I don't know how things work on the back end, if there could be an argument in extract_tibble that provides NA strings? Best, Emory Neer, B.S. Research Support Analyst bit.ly/emoryumzoom<http://bit.ly/emoryumzoom> | (305) 355-9081 | bit.ly/emoryneer<http://bit.ly/emoryneer> ***@***.***?anonymous&ep=signature> Book time to meet with ***@***.***?anonymous&ep=signature> [cid:c1f26c2e-4688-4791-9ceb-460522e130a9] The information contained in this transmission may contain privileged and/or confidential information, including patient information protected by federal and state privacy laws. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.

…

________________________________ From: Neer, Emory ***@***.***> Sent: Friday, August 16, 2024 12:44 PM To: CHOP-CGTInformatics/REDCapTidieR ***@***.***>; CHOP-CGTInformatics/REDCapTidieR ***@***.***> Cc: Author ***@***.***> Subject: Re: [EXTERNAL] Re: [CHOP-CGTInformatics/REDCapTidieR] Option to convert all MDCs to NA even on numeric data (Issue #195) Hello, What I've been doing is data[data=="MDC"] <- NA for ever data code. It's kind of cumbersome and on the date variables I'm getting an error about string not being in an unambiguous format. I think the biggest issue is that it's converting numeric variables to character strings when there's an MDC in the column, so I also have to go through and mutate(variable=as.numeric(variable)) to each one which is quite a task when there are hundreds of variables. Best, Emory Neer, B.S. Research Support Analyst bit.ly/emoryumzoom<http://bit.ly/emoryumzoom> | (305) 355-9081 | bit.ly/emoryneer<http://bit.ly/emoryneer> ***@***.***?anonymous&ep=signature> Book time to meet with ***@***.***?anonymous&ep=signature> [cid:75a44a12-7cb6-4646-bc0d-67bd4528f2ad] The information contained in this transmission may contain privileged and/or confidential information, including patient information protected by federal and state privacy laws. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.

________________________________ From: Ezra Porter ***@***.***> Sent: Thursday, July 11, 2024 2:59 PM To: CHOP-CGTInformatics/REDCapTidieR ***@***.***> Cc: Neer, Emory ***@***.***>; Author ***@***.***> Subject: [EXTERNAL] Re: [CHOP-CGTInformatics/REDCapTidieR] Option to convert all MDCs to NA even on numeric data (Issue #195) CAUTION: This email originated from outside the organization. DO NOT CLICK ON LINKS or OPEN ATTACHMENTS unless you know and trust the sender. Thanks for opening the issue! We had some discussion of this on #181<https://urldefense.com/v3/__https://github.com/CHOP-CGTInformatics/REDCapTidieR/issues/181__;!!KVu0SnhVq1hAFvslES2Y!IiyY03wUkS2Vab5XUfdGlLPiEgDNlsRJOOmwe9YiEuqoOaXjRQ-LijjkN705U-xfRvOyU9rC4ZGzf7ekFGQBxu12PDBzCA$>. What we concluded at the time was that implementing this for general field types would require us to add an additional API call to retrieve the MDCs from the REDCap project info and we didn't think the benefit justified the extra API call. For you use case, how cumbersome is manually replacing the MDCs? @skadauke<https://urldefense.com/v3/__https://github.com/skadauke__;!!KVu0SnhVq1hAFvslES2Y!IiyY03wUkS2Vab5XUfdGlLPiEgDNlsRJOOmwe9YiEuqoOaXjRQ-LijjkN705U-xfRvOyU9rC4ZGzf7ekFGQBxu3JB5lpLA$> did say at the time we might reconsider if someone opened an issue 😊 — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https://github.com/CHOP-CGTInformatics/REDCapTidieR/issues/195*issuecomment-2223676802__;Iw!!KVu0SnhVq1hAFvslES2Y!IiyY03wUkS2Vab5XUfdGlLPiEgDNlsRJOOmwe9YiEuqoOaXjRQ-LijjkN705U-xfRvOyU9rC4ZGzf7ekFGQBxu3BINxeRg$>, or unsubscribe<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AQYIHL4IUCU7OZ2PTV7OEULZL3IXPAVCNFSM6AAAAABKVBYQC6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRTGY3TMOBQGI__;!!KVu0SnhVq1hAFvslES2Y!IiyY03wUkS2Vab5XUfdGlLPiEgDNlsRJOOmwe9YiEuqoOaXjRQ-LijjkN705U-xfRvOyU9rC4ZGzf7ekFGQBxu3yUBRyMg$>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[ezraporter]

Thanks for the additional info! We have to think a little more about the best way to proceed on this given competing priorities and the (small) number of users we anticipate needing this feature.

In general, I think we'd favor the approach of having a separate function that modifies the supertibble to convert MDCs to NAs rather than baking it into extract_tibble(). It's possible I'll have some time in the next couple weeks to work on this and will report back here if I do.

[emoryn]

That's totally understandable. I look forward to seeing what you do. I honestly love this package! Best, Emory Neer, B.S. Research Support Analyst bit.ly/emoryumzoom<http://bit.ly/emoryumzoom> | (305) 355-9081 | bit.ly/emoryneer<http://bit.ly/emoryneer> ***@***.***?anonymous&ep=signature> Book time to meet with ***@***.***?anonymous&ep=signature> [cid:c0fac72f-40bb-4803-9f5b-c6373812a04f] The information contained in this transmission may contain privileged and/or confidential information, including patient information protected by federal and state privacy laws. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.

…

________________________________ From: Ezra Porter ***@***.***> Sent: Monday, August 19, 2024 5:47 PM To: CHOP-CGTInformatics/REDCapTidieR ***@***.***> Cc: Neer, Emory ***@***.***>; Author ***@***.***> Subject: [EXTERNAL] Re: [CHOP-CGTInformatics/REDCapTidieR] Option to convert all MDCs to NA even on numeric data (Issue #195) CAUTION: This email originated from outside the organization. DO NOT CLICK ON LINKS or OPEN ATTACHMENTS unless you know and trust the sender. Thanks for the additional info! We have to think a little more about the best way to proceed on this given competing priorities and the (small) number of users we anticipate needing this feature. In general, I think we'd favor the approach of having a separate function that modifies the supertibble to convert MDCs to NAs rather than baking it into extract_tibble(). It's possible I'll have some time in the next couple weeks to work on this and will report back here if I do. — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https://github.com/CHOP-CGTInformatics/REDCapTidieR/issues/195*issuecomment-2297515298__;Iw!!KVu0SnhVq1hAFvslES2Y!NWW8QvxeSYn7ESmqylfPj34VBllQWnRMkEQGY7AcrDCJBfm0bwbRl6VuZ0uinjVI-oB_8a1xUnoritmwxBt2R_hPgRLDHQ$>, or unsubscribe<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AQYIHL4HK7OFRJUWQGZNDHDZSJRY3AVCNFSM6AAAAABKVBYQC6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJXGUYTKMRZHA__;!!KVu0SnhVq1hAFvslES2Y!NWW8QvxeSYn7ESmqylfPj34VBllQWnRMkEQGY7AcrDCJBfm0bwbRl6VuZ0uinjVI-oB_8a1xUnoritmwxBt2R_icD26jeA$>. You are receiving this because you authored the thread.Message ID: ***@***.***>