Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drugs@FDA data set is inconsistent #261

Open
sho13 opened this issue Jul 16, 2024 · 14 comments
Open

Drugs@FDA data set is inconsistent #261

sho13 opened this issue Jul 16, 2024 · 14 comments

Comments

@sho13
Copy link

sho13 commented Jul 16, 2024

Hello!

Just attempted to search for drugs that were previously existing on the Open FDA API for Drugs -

Drugs@FDA [/drug/drugsfda]
This endpoint's data may be downloaded in zipped JSON files. Records are represented in the same format as API calls to this endpoint. Each update to the data in this endpoint could change old records. You need to download all the files to ensure you have a complete and up-to-date dataset, not just the newest files. For more information about openFDA downloads, see the API basics.

There are 1 files, last updated on 2024-07-16.

As you can see it's been updated today on July 16th, 2024, but there's inconsistencies with the accessdata website

For example - Eylea HD and Beovu are nonexistent in the API but available on the accessdata website

Is this discrepancy a bug?

Thanks!

@ketkijsane
Copy link
Collaborator

openFDA and AccessData do not share a release cycle, so there may be differences in data between the two.

@outstandy
Copy link

@ketkijsane the new release of openFDA is now missing data. We've found at least 4 drugs that were present in the previous release that are not now, and none of them have had a change in their approval or marketing status as far as we can tell. Do you know why they might have been removed?

@ketkijsane
Copy link
Collaborator

Thank you @outstandy for your response. Can you please provide more information about what endpoint you are hitting and which specific drugs you have found that are missing? Thanks!

@ketkijsane ketkijsane reopened this Jul 17, 2024
@sho13
Copy link
Author

sho13 commented Jul 17, 2024

Hi @ketkijsane the endpoints that have been hit are - the https://api.fda.gov/drug/drugsfda.json?search=openfda.brand_name end point and the drugs are -

Eylea, Beovu, Vabysmo and Zolgensma. Eylea previously had two results (Eylea and Eylea HD) Beovu, Vabysmo and Zolgensma all previously had a single result

Again is this discrepancy a bug or have these drugs been pulled?

@ketkijsane
Copy link
Collaborator

ketkijsane commented Jul 18, 2024

Thank you @sho13 for your response. We've investigated the raw data files which serve as our data source and found that the data in openFDA matches what was most recently released: https://www.fda.gov/drugs/drug-approvals-and-databases/drugsfda-data-files.
Please let us know if this looks good.
Thanks!

@sho13
Copy link
Author

sho13 commented Jul 18, 2024

hey @ketkijsane thank you for investigating! I can confirm your investigation of raw data files and data are aligned, however the confusion is that it's missing drugs from the Purple Book (if you search for all the drugs mentioned above on purple book, they are still present),

Do you know if the FDA has pulled these from the data set?

Thanks!

@sho13
Copy link
Author

sho13 commented Jul 18, 2024

@ketkijsane a follow up, in the raw data set I was able to find the SubmissionPropertyType and Submissions via Application number for the drug Beovu - 761125, there is not Product for it though, the other three drugs in question only Eylea existed but not the High dose version.

@ketkijsane
Copy link
Collaborator

ketkijsane commented Jul 18, 2024

@sho13 , thank you for the confirmation. Unfortunately, we don't have any information on how the FDA puts together the source data files. The Drugs@FDA team would know better for that data. Not having a listing in the product source file is why there's no results for "brand_name", as that field is populated from the product file.
Please let us know if this sounds good.
Thanks!

@sho13
Copy link
Author

sho13 commented Jul 18, 2024

@ketkijsane thanks, I would love to reach out to the appropriate team- do you know the best way to reach the Drugs@FDA team?

@ketkijsane
Copy link
Collaborator

https://www.fda.gov/drugs/drug-approvals-and-databases/drugsfda-data-files has a link at the top with an email address.

Thanks!

@sho13
Copy link
Author

sho13 commented Jul 18, 2024

Thank you @ketkijsane , but that link is an a tag

<a href="mailto:?subject=Contact%20FDA&amp;body=https://www.fda.gov%2Fabout-fda%2Fcontact-fda" class="lcds-share__btn lcds-share--default__btn-mail"><span class="fa icon-envelope" aria-hidden="true"></span>Email</a>

That doesn't send an email to anyone in Drugs@FDA, it fills my subject and body.

Is there any direct contact you can provide me with, thank!

@ketkijsane
Copy link
Collaborator

ketkijsane commented Jul 19, 2024

@sho13, we're looking for contact info now and will let you know as soon as we have it. Please note that there may be a delay due to the global tech outage. Apologies for any inconvenience.

Thanks!

@sho13
Copy link
Author

sho13 commented Jul 24, 2024

Hello @ketkijsane i see that the dataset's been updated yesterday but it's still missing the drugs as mentioned... would you know if this issue has been resolved and these drugs are just pulled permanently from the FDA?

@ketkijsane
Copy link
Collaborator

Hello @ketkijsane i see that the dataset's been updated yesterday but it's still missing the drugs as mentioned... would you know if this issue has been resolved and these drugs are just pulled permanently from the FDA?

Hi @sho13, that would also be a question for the Drugs@FDA people since we had no knowledge about inclusion of specific results in the source data.
We are still looking for a contact information and will let you know as soon as we have it.

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants