Make the content of the PPMS trainings array available via vets-api

[kmariepat-cityfriends]

Background

There is an array from PPMS that includes information about providers that have completed the provider core training. We need to make that array consumable by vets-api so that it can be used on front end views. This work aligns to the updates being made to facility locator in order to identify providers that have completed the training on the front end.

Example data

Queries that successfully return trainings are available here:
#11627 (comment)

Acceptance Criteria

• I expect that the vets-api can return the values of the trainings array in the ProviderLocator end point
• I expect that if possible this is an optional request
• I expect that vets-api console access is verified
• I expect that documentation in https://github.com/department-of-veterans-affairs/va.gov-team/blob/master/products/facilities/facility-locator/data/ppms/ppms-data-discovery.md is updated to modern state of relevance after these changes, in any location that references PPMS, API data sources, or how filters connect to API endpoints
  • Add any learnings about use / data returned from facility-services endpoint
  • Add any new notes to Provider Locator section, regarding training data
  • Read / review the rest of the doc & update as needed
• I expect that tests are added to cover code changes

[kmariepat-cityfriends]

Will also need a front end card to update for facility locator (Only need to apply it under community providers in facility locator)

[jilladams]

@maxx1128 @kmariepat-cityfriends @ryguyk am I right that this card is Ruby, and is the one Ryan alluded to during planning?

[maxx1128]

@jilladams This is definitely a Ruby ticket, but I'm unsure if it's the one @ryguyk was talking about.

[ryguyk]

This is the ticket I mentioned.

[jilladams]

In sprint report for SPrint 85, we reported: "Endpoint is not public; working out access, WIP." More accurately, this should say " Endpoint is not returning expected results; working out access, WIP."

That was based on this thread: https://dsva.slack.com/archives/C0FQSS30V/p1685029342799809?thread_ts=1684962329.960919&cid=C0FQSS30V

I have a vets-api pull request up with a staging environment, and I need to confirm which PPMS endpoint is being called through it. The VA network lets me access the PIMS system which has endpoints for north, south, and east - I have access to the East one, but I am unsure if that’s the exact PPMS endpoint my staging environment is calling.

I need to confirm this since the East PIMS endpoint has the trainings attribute for provider data, but when I try to access it via vets-api , that attribute is returning empty when it shouldn’t. So I need to find out if the problem is either a) I’m not hitting the right PPMS endpoint that has trainings data, or b) the trainings data is there but I’m not accessing it properly through vets-api.

Previously, @omahane was able to find a provider with training data, but when @maxx1128 queries for that provider, the array is empty. So: Michelle is helping us connect to folks on the PPMS side and has reached out to Michael Johnson at the VA for help finding the right PPMS endpoint / permissions to sort this out.

[mmiddaugh]

@jilladams @kmariepat-cityfriends should we mark this as "blocked" pending assistance from the PPMS dev team?

[jilladams]

@mmiddaugh @kmariepat-cityfriends yes - also - as a sideline, @maxx1128 is pursuing vets-api console access (via department-of-veterans-affairs/va.gov-team#59791) and there's a slim chance that will tell us things we don't know right now about how to get at this data. So: I think we could mark it blocked, by either both of those things (PPMS dev team notes and/or console access.)

[jilladams]

from that email:

I have sent another message to try and get your answers but in the meantime. Provider trainings should be located on the Provider Locator endpoint. I have a screenshot of what the result should look like from the APIM end point.

[jilladams]

Lane was affected by the workflow integration with the Speedrun board. Moving back to In progress.

[mmiddaugh]

Found the following info posted in the PPMS Integration OFfice Hours Forum in Teams
[5/6/2022 8:47 AM] Snyder, Linda L. (GovCIO)

The following DWS endpoints will be updated with Sprint 16.0 to allow downstream partners to pull Provider Training Information. The updates are currently deployed to INT (NPROD)

• get /ProviderLocator
• get /Providers
• get /ProviderServices

The following Provider Training information will be provided in these updated endpoints:

• Course Name
• Course Code
• Course Completion Date
• Course Expiration Date

[davidmpickett]

@mmiddaugh Do you have a sense of if this is unblocked and @maxx1128 could move forward once he returns?

[jilladams]

(We need Max to try and confirm if that's true or not.)

[maxx1128]

I believe this will unblock the ticket but am trying to confirm it now.

[jilladams]

Slack thread from 7/11: https://dsva.slack.com/archives/C0FQSS30V/p1689090373754209

[davidmpickett]

@maxx1128 Can you add a short summary of what you accomplished this sprint and what the current status is going into next sprint?

[maxx1128]

@davidmpickett @omahane and I both reached out to the PPMS support team to find out if the Training data is available on the NProd (non-production) environment. They said they would open a support ticket for us to either confirm if the data was on NProd and show us where, or add an instance of the needed data for us. The ticket was opened roughly a week ago and we have not had any updates since.

Michelle also connected me to Mark Dewey, who used to be on the Facilities team. He looked at my current code and said it looks like it should get the data from the API, and agreed the best course of action is to confirm if it's on the API endpoint I'm hitting or not.

[jilladams]

Noting from 1:1 with Max:

• ACs updated for modernity
• Right now endpoint / and vets-api console access are working
• Part 1: expanding the data net to pull in more data from PPMS: easy
• Part 2: verifying that we can catch the right data -- we have to merge adding the data to vets-api, in order to then query vets-api to receive it back to make sure that worked:
  • isn't clear if it can be feature flagged
  • we don't think adding new data will break anything - it's not exposed in the FE. But: that means people may be able to find that data in the network response via dev tools. Do we care about this? (aka is it worth the effort to try and feature flag?)

[maxx1128]

I updated the pull request to hide the training data behind a feature flag, I believe I found a way to make it work. If the tests pass, I can try merging it and enabling the terminal instance's feature flag to check it. If that is not working, it may be best to merge it without the feature flag.

cc: @jilladams

[jilladams]

Since we got signoff from Platform, our PO, and our team re: merging without a feature flag, I have no concerns about going that route.

[jilladams]

@maxx1128 department-of-veterans-affairs/vets-api#13213 merged. What remains here to close out, from the ACs?

[jilladams]

Confirmed in planning: in Prod we have not yet been able to verify post-merge.

[jilladams]

From data Michelle provided, Max was hitting same endpoint / location and getting different results than expected. Got back the expected provider, but without training data.

Would need location + specialty for each provider. Max: Need to double check that location is present for each in the data provided.

Hitting staging endpoint. Max: Check production endpoint access. We think the dataset is pointing to records in production that may not contain the same data in staging.

Can we request that Prod data gets synced to Staging? Not likely to get traction.

[mmiddaugh]

To confirm, this is how results from the APIM endpoint should look

[mmiddaugh]

@maxx1128 please see above

[jilladams]

Per discussion with @mmiddaugh & @xiongjaneg (dms in slack, here), this should be considered paused until we can identify help from PPMS to work out the Staging vs. Prod data problems. Removing from Sprints til we hear more / pull it back in, and adding Blocked label, moving back to Stretch / Upcoming.

[jilladams]

From planning: noting that we have a meeting with PPMS next week (week of 10/2). Facility Locator FE is not necessarily a priority for right now, but confirming that the PPMS data exists where they say it exists is.

Per Max, the meeting could result in 2 paths:

1. Current work may be correct, we can pull the data from their indicated location. Done.
2. May find that our work doesn't expose the data where we are pointed to pull it from, and need to revise our approach.

[xiongjaneg]

received access, keys, and sample data so this should be unblocked, will review at pre-refinement to see if any updates are needed to this ticket

[jilladams]

IT'S HAPPENING!!

[jilladams]

Ticket = connect API to training data in PPMS.

• If training data isn’t available through the endpoint we are. using, there’s nothing we can do til it is available.
• We give the endpoint parameters like radius, provider specialty, and PPMS returns the providers. We should be getting ALL data for those providers.
• We would expect training data to be in that response, with an empty array, if the provider hasn’t done any trainings. But we don’t receive even an empty array.
• PPMS sends a raw response, and on our side, we reformat that response (rename properties, rearrange, etc.). The code on our side was saying: look for training data, and if it’s not there, show an empty array.

Contradictions:

• When we inspect providers that we know have training data, in the raw response, there is no Training property in the raw data.(Alicia Baker, eg).
• When we ping the endpoint through vets-api, PPMS doesn’t have trainings in the provider locator endpoint.
  • Christian can see it in the provider locator endpoint, when going through the API portal, not through vets-api - and gets back a list of providers with the training property in the response, some with trainings, some not.

Vets-api seems like the smoking gun.

Blockers:
We can’t look at vets-api config to see what endpoint URL is being used. This information is stored in a Secrets key in AWS. We have previously asked Platform for help and they were unable to help.

Next steps:

• Max to find the support ticket requesting help with the AWS secret from Platform, and share with Jill / Jane / Michelle to escalate and get it figured out.

[maxx1128]

This pull request has been merged for this sprint (95), and it just needs to be confirmed as working tomorrow in production. Once that's done, it can be closed.

[maxx1128]

I couldn't wait and did a quick confirmation at this production API endpoint. The training data is coming through! cc: @jilladams @mmiddaugh

[maxx1128]

Closing for end of Sprint 95 since the data is confirmed available in Production.