2025-01-17 22:29:21 UTC - Monitor incident - Missing PEGA status

[michaelclement]

Per playbook, opening ticket to track details of this monitor alert in slack.

The affected UUIDs:

Form UUID	Uploaded at
011df56a-91ef-40d8-a92e-86a984683bc0	2025-01-17 22:29:21 UTC

Intake

• PM (Bo) and DM (Andrea) have been notified
• PM and DM confirmation of ticket creation
• Incident ticket linked to sprint monitoring ticket

Investigation

• Verify that vets-api is receiving submits from vets-website
• Verify that files are sent successfully to s3
• Verify that the callback API is available
• Determine whether the issue arose due to a deployment, either by the IVC Forms team or the PEGA team.
  • IVC
    • If root cause has been identified as an issue in vets-website or vets-api, notify PM and DM to determine next stpes with PO
  • PEGA
    • If root cause has been identified as an issue outside of vets-website and vets-api, work with the PM, DM, and PO to determine contacts and next steps.

Resolution

• Acknowledge monitoring alerts in slack
• IVC Forms engineering will provide updates on development and deployment of a solution and its status in each environment
• IVC Forms engineering will notify PM and DM when resolved
• IVC Forms engineering will monitor outstanding missing statuses and ensure they are cleared appropriately. No statuses should be deleted or cleared without agreement from PO and or PM
• If traffic to the site has been altered, PO or PM will determine when to return to previous levels.