Review exception handling for Platform-owned Sidekiq jobs [Part 1 of 2]

[rmtolmach]

User Story

As a platform backend dev, I want to know that all of the Sidekiq jobs we own are functioning properly, so that the Platform is stable.

Issue Description

As a follow-up to the review of Sidekiq jobs done in #94632, we need to go a little deeper into the low-priority jobs to review them for error handling. Use the sidekiq-exception-handling-review board (filtered by owner-(team):Platform) and go through the jobs labeled Low and review the four listed in the Tasks section for the following:

1. What does the job do?
2. Is the job still needed?
3. Answer the questions in the "Draft" for each job (where applicable).
4. Does the job need exception handling added?
5. Any other modifications needed?

Add findings to the Draft for each job and then move each into the "Assessment Review Complete" column. For any that need work done, click "Convert to issue" on the Draft and add our team label.

Tasks

• Review the exception handling and answer the questions listed above in the Draft for each of the platform-owned jobs:
  • ExportBreakerStatus
  • ExternalServicesStatusJob
  • FeatureCleanerJob
  • PagerDuty::CacheGlobalDowntime
• Convert to issue any that need more work done.

Success Metrics

Platform-owned Sidekiq jobs are audited for necessity and error handling. Tickets are created for future work.

Acceptance Criteria

• The four jobs listed above have been reviewed and filled out in the Sidekiq board.
• Any that need more work are converted to Issues.
• Any that don't need more work are moved to the "Action Items/Review Completed" column.

---

Validation

• Check Sidekiq board and review the three issues in "Action Items/Review Completed" column.

[AshleyGuerrant]

Standup note: Breaking this ticket up into 2 tickets, with 4 classes per ticket. Tracking part 2 in this ticket.

[rmtolmach]

Progress so far:

• FeatureCleanerJob done. Action needed: none for now. This job doesn't run. Some day if/when we ever want to spend time investigating the bug that makes this job a liability, we can make a ticket to re-vamp it.
• PagerDuty::CacheGlobalDowntime done Converted the draft to an issue to cover the extra error handling necessary. One small cleanup done in https://github.com/department-of-veterans-affairs/vsp-infra-application-manifests/pull/3309.
• ExternalServicesStatusJob. In progress. This was all part of an external service status page which no longer exists (AFAIK. Will need to confirm). If that's the case, I'll convert the draft to a ticket to rip it all out! tbd, though.

[rmtolmach]

I merged https://github.com/department-of-veterans-affairs/vsp-infra-application-manifests/pull/3309 as a small fix to an error I found while working onCacheGlobalDowntime.

[rmtolmach]

✅ converted ExternalServicesStatusJob to an issue. It needs some error handling.

Moving on to the last one! ExportBreakerStatus

[rmtolmach]

✅ Converted ExportBreakerStatus to an issue. It needs some more investigation and a few modifications.

Moving to In Review

[LindseySaari]

Follow up ticket looks great!

[LindseySaari]

Reviewed the board and ✔