Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

instead of sending audit outcomes to workflow service, just expose status via REST endpoint #1249

Closed
2 tasks
jmartin-sul opened this issue Dec 2, 2019 · 5 comments

Comments

@jmartin-sul
Copy link
Member

reporting audit results to workflow server was always a hack for development expediency, because we thought it'd be a faster way (in terms of development time) to expose audit results to argo users. however, this mechanism has been brittle in a number of unexpected ways, and it seems like it would be better to:

  • expose the relevant status info for CompleteMoabs and ZipParts (and whatever else) via REST (e.g. caller could hit pres cat with a given druid, and relevant status info etc for Moab(s) and archive copies could be returned).
  • as it does with workflow service, the dor-services code that dor_indexing_app uses to build the solr doc could hit pres cat and index e.g. moab status and some sort of summary archive status (e.g. archived, unreplicated, some_parts_not_found, etc).
    • downside: one more thing dor_indexing_app has to call out to
    • upside: reduced fragility from the pres cat side, because there's one less thing (workflow server) that it has to call out to.
    • upside: pres cat no longer has to know about workflow service. less circular dependency at the service level (the argo/DOR side already knows about preservation -- this would be one less argo/DOR side thing preservation has to know about)
    • upside: allows for more nuanced indexing/faceting/searching in argo

to help reduce worry about breaking the existing reporting functionality, we could implement the indexing, let the two approaches live side by side for a bit, and then get rid of the workflow reporting.

codebases that this would touch:

  • pres cat: new controller method to expose the above described info
  • dor-services: where the new indexing code that hits pres cat would live
  • workflow services: to get rid of the preservationAuditWF

doing this would obviate the need for #1240, since we'd no longer need the workflow reporting mechanism at all.

@ndushay ndushay changed the title instead of sending audit outcomes to workflow service, just expose status via REST endpoint, and let it get indexed instead of sending audit outcomes to workflow service, just expose status via REST endpoint Dec 9, 2019
@ndushay
Copy link
Contributor

ndushay commented Dec 9, 2019

only index it if we are doing something more than having this display for an object at API endpoint (part of /objects/druid or separately)

@ndushay
Copy link
Contributor

ndushay commented Dec 12, 2019

Note that however audit outcomes are exposed, we want the DETAILS of the audits -- WHY did it fail?

This might be best handled by adding a status_details field to complete_moab and to zip_parts. Note that field size is finite, and we've run into zip_part audits sending too much text to workflow-service, so we would want to avoid this #1240. It might be addressed by having each zip part get its own status details.

  • audit_results would update these fields when necessary
  • ReST API call to prescat would get info from all these places and combine it
    • this info could be added to objects/show result?

@ndushay
Copy link
Contributor

ndushay commented Jan 7, 2020

I just barely started this work. It is is the audit-status-in-db branch.

@ndushay
Copy link
Contributor

ndushay commented Feb 13, 2020

This may be superseded by writing info to event service (#1357) and plans for reporting out errors detailed in #1363

@jmartin-sul
Copy link
Member Author

closing -- i think #1363 covers this ground better

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants