Test the delta feature on COSMOS for robustness

[bishwaspraveen]

Description

Test the backend and frontend delta feature integration on COSMOS thoroughly.

Implementation Considerations

The initial design doc has been saved here for reference.
Is this the right time to start writing tests for our codebase?

Deliverable

The new delta feature integrated and tested out on COSMOS for bugs.

Local Testing Specifics

• 1. choose a small collection with ~10 urls
• 2. make a title pattern for multiple, etc
• 3. change the scraped_title for one of them, so it registers as a change
• 4. delete a deltaurl of them, so it registers as an add on our second run
• 5. add a fake one so it registers as a delete
• 6. run the import a second time, expect everything to work as expected

Additional Testing

• import a collection from LRM dev
• migrated it and what not
• curate it, add new rules ect.

[CarsonDavis]

@dhanur-sharma @CarsonDavis we need to write a quick testing procedure for emily and nidhi, as well as get a list of collections with content on lrm-dev.

[dhanur-sharma]

Screen recording of a positive happy flow where URLs are pulled in from LRM dev through the reindexing status change, adding some title patterns, and deleting Curated URLs as well upon merging with Delta URLs that are set to be deleted.