Rules affecting large url counts produce a bad gateway

[CarsonDavis]

Description

Rules affecting large url counts produce a bad gateway

Implementation Considerations

• kick these over to celery and have it be async?

Example

check-https://sde-indexing-helper-staging.nasa-impact.net/1355/delta-urls
Pattern “ascl.net/” to document

[CarsonDavis]

Has been tested to break on 14,000 but work on 500. This is definitely a potential problem, but based on curator feedback, if you are making a pattern for 14k urls, you might just need to change the collection default. Lets make this a priority in 25.2