♻️ Refactor task queue implementation and usage within Control Panel

[michaeljcollinsuk]

Proposal

The way that celery tasks are called are unusual, and could be changed to be closer to the recommended way in the documentation. Some options to consider

• Call tasks directly with Celery as per documentation
  • Look at Django-Q2 to consider benefits of using this instead. Although this is not preferred as would require more work.
• Remove custom message broker classes (and related code)
• Install celery flower to help with monitoring
• Improve the Tasks List UI page
  • Better ordering
  • Add filters
  • Update displayed details
  • Add pagination

Definition of done

• Above options considered and the way that tasks are called has been refactored if necessary
• The "Tasks List" UI has been updated to be more user friendly
• Celery flower has been installed for better monitoring (if celery remains)

[github-actions]

This issue is being marked as stale because it has been open for 60 days with no activity. Remove stale label or comment to keep the issue open.

[github-actions]

This issue is being closed because it has been open for a further 7 days with no activity. If this is still a valid issue, please reopen it, Thank you!

[github-actions]

This issue is being marked as stale because it has been open for 60 days with no activity. Remove stale label or comment to keep the issue open.

[github-actions]

This issue is being closed because it has been open for a further 7 days with no activity. If this is still a valid issue, please reopen it, Thank you!