Meta model load command ALL NEW/NEW #, other adjustments

[tazlin]

As requested by @db0 as a result of @Unit1208's recent attention (which we are very thankful for) to the image model reference repository, more models have been added at an increased rate than historically.

The problems

• Workers may select commands like TOP 10 and never offer models new to the horde or outside of a very limited pool of already established models.
• Workers may select commands like ALL, download 600+gb, only to find that within a few days or weeks, they now have to download 40gb upon starting their worker up, and either not have the budgeted space or may not be willing to dedicate hours to downloading (which also relates to a long standing issue).
• Users continue to use popular models more often due to better response times and worker availability

These issues potentially cause the following issues that need addressing:

• Established models with wide appeal are offered more often than new-to-the-horde models, with causes on both the user-side and worker-side.
• New models are often not available shortly after being officially allowed
• Worker operators who deliberately offer few models have to stay apprised of image model reference changes if they want to offer new models.
• Worker operators may suddenly need to download many gigabytes of data, which they either may not have room for, may not have budgeted the time for, or may simply want to forgo at the moment.

Proposed solution

Therefore, in response, I propose the following:

• Changes to the image model reference
  • The workflow for release should be adjusted to either generate a new json file of the models in stable_diffusion.json with the following information:
    • date_added
    • date_removed
    • date_modified
• New model meta load instructions
  • ALL NEW
  • # NEW / NEW #
  • NEW SDXL #
• Possible new other configuration options
  • avoid_new_models / avoid_new_models_day_threshold
    • For workers who use model meta load commands such as ALL or TOP, this being on should prevent a model within the avoid_new_models_day_threshold from being downloaded/offered
    • Models skipped over in this way should be included as a warning message to the operator at start and while the worker is running to give them a heads up of the incoming download and the download sizes.
    • This will give worker operators more control over when to opt-in to a download.
  • freeze_models_to_load
    • If true, load the latest resolved model list when the worker was actually running (after resolving all meta load commands), ignoring what the current state would be.
    • This would require the worker to store the last loaded models list somewhere
    • This hopefully should allow workers who find that the model load resolution has changed but do not currently want to download them and want to just run the latest stable config

It is worth pointing out that certain worker operators have explicitly requested some or all of these features.