You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe
Default score threshold values are read from metadata.json which is loaded from the config.yml's. When loaded, we look at the root thresholds node which is a list of numeric values (unsorted)
This means when we plot model performance, we just sort these high to low, and treat the highest threshold as "Red" and the second highest as "Orange", and we don't provide a name for the threshold like "Telephone Call", "Rapid Response", or "Screening"
This also means that we can only sport one global threshold, even though we could support more than one model output (score)
Describe the solution you'd like
Ideally metadata.json is improved to support different thresholds for different score names (if a model can predict more than one score) and different thresholds for different model outputs.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe
Default score threshold values are read from
metadata.json
which is loaded from theconfig.yml
's. When loaded, we look at the rootthresholds
node which is a list of numeric values (unsorted)This means when we plot model performance, we just sort these high to low, and treat the highest threshold as "Red" and the second highest as "Orange", and we don't provide a name for the threshold like "Telephone Call", "Rapid Response", or "Screening"
This also means that we can only sport one global threshold, even though we could support more than one model output (score)
Describe the solution you'd like
Ideally
metadata.json
is improved to support different thresholds for different score names (if a model can predict more than one score) and different thresholds for different model outputs.The text was updated successfully, but these errors were encountered: