You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I just noticed that the production config was setting prune to true. I think the problem is relatively contained and I implemented a really janky overwrite to fix it right now.
We should think more generally how to handle overwriting a store here.
Currently if a store has been written at all, it is not reattempted. This means if there was some small bug like the one above (which apparently also impacted #72, see #98 for manual solution attempt) these would not be automatically overwritten.
There are a few ways to handle this:
Rerun all stores that fail the QC. This would be pretty easy to implement (by modifying the filtering logic here to only remove the datasets that are passing the quality control). Combined with regular retesting (and expanding tests as new bugs are discovered this would presumably take care of this issue over time. But this is really expensive! I assume we will have many datasets that are simply 'broken' at the moment and might never pass the tests. We would rerun these again and again creating a lot of cost.
Alternatively we could implement a 'one-time' trigger to overwrite all or a set of iids. Currently I have added selected iids to the recipe directly, but they will be rerun every time the recipe is run, which is not ideal. Curious to discuss this.
The text was updated successfully, but these errors were encountered:
jbusecke
changed the title
We need a way to reliably trigger a new write of a given iid
How to overwrite. We need a way to reliably trigger a new write of a given iid
Feb 12, 2024
I just noticed that the production config was setting prune to true. I think the problem is relatively contained and I implemented a really janky overwrite to fix it right now.
We should think more generally how to handle overwriting a store here.
Currently if a store has been written at all, it is not reattempted. This means if there was some small bug like the one above (which apparently also impacted #72, see #98 for manual solution attempt) these would not be automatically overwritten.
There are a few ways to handle this:
The text was updated successfully, but these errors were encountered: