Bug fixes and rubocop cleanup #5
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
If the garbage collector is called prior to there being any images in the trow registry, the stat call will cause a RuntimeError. Adding --no-run-if-empty ensures the command completes successfully.
If there are no manifests we generate the following errors: tar: Cowardly refusing to create an empty archive Try 'tar --help' or 'tar --usage' for more information. command terminated with exit code 123 tar: This does not look like a tar archive tar: Exiting with failure status due to previous errors I, [2022-03-21T20:10:18.349339 #1] INFO -- : No JSONs to retrieve. Skipping Skip calling tar if no manifests are found.
The delete_orphaned_blobs method was raising RuntimeErrors as it was trying to remove files that were deleted in previous iterations of the main_loop. Altered the Repository orphaned_blobs method to be private and added a filenames_to_delete method that yields the filenames to delete, then uses a delete_if on the blobs instance variable to remove the successfully deleted files from the hash.
split will split strings at each space by default
As the GC configurable options were driven by environment variables, added support for a DRY_RUN variable.
Although unlikely, as the MIN_GC_BLOB_AGE is in seconds, it is possible that the blob count and blob sum could be based on different hashes as the v.mtime < time_cutoff component of the filter could be different between the calls. Use a tap to ensure consistency.
To ease reading the summary information, add delimiters to the blob count and size numbers.
The command pipelines which are used in `#fetch_manifests` and `#fetch_jsons` were previously not checking for any command failures. If `#fetch_jsons` fails but continues the rest of the routine, it means that blobs that are actually still in use will remain as candidates for garbage collection. This appears to be the cause of images being prematurely garbage collected and appearing as 'failed to solve' errors when trying to use the images during builds. By ensuring we raise an error if one of the pipeline commands fails, it will ensure the GC routine does not continue.
Ensure errors are raised for any pipeline failures
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We've seen a variety of errors and runtime warnings whilst using the collector on trow repos in various states. This PR is a rollup of various atomic commits to fix the issues seen. (More than happy to drop any that you're not interested in merging of course).