Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure that collated subdirectories are archived deterministically #1150

Open
Donaim opened this issue Sep 4, 2024 · 2 comments
Open

Ensure that collated subdirectories are archived deterministically #1150

Donaim opened this issue Sep 4, 2024 · 2 comments

Comments

@Donaim
Copy link
Member

Donaim commented Sep 4, 2024

Currently, even if every output file of the main pipeline is not changed between multiple reruns, when we archive coverage maps, the archives themselves will be different in terms of their binary contents.

This is likely due to the fact that our archival program, tar, is capturing timestamps (ctime, mtime, atime) and stores them in the archive.

As the result, it is always true that Kive reports a change between reruns, even if they produced identical results.

For the fix, we should ensure that tar behaves time-independently.

@donkirkby
Copy link
Member

Another option is to deploy the next release of Kive, so we can have output directories, instead of only output files. That way, we won't need any tar or zip files.

I can't remember if we also had a problem with the PDF files' time stamps, or if we used a constant timestamp to avoid the problem.

@Donaim
Copy link
Member Author

Donaim commented Sep 4, 2024

@donkirkby yes, PDF files have the same issue. See container_runs/1408362/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants