Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OMF zip files won't open in 7-Zip or Windows #5

Closed
Tim-Evans-Seequent opened this issue Jun 18, 2024 · 1 comment · Fixed by #6
Closed

OMF zip files won't open in 7-Zip or Windows #5

Tim-Evans-Seequent opened this issue Jun 18, 2024 · 1 comment · Fixed by #6
Assignees
Labels
bug Something isn't working

Comments

@Tim-Evans-Seequent
Copy link
Contributor

Tim-Evans-Seequent commented Jun 18, 2024

The zip files created by OMF have incorrect CRCs on the files, which stops some tools from listing the contents and extracting files. We should be creating valid zip files. This may be caused by not using the zip crate correctly.

Running unzip -t tests/one_of_everything.omf from the command line shows these errors:

Archive:  tests/one_of_everything.omf
Open Mining Format 2.0-beta.1
    testing: 1.parquet                bad CRC 6844d846  (should be 0148723f)
    testing: 2.parquet                bad CRC 37aff064  (should be 23729fb9)
    testing: 3.parquet                bad CRC 4d1031c7  (should be a0886785)
    testing: 4.parquet                bad CRC b73cfb57  (should be 156fef01)
    testing: 5.parquet                bad CRC 653bed3b  (should be 8229cafc)
    testing: 6.parquet                bad CRC ab9187d4  (should be 67322c54)
    testing: 7.parquet                bad CRC 26872cb7  (should be 0be6ada5)
    testing: 8.parquet                bad CRC 2bc7f3c3  (should be 0148723f)
    testing: 9.parquet                bad CRC c749eab2  (should be 7053f08a)
    testing: 10.parquet               bad CRC b9f234d0  (should be 0da21a9b)
    testing: 11.parquet               bad CRC 6e7048c5  (should be 1667d319)
    testing: 12.parquet               bad CRC e762130b  (should be 0e66161b)
    testing: 13.parquet               bad CRC 905b54de  (should be 998a990b)
    testing: 14.parquet               bad CRC 9d3bd9a1  (should be c8017148)
    testing: 15.parquet               bad CRC 9819aec8  (should be 0148723f)
    testing: 16.parquet               bad CRC ac19d6d9  (should be 550db371)
    testing: 17.parquet               bad CRC ae240291  (should be 8bb80863)
    testing: 18.parquet               bad CRC 73fadcff  (should be 6c3c50fb)
    testing: 19.parquet               bad CRC 11e6fe61  (should be 5424f9df)
    testing: 20.parquet               bad CRC e89e92ee  (should be bdb37e8a)
    testing: 21.parquet               bad CRC 16638b68  (should be e867eab4)
    testing: 22.parquet               bad CRC fbba9f16  (should be 3013111b)
    testing: 23.parquet               bad CRC d0db62d3  (should be 3013111b)
    testing: 24.parquet               bad CRC 9fb5f14a  (should be d9a8d581)
    testing: 25.parquet               bad CRC 52b1a144  (should be 1103927f)
    testing: 26.parquet               bad CRC 5e8ec0f7  (should be 9784fbf5)
    testing: 27.parquet               bad CRC 99e84489  (should be 4239c7dc)
    testing: 28.parquet               bad CRC 1dc447b4  (should be 4675cb82)
    testing: 29.parquet               bad CRC 780e1e7a  (should be 4239c7dc)
    testing: 30.parquet               bad CRC e7787b79  (should be 378d86b0)
    testing: 31.parquet               bad CRC f59d5f0e  (should be c0e9125c)
    testing: 32.parquet               bad CRC 91469288  (should be 88e43aad)
    testing: 33.png                   bad CRC a0eb26a7  (should be 6d7a2113)
    testing: 34.png                   bad CRC fda6be3d  (should be 6d7a2113)
    testing: 35.parquet               bad CRC 32a80f0e  (should be f86c88c5)
    testing: index.json.gz            bad CRC b9d3b699  (should be e9a30c8c)
At least one error was detected in tests/one_of_everything.omf.
@Tim-Evans-Seequent Tim-Evans-Seequent added the bug Something isn't working label Jun 18, 2024
@Tim-Evans-Seequent Tim-Evans-Seequent changed the title OMF zip files won't open in Z-Zip or Windows OMF zip files won't open in 7-Zip or Windows Jun 18, 2024
@Tim-Evans-Seequent Tim-Evans-Seequent self-assigned this Jun 18, 2024
@Tim-Evans-Seequent
Copy link
Contributor Author

Tim-Evans-Seequent commented Jun 18, 2024

This appears to be a bug in the zip crate, and only occurs when using ZIP64 support. I believe OMF will at times require large files, so ZIP64 support can't be omitted here.

I have filed zip-rs/zip2#195 upstream, and for now we can work around the bug by using an older version. We can switch back to the latest version once the upstream bug is fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant