Implement PEP 639: support for Metadata-Version: 2.4; Part II

[agriyakhetarpal]

Description

This PR comes in continuation of #24 and resolves a few prior shortcomings in the implementation. Here's a summary of the changes:

• The licenses are now copied into a ziglang-0.X.Y.dist-info/licenses/ folder with their contents intact and the original structure of the licenses in place, i.e., ziglang/lib/libc/mingw/COPYING gets copied into ziglang-0.X.Y.dist-info/licenses/ziglang/lib/libc/mingw/COPYING and so on. This does generate duplicate license files, but I tested this by creating a dummy package using Hatchling as the build backend, which has support for PEP 639, and it does the same thing. Plus, text does compress well, of course. Also,

  there's an issue related to the file paths: the paths to the license files are supposed to be relative to the source folder (ziglang/ in this case)

  This is no longer an issue, since the paths to the license files are correct indeed (wheels generated by Hatchling preserve the ziglang/ folder).

• A simple warning is printed to stdout in the case of license files that go missing when copying – this would be useful in case the Zig source upstream changes the license structure by moving/deleting one or more license files, since we are currently hardcoding the filenames and their paths. The current license structure is accurate for Zig 0.13.0. What it can't do is detect new license files that are not listed in the metadata. That's difficult to do here, but I'm open to thoughts. A pre-commit hook of sorts could be implemented but I need to think about how it would work. We could also write a simple release checklist (release_checklist.md) that lists a few things to manually verify before pushing wheels?

• I also ran black on the file in 4573248 to clean up the code quality and style a bit, so this PR is best reviewed with diffs for whitespace changes turned off. But if it doesn't seem effective, we can drop it, too.

Additional information

The wheel has not included the LICENSE.txt file for this repository in general in previous wheels uploaded to PyPI, so I've decided to not include it here (my understanding is because it's a license for the make_wheels.py script and not the contents of the wheels). The main license file from the Zig source that accredits the Zig contributors, i.e., LICENSE, continues to remain included through ziglang/LICENSE.

[agriyakhetarpal]

Ready for review, @whitequark! We should be able to publish wheels with this, too, (such as #25), now that PyPI has support. I think it should all work, but twine may have some troubles with validation until pypa/twine#1123 is resolved (which can be managed by reinstalling pkginfo>=1.11 manually). This was the case the last time I tried. Installing wheels should have no issues, since the PEP mentions, "end-user-facing install tools MAY be less strict than mentioned here when encountering malformed metadata". I tried with both pip and uv and faced no problems with them.

[whitequark]

Okay, could you please not reformat the entire codebase in a functional PR?

[agriyakhetarpal]

Yes, reverted it, the diff should be a lot cleaner now. I'll move that to a different/dedicated PR for adding pre-commit hooks if needed.

[whitequark]

I don't really use autoformatters in any of my codebases, so I'd rather avoid that.

[whitequark]

What about going through all files in the tarball matching the regex /license/i and warning if any of them are missing from license_paths?

[agriyakhetarpal]

I thought about it, it's not going to be robust enough to cover cases where the license is called "COPYRIGHT" or "COPYING". I left it alone after that, but a partial check sounds good to me.

[whitequark]

What about /\b(license|copyright|copying)\b/i?

[agriyakhetarpal]

Sounds good!

[agriyakhetarpal]

Addressed in 67b9200. I added a larger pattern and the regex caught two more files – added them to the list! :)

[whitequark]

That's how you know it was working :D

[whitequark]

LGTM, please squash the commits.

[agriyakhetarpal]

Thanks for the review! Done.

[whitequark]

Thanks!

[agriyakhetarpal]

You're welcome! Could you please try to upload new wheels to PyPI with this change? That can help check in case we need to change anything else.

[whitequark]

I can't:

$ twine upload dist/*
Uploading distributions to https://upload.pypi.org/legacy/
ERROR    InvalidDistribution: Metadata is missing required fields: Name, Version.
         Make sure the distribution includes the files where those fields are specified, and is using a supported Metadata-Version: 1.0, 1.1, 1.2, 2.0, 2.1,
         2.2, 2.3.

[whitequark]

Ah okay, this solved it:

$ pipx inject twine 'pkginfo>=1.11' --force

[agriyakhetarpal]

Yes, thanks! I've cross-linked the issue related to this above. It would be great if you could build and publish some of the ppc64le wheels for recent versions so that I can start using them :)

[whitequark]

Actually no, I still can't:

$ twine upload dist/*
Uploading distributions to https://upload.pypi.org/legacy/
/home/whitequark/.local/share/pipx/venvs/twine/lib/python3.12/site-packages/pkginfo/distribution.py:175: NewMetadataVersion: New metadata version (2.4) higher than latest supported version: parsing as 2.3
  warnings.warn(NewMetadataVersion(self.metadata_version))
Uploading ziglang-0.13.0.post1-py3-none-macosx_12_0_arm64.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 81.3/81.3 MB • 00:24 • 9.7 MB/s
WARNING  Error during upload. Retry with the --verbose option for more details.
ERROR    HTTPError: 400 Bad Request from https://upload.pypi.org/legacy/
         'description-content-type' must be one of ['text/plain', 'text/x-rst', 'text/markdown'], not "'text/markdown'; charset=UTF-8; variant=GFM". See
         https://packaging.python.org/specifications/core-metadata for more information.

[agriyakhetarpal]

I added this from the specs mentioned in the Python Packaging Guide; it looks like the information is incorrect therein in that case... I opened a quick fix in #28.

[whitequark]

Okay, should be uploaded now: https://pypi.org/project/ziglang/0.13.0.post1/#files

[agriyakhetarpal]

Thank you so much, @whitequark!