py3-cassandra-medusa: pin to python3.11, use wolfi deps where possible, restoring & multiversioning ssh python libs #38209
+355
−40
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dannf
added
the
approved-to-run
octo-sts
bot
added
the
bincapz/pass
dannf
force-pushed
the
py3-cassandra-pin-python+use-systemlibs-for-security
branch
from
201edc0 to
e073f2b
Compare
No functional change. Signed-off-by: dann frazier <[email protected]>
It builds and tests fine w/o this. Signed-off-by: dann frazier <[email protected]>
Remove the dependency on /usr/bin/python3 and /usr/bin/pip being the same version of python that we are using by using the versioned binaries. If an image needs python3 to be python3.11 for compat reasons, then it should explicitly install python-3.11. Signed-off-by: dann frazier <[email protected]>
Replace dependencies that py3.11-build-base provides with py3.11-build-base. Signed-off-by: dann frazier <[email protected]>
Otherwise apk will resolve the dependency using provide priorities, which currently the python 3.13 version will win. That drags in an entirely new python environment that we don't need. Signed-off-by: dann frazier <[email protected]>
Signed-off-by: dann frazier <[email protected]>
`sed -i` will replace a symlink with a full copy. Only run it on regular files. This should make sure we're always running the latest patched interpreter, as well as decrease package and image sizes. Signed-off-by: dann frazier <[email protected]>
Without this, our compat symlink dangles. Signed-off-by: dann frazier <[email protected]>
…arm64 psutil, and possibly other module dependencies from PyPI, are not pre-compiled for arm64, so we need the C build environment. Signed-off-by: dann frazier <[email protected]>
We are relying on system-installed poetry at runtime, so let's also use it at build-time to reduce external dependencies. Signed-off-by: dann frazier <[email protected]>
ci-cve-scan currently fails with the errors below[*]. This is a result of us installing pinned versions of dependencies from PyPI. Most of these packages are available in wolfi, and using the wolfi packages would keep us up to date with the latest upstream fixes. Since we no longer need to modify pins, we can build with the py/pip-build-install pipeline instead of poetry. [*] ├── 📄 /home/cassandra/.venv/lib/python3.11/site-packages/pip/_vendor/vendor.txt │ 📦 certifi 2023.7.22 (python) │ Low CVE-2024-39689 GHSA-248v-346w-9cwc fixed in 2024.07.04 │ 📦 idna 3.4 (python) │ Medium CVE-2024-3651 GHSA-jjg7-2v4v-x38h fixed in 3.7 │ 📦 requests 2.31.0 (python) │ Medium CVE-2024-35195 GHSA-9wx4-h78v-vm56 fixed in 2.32.0 │ 📦 setuptools 68.0.0 (python) │ High CVE-2024-6345 GHSA-cx63-2mw6-8hw5 fixed in 70.0.0 │ 📦 urllib3 1.26.17 (python) │ Medium CVE-2024-37891 GHSA-34jh-p97f-mpxf fixed in 1.26.19 │ Medium CVE-2023-45803 GHSA-g4mx-q9vg-27p4 fixed in 1.26.18 ├── 📄 /home/cassandra/.venv/lib/python3.11/site-packages/virtualenv/seed/wheels/embed/setuptools-68.0.0-py3-none-any.whl │ 📦 setuptools 68.0.0 (python) │ High CVE-2024-6345 GHSA-cx63-2mw6-8hw5 fixed in 70.0.0 └── 📄 /home/cassandra/.venv/lib/python3.11/site-packages/virtualenv/seed/wheels/embed/setuptools-69.5.1-py3-none-any.whl 📦 setuptools 69.5.1 (python) High CVE-2024-6345 GHSA-cx63-2mw6-8hw5 fixed in 70.0.0 Signed-off-by: dann frazier <[email protected]>
…lfi-dev#26270)" This reverts commit a5e4968. Restore these dependencies for py3-cassandra-medusa now that we're using wolfi-python-deps again. Signed-off-by: dann frazier <[email protected]>
Signed-off-by: dann frazier <[email protected]>
This leaves python-snappy as the only remaining package that we're still pulling from PyPI. Signed-off-by: dann frazier <[email protected]>
Signed-off-by: dann frazier <[email protected]>
dannf
force-pushed
the
py3-cassandra-pin-python+use-systemlibs-for-security
branch
from
e073f2b to
c54a1c1
Compare
dannf
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
My goal here was initially just to fix up the python dependencies to make sure the package uses the correct python interpreter version, and avoid dragging in additional python version stacks. But the vulnerability scan in CI detected a number of issues caused by pulling in pinned versions of packages from PyPI, so about half of these changes are switching back over to wolfi deps where possible.
Note that this resurrects various python ssh packages that were previously removed. I believe that their removal predated the multi-versioning technique when there wasn't a clean way to only build for older/supported Python versions. I've gone ahead and multi-versioned them here.
More details as always in the individual commits.