bazel: use a sysroot for compilation #24669
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rockwotj
force-pushed
the
bazel-image
branch
3 times, most recently
from
8b02ac6 to
4e111a9
Compare
rockwotj
requested review from
twmb,
r-vasquez,
gene-redpanda and
Deflaimun
as code owners
rockwotj
force-pushed
the
bazel-image
branch
3 times, most recently
from
2b99fef to
4430087
Compare
rockwotj
requested review from
dotnwat
and removed request for
twmb,
Deflaimun,
r-vasquez and
gene-redpanda
dotnwat
previously approved these changes
Jan 3, 2025
Big picture: * Use Distroless containers from Google to have a minimal base image * Since we don't have bash in these containers change our LD_LIBRARY_PATH loader script to static Golang binaries * Mimic the layout/functionality of our existing images We don't package host shared libraries (glibc, libgcc) however, so we still need a solution to package those up, ideally with a clang sysroot.
Provide a sysroot from an ubuntu jammy docker image to provide a consistent set of headers, and system libraries for us to use to build Redpanda. This way there should be less differences between machines and we can remove a couple of required packages for development.
Required to pull in fixes for a custom sysroot
CI test resultstest results on build#60473
|
|
CI is failing with the reduced set of packages installed in the docker container because there is no longer a system python interpreter. The latest commit fixes our python3 binaries to use the Bazel toolchain instead. |
Without this one neat trick, genrules are using the system python3 to run these python scripts (which are chmod +x python files with #!/usr/bin/env python3 at the top). This change explicitly uses the Bazel toolchain. We must switch to the bazel toolchain because after removing valgrind from the list of system packages in `install-deps.sh` there is no longer python available in the docker image we build with. It's better this way anyways.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This patchset is two fold:
First and foremost we have a hermetic sysroot we compile against for all builds so now system dependencies should not leak into the build, and we can define them from a docker image. This means we can remove some dependencies from the
install-deps.shscript.Secondly, wire up a docker image that can run the broker using the ultra slim distroless series of containers. There is probably more we can do on the packaging front, but this is a good step. I moved the
LD_LIBRARY_PATHscript to a golang binary since bash is not installed on the distroless images. I'm also thinking that we could move completely to the base image that distroless provides (no shared libraries present), then copy in our shared libraries from our sysroot, but I'm punting on that for now.Backports Required
Release Notes