Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add sles support #12

Merged
merged 1 commit into from
Nov 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ Before you begin, ensure that you have the following prerequisites:
1. `oc` CLI tool [^3].
2. OpenShift pull secret [^4].

**SLES**:
1. Active subscription.

## Dockerfile Overview

To build the precompiled container, the Dockerfile is constructed in a multistage fashion.
Expand Down Expand Up @@ -124,6 +127,19 @@ podman pull --authfile=/path/to/pull-secret.txt docker://quay.io/openshift-relea
--target precompiled .
```

**SLES example**:
```bash
# docker build \
--build-arg D_OS=sles15.5 \
--build-arg D_ARCH=x86_64 \
--build-arg D_BASE_IMAGE=registry.suse.com/suse/sle15:15.5 \
--build-arg D_KERNEL_VER=5.14.21-150500.55.83-default \
--build-arg D_OFED_VERSION=24.01-0.3.3.1 \
--tag 24.01-0.3.3.1-0-5.14.21-150500.55.83-default-sles15.5-amd64 \
-f SLES_Dockerfile \
--target precompiled .
```

>[!IMPORTANT]
>Dockerfiles contain default build parameters, which may fail build proccess on your system if not overridden.

Expand Down
150 changes: 150 additions & 0 deletions SLES_Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
# Common (multistage) args
ARG D_OS="sles15sp2"
ARG D_ARCH="x86_64"
ARG D_CONTAINER_VER="0"
ARG D_OFED_VERSION="23.10-0.5.5.0"
ARG D_KERNEL_VER="5.3.18-22-default"
ARG D_OFED_SRC_DOWNLOAD_PATH="/run/mellanox/src"
ARG OFED_SRC_LOCAL_DIR=${D_OFED_SRC_DOWNLOAD_PATH}/MLNX_OFED_SRC-${D_OFED_VERSION}

# Common for build and final clean image of precompiled driver container
ARG D_BASE_IMAGE="registry.suse.com/suse/sle15:15.5"

##################################################################
# Stage: Minimal base image update and install common requirements
FROM $D_BASE_IMAGE AS base

ARG D_OFED_VERSION
ARG D_CONTAINER_VER
ARG D_OFED_SRC_DOWNLOAD_PATH

ENV NVIDIA_NIC_DRIVER_VER=${D_OFED_VERSION}
ENV NVIDIA_NIC_CONTAINER_VER=${D_CONTAINER_VER}

WORKDIR /root

RUN set -x && \
# Perform distro update and install prerequirements
zypper --gpg-auto-import-keys ref -s && \
zypper --non-interactive update && \
zypper --non-interactive install --no-recommends \
# Driver build / install script requirements
perl pciutils kmod lsof python3 \
# dh-python not found
# Container functional requirements
jq iproute2 udev ethtool awk

WORKDIR /
ADD ./entrypoint.sh /root/entrypoint.sh

ENTRYPOINT ["/root/entrypoint.sh"]

##############################################################################################
# Stage: Download NVIDIA driver sources and install src driver container packages requirements

FROM base AS driver-src

# Inherited global args
ARG D_OFED_VERSION
ARG D_OFED_SRC_DOWNLOAD_PATH

# Stage args
ARG D_OFED_BASE_URL="https://www.mellanox.com/downloads/ofed/MLNX_OFED-${D_OFED_VERSION}"
ARG D_OFED_SRC_TYPE=""

ARG D_OFED_SRC_ARCHIVE="MLNX_OFED_SRC-${D_OFED_SRC_TYPE}${D_OFED_VERSION}.tgz"
ARG D_OFED_URL_PATH="${D_OFED_BASE_URL}/${D_OFED_SRC_ARCHIVE}"

ENV NVIDIA_NIC_DRIVER_PATH="${D_OFED_SRC_DOWNLOAD_PATH}/MLNX_OFED_SRC-${D_OFED_VERSION}"

WORKDIR /root

RUN set -x && \
for source in ${D_ZYPPER_REMOVE}; do rm -f /etc/zypp/repos.d/${source}.repo; done && \
# Perform distro update and install prerequirements
zypper --gpg-auto-import-keys ref -s && \
zypper --non-interactive update && \
zypper --non-interactive install --no-recommends \
make autoconf chrpath automake hostname gcc quilt dracut rpm-build sysvinit-tools

RUN set -x && \
# Download NVIDIA NIC driver sources
mkdir -p ${D_OFED_SRC_DOWNLOAD_PATH} && \
cd ${D_OFED_SRC_DOWNLOAD_PATH} && (curl -sL ${D_OFED_URL_PATH} | tar -xzf -)

CMD ["sources"]

#####################
# Stage: Build driver

FROM driver-src AS driver-builder

# Inherited global args
ARG D_OS
ARG D_KERNEL_VER
ARG OFED_SRC_LOCAL_DIR

# Driver build mandatory packages
RUN set -x && \
# MOFED installation requirements
zypper --non-interactive install --no-recommends autoconf gcc make rpm-build dracut
RUN set -x && \
zypper search -s kernel-default-devel

# Remove '-default' suffix if it exists
RUN CLEANED_KERNEL_VER=${D_KERNEL_VER%-default} && \
echo "Installing kernel-default-devel=${CLEANED_KERNEL_VER}" && \
# Install the kernel source
zypper --non-interactive install --no-recommends kernel-default-devel=${CLEANED_KERNEL_VER}

RUN ls -l /lib/modules

# Build driver --distro ${D_OS}
RUN set -x && \
${OFED_SRC_LOCAL_DIR}/install.pl -vvv --distro ${D_OS} --kernel-sources /lib/modules/${D_KERNEL_VER}/build --without-depcheck --kernel ${D_KERNEL_VER} --kernel-only --build-only --copy-ifnames-udev --with-mlnx-tools --without-knem-modules --without-srp-modules --without-kernel-mft-modules --without-iser-modules --without-isert-modules
RUN ls -l ${OFED_SRC_LOCAL_DIR}/RPMS/*/x86_64/
###################################
# Stage: Install precompiled driver

FROM base AS precompiled

# Inherited global args
ARG D_ARCH
ARG D_KERNEL_VER
ARG D_OFED_VERSION
ARG D_CONTAINER_VER
ARG OFED_SRC_LOCAL_DIR

ENV NVIDIA_NIC_DRIVER_VER=${D_OFED_VERSION}
ENV NVIDIA_NIC_DRIVER_PATH=""
ENV NVIDIA_NIC_CONTAINER_VER=${D_CONTAINER_VER}

ENV NVIDIA_NIC_DRIVER_PATH=""
COPY --from=driver-builder ${OFED_SRC_LOCAL_DIR}/RPMS/sles-release-*/${D_ARCH}/*.rpm /root/

WORKDIR /root/
RUN set -x && \
rpm -ivh --nodeps \
./mlnx-nfsrdma-*.rpm \
./mlnx-nvme-*.rpm \
./mlnx-ofa_kernel-*.rpm \
./mlnx-tools-*.rpm

RUN set -x && \
zypper --non-interactive install --no-recommends \
# MOFED functional requirements
make autoconf chrpath automake hostname gcc quilt dracut rpm-build sysvinit-tools \
# Container functional requirements
jq kmod

# Prevent modprobe from giving a WARNING about missing files
RUN touch /lib/modules/${D_KERNEL_VER}/modules.order /lib/modules/${D_KERNEL_VER}/modules.builtin && \
# Introduce installed kernel modules
depmod ${D_KERNEL_VER}

WORKDIR /
ADD ./entrypoint.sh /root/entrypoint.sh
ADD ./dtk_nic_driver_build.sh /root/dtk_nic_driver_build.sh

ENTRYPOINT ["/root/entrypoint.sh"]
CMD ["precompiled"]
24 changes: 21 additions & 3 deletions entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -192,10 +192,16 @@ function ubuntu_install_prerequisites() {
exec_cmd "apt-get -yq install pkg-config linux-headers-${FULL_KVER}"
}

function sles_install_prerequisites() {
debug_print "Function: ${FUNCNAME[0]}"
CLEANED_KERNEL_VER=${FULL_KVER%-default}
exec_cmd "zypper --non-interactive install --no-recommends kernel-default-devel=${CLEANED_KERNEL_VER}"
}

function redhat_fetch_major_ver() {
debug_print "Function: ${FUNCNAME[0]}"

if ${IS_OS_UBUNTU}; then
if ${IS_OS_UBUNTU} || ${IS_OS_SLES}; then
debug_print "${FUNCNAME[0]}: unexpected call"
exit_entryp 1
else
Expand Down Expand Up @@ -359,6 +365,14 @@ function build_driver_from_src() {
append_driver_build_flags="$append_driver_build_flags --without-dkms"
fi

if ${IS_OS_SLES}; then
sub_path_str="RPMS"
os_str="sles"
package_type="rpm"
append_driver_build_flags="$append_driver_build_flags --disable-kmp"
append_driver_build_flags="$append_driver_build_flags --kernel-sources /lib/modules/${FULL_KVER}/build"
fi

timestamp_print "Starting driver build"
exec_cmd "${NVIDIA_NIC_DRIVER_PATH}/install.pl --without-depcheck --kernel ${FULL_KVER} --kernel-only --build-only --with-mlnx-tools --without-knem${pkg_dkms_suffix} --without-iser${pkg_dkms_suffix} --without-isert${pkg_dkms_suffix} --without-srp${pkg_dkms_suffix} --without-kernel-mft${pkg_dkms_suffix} --without-mlnx-rdma-rxe${pkg_dkms_suffix} ${append_driver_build_flags}"

Expand Down Expand Up @@ -479,7 +493,7 @@ function restart_driver() {

# ARM does not contain relevant packages, also not a blocker for this OS type for mlx5_core load
if [ "${ARCH}" != "aarch64" ]; then
if ! ${IS_OS_UBUNTU}; then
if ! ${IS_OS_UBUNTU} && ! ${IS_OS_SLES}; then
redhat_fetch_major_ver
[[ $RHEL_MAJOR_VERSION -ge $RH_RT_MIN_MAJOR_VER ]] && load_pci_hyperv_intf=true
else
Expand Down Expand Up @@ -1065,7 +1079,6 @@ function install_driver() {

if ${IS_OS_UBUNTU}; then
exec_cmd "dpkg -i ${driver_inventory_path}/*.deb"

else
exec_cmd "rpm -ivh --replacepkgs --nodeps ${driver_inventory_path}/*.rpm"
fi
Expand Down Expand Up @@ -1145,6 +1158,8 @@ function build_driver() {
if ${build_src}; then
if ${IS_OS_UBUNTU}; then
ubuntu_install_prerequisites
elif ${IS_OS_SLES}; then
sles_install_prerequisites
else
redhat_install_prerequisites
fi
Expand Down Expand Up @@ -1272,6 +1287,7 @@ timestamp_print "NVIDIA driver container exec start"
ARCH=$(uname -m)
FULL_KVER=$(uname -r)
IS_OS_UBUNTU=true; [[ "$(grep -i ubuntu /etc/os-release -c)" == "0" ]] && IS_OS_UBUNTU=false
IS_OS_SLES=true; [[ "$(grep -i sles /etc/os-release -c)" == "0" ]] && IS_OS_SLES=false
RHEL_MAJOR_VERSION=0
OPENSHIFT_VERSION=""

Expand Down Expand Up @@ -1311,6 +1327,8 @@ mlx_dev_record_idx=0
if ${IS_OS_UBUNTU}; then
debug_print "OS is Ubuntu"
pkg_dkms_suffix="-dkms"
elif ${IS_OS_SLES}; then
debug_print "OS is SLES"
else
debug_print "OS is Red Hat"
fi
Expand Down