Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move overriding Scylla Operator's log level to CI scripts and make it configurable #2297

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
+89 −38
Open

Move overriding Scylla Operator's log level to CI scripts and make it configurable #2297

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 0 additions & 9 deletions hack/.ci/lib/e2e.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -211,15 +211,6 @@ function run-e2e {
kubectl create clusterrolebinding e2e --clusterrole=cluster-admin --serviceaccount=e2e:default --dry-run=client -o=yaml | kubectl_create -f=-
kubectl create -n=e2e pdb my-pdb --selector='app=e2e' --min-available=1 --dry-run=client -o=yaml | kubectl_create -f=-


# Raise loglevel in CI.
# TODO: Replace it with ScyllaOperatorConfig field when available.
kubectl -n=scylla-operator patch --field-manager="${FIELD_MANAGER}" deployment/scylla-operator --type=json -p='[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value": "--loglevel=4"}]'
kubectl -n=scylla-operator rollout status deployment/scylla-operator

kubectl -n=scylla-manager patch --field-manager="${FIELD_MANAGER}" deployment/scylla-manager-controller --type=json -p='[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value": "--loglevel=4"}]'
kubectl -n=scylla-manager rollout status deployment/scylla-manager-controller

kubectl create -n=e2e secret generic kubeconfigs ${KUBECONFIGS[@]/#/--from-file=} --dry-run=client -o=yaml | kubectl_create -f=-
kubeconfigs_in_container_path=$( IFS=','; basenames=( "${KUBECONFIGS[@]##*/}" ) && in_container_paths="${basenames[@]/#//var/run/secrets/kubeconfigs/}" && echo "${in_container_paths[*]}" )

Expand Down
3 changes: 3 additions & 0 deletions hack/.ci/run-e2e-gke-release.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ export SO_NODECONFIG_PATH
SO_SCYLLACLUSTER_STORAGECLASS_NAME="${SO_SCYLLACLUSTER_STORAGECLASS_NAME=scylladb-local-xfs}"
export SO_SCYLLACLUSTER_STORAGECLASS_NAME

SO_SCYLLA_OPERATOR_LOGLEVEL="${SO_SCYLLA_OPERATOR_LOGLEVEL:-4}"
export SO_SCYLLA_OPERATOR_LOGLEVEL

Comment on lines +26 to +28
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about creating a _run-e2e-common.inc.sh file with some of the bash boilerplate? (to get rid of the "repeated constants" code smell)

that would then be included using something like

source "$( dirname "${BASH_SOURCE[0]}" )/_run-e2e-common.inc.sh"

SCYLLA_OPERATOR_FEATURE_GATES="${SCYLLA_OPERATOR_FEATURE_GATES:-AllAlpha=true,AllBeta=true}"
export SCYLLA_OPERATOR_FEATURE_GATES

Expand Down
3 changes: 3 additions & 0 deletions hack/.ci/run-e2e-gke.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@ if [[ "${SO_DISABLE_NODECONFIG:-false}" == "true" ]]; then
SO_CSI_DRIVER_PATH=""
fi

SO_SCYLLA_OPERATOR_LOGLEVEL="${SO_SCYLLA_OPERATOR_LOGLEVEL:-4}"
export SO_SCYLLA_OPERATOR_LOGLEVEL

SCYLLA_OPERATOR_FEATURE_GATES="${SCYLLA_OPERATOR_FEATURE_GATES:-AllAlpha=true,AllBeta=true}"
export SCYLLA_OPERATOR_FEATURE_GATES

Expand Down
3 changes: 3 additions & 0 deletions hack/.ci/run-e2e-openshift-aws-release.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ export SO_NODECONFIG_PATH
SO_SCYLLACLUSTER_STORAGECLASS_NAME="${SO_SCYLLACLUSTER_STORAGECLASS_NAME=scylladb-local-xfs}"
export SO_SCYLLACLUSTER_STORAGECLASS_NAME

SO_SCYLLA_OPERATOR_LOGLEVEL="${SO_SCYLLA_OPERATOR_LOGLEVEL:-4}"
export SO_SCYLLA_OPERATOR_LOGLEVEL

SCYLLA_OPERATOR_FEATURE_GATES="${SCYLLA_OPERATOR_FEATURE_GATES:-AllAlpha=true,AllBeta=true}"
export SCYLLA_OPERATOR_FEATURE_GATES

Expand Down
3 changes: 3 additions & 0 deletions hack/.ci/run-e2e-openshift-aws.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,9 @@ export SO_CSI_DRIVER_PATH
SO_SCYLLACLUSTER_STORAGECLASS_NAME="${SO_SCYLLACLUSTER_STORAGECLASS_NAME=scylladb-local-xfs}"
export SO_SCYLLACLUSTER_STORAGECLASS_NAME

SO_SCYLLA_OPERATOR_LOGLEVEL="${SO_SCYLLA_OPERATOR_LOGLEVEL:-4}"
export SO_SCYLLA_OPERATOR_LOGLEVEL

SCYLLA_OPERATOR_FEATURE_GATES="${SCYLLA_OPERATOR_FEATURE_GATES:-AllAlpha=true,AllBeta=true}"
export SCYLLA_OPERATOR_FEATURE_GATES

Expand Down
100 changes: 71 additions & 29 deletions hack/ci-deploy-release.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,25 @@ patches:
- name: SCYLLA_OPERATOR_IMAGE
value: "${operator_image_ref}"
EOF

# TODO: Replace it with ScyllaOperatorConfig field when available.
if [[ -n "${SO_SCYLLA_OPERATOR_LOGLEVEL:-}" ]]; then
# SO_SCYLLA_OPERATOR_LOGLEVEL is set and nonempty.
cat > "${ARTIFACTS_DEPLOY_DIR}/operator/loglevel.scylla-operator.deployment.patch.json" << EOF
[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value": "--loglevel=${SO_SCYLLA_OPERATOR_LOGLEVEL}"}]
EOF

cat << EOF | \
yq eval-all --inplace 'select(fileIndex == 0) as $f | select(fileIndex == 1) as $p | with( $f.patches; . += $p | ... style="") | $f' "${ARTIFACTS_DEPLOY_DIR}/operator/kustomization.yaml" -
path: "${ARTIFACTS_DEPLOY_DIR}/operator/loglevel.scylla-operator.deployment.patch.json"
target:
group: apps
version: v1
kind: Deployment
name: scylla-operator
EOF
fi

kubectl kustomize "${ARTIFACTS_DEPLOY_DIR}/operator" | kubectl_create -n=scylla-operator -f=-

# Manager needs scylla CRD registered and the webhook running
Expand All @@ -96,41 +115,64 @@ else
echo "Skipping CSI driver creation"
fi

if [[ -z "${SO_SCYLLACLUSTER_STORAGECLASS_NAME+x}" ]]; then
kubectl_create -n=scylla-manager -f="${source_url}/${revision}/deploy/manager-prod.yaml"
elif [[ -n "${SO_SCYLLACLUSTER_STORAGECLASS_NAME}" ]]; then
cat > "${ARTIFACTS_DEPLOY_DIR}/manager/kustomization.yaml" << EOF
cat > "${ARTIFACTS_DEPLOY_DIR}/manager/kustomization.yaml" << EOF
resources:
- ${source_url}/${revision}/deploy/manager-prod.yaml
patches:
- target:
group: scylla.scylladb.com
version: v1
kind: ScyllaCluster
name: scylla-manager-cluster
patch: |
- op: replace
path: /spec/datacenter/racks/0/storage/storageClassName
value: "${SO_SCYLLACLUSTER_STORAGECLASS_NAME}"
patches: []
EOF
kubectl kustomize "${ARTIFACTS_DEPLOY_DIR}/manager" | kubectl_create -n=scylla-manager -f=-
else
cat > "${ARTIFACTS_DEPLOY_DIR}/manager/kustomization.yaml" << EOF
resources:
- ${source_url}/${revision}/deploy/manager-prod.yaml
patches:
- target:
group: scylla.scylladb.com
version: v1
kind: ScyllaCluster
name: scylla-manager-cluster
patch: |
- op: remove
path: /spec/datacenter/racks/0/storage/storageClassName

if [[ -n "${SO_SCYLLACLUSTER_STORAGECLASS_NAME:-}" ]]; then
# SO_SCYLLACLUSTER_STORAGECLASS_NAME is set and nonempty.
cat > "${ARTIFACTS_DEPLOY_DIR}/manager/storageclass-name.scylla-manager-cluster.scyllacluster.patch.json" << EOF
[{"op": "replace", "path": "/spec/datacenter/racks/0/storage/storageClassName", "value": "${SO_SCYLLACLUSTER_STORAGECLASS_NAME}"}]
EOF

cat << EOF | \
yq eval-all --inplace 'select(fileIndex == 0) as $f | select(fileIndex == 1) as $p | with( $f.patches; . += $p | ... style="") | $f' "${ARTIFACTS_DEPLOY_DIR}/manager/kustomization.yaml" -
path: "${ARTIFACTS_DEPLOY_DIR}/manager/storageclass-name.scylla-manager-cluster.scyllacluster.patch.json"
target:
group: scylla.scylladb.com
version: v1
kind: ScyllaCluster
name: scylla-manager-cluster
EOF
elif [[ -n "${SO_SCYLLACLUSTER_STORAGECLASS_NAME+x}" ]]; then
# SO_SCYLLACLUSTER_STORAGECLASS_NAME is set and empty.
cat > "${ARTIFACTS_DEPLOY_DIR}/manager/storageclass-name.scylla-manager-cluster.scyllacluster.patch.json" << EOF
[{"op": "remove", "path": "/spec/datacenter/racks/0/storage/storageClassName"}]
EOF

cat << EOF | \
yq eval-all --inplace 'select(fileIndex == 0) as $f | select(fileIndex == 1) as $p | with( $f.patches; . += $p | ... style="") | $f' "${ARTIFACTS_DEPLOY_DIR}/manager/kustomization.yaml" -
path: "${ARTIFACTS_DEPLOY_DIR}/manager/storageclass-name.scylla-manager-cluster.scyllacluster.patch.json"
target:
group: scylla.scylladb.com
version: v1
kind: ScyllaCluster
name: scylla-manager-cluster
EOF
kubectl kustomize "${ARTIFACTS_DEPLOY_DIR}/manager" | kubectl_create -n=scylla-manager -f=-
fi

# TODO: Replace it with ScyllaOperatorConfig field when available.
if [[ -n "${SO_SCYLLA_OPERATOR_LOGLEVEL:-}" ]]; then
# SO_SCYLLA_OPERATOR_LOGLEVEL is set and nonempty.
cat > "${ARTIFACTS_DEPLOY_DIR}/manager/loglevel.scylla-manager-controller.deployment.patch.json" << EOF
[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value": "--loglevel=${SO_SCYLLA_OPERATOR_LOGLEVEL}"}]
EOF

cat << EOF | \
yq eval-all --inplace 'select(fileIndex == 0) as $f | select(fileIndex == 1) as $p | with( $f.patches; . += $p | ... style="") | $f' "${ARTIFACTS_DEPLOY_DIR}/manager/kustomization.yaml" -
path: "${ARTIFACTS_DEPLOY_DIR}/manager/loglevel.scylla-manager-controller.deployment.patch.json"
target:
group: apps
version: v1
kind: Deployment
name: scylla-manager-controller
EOF
fi

kubectl kustomize "${ARTIFACTS_DEPLOY_DIR}/manager" | kubectl_create -n=scylla-manager -f=-

kubectl -n=scylla-manager wait --timeout=5m --for='condition=Progressing=False' scyllaclusters.scylla.scylladb.com/scylla-manager-cluster
kubectl -n=scylla-manager wait --timeout=5m --for='condition=Degraded=False' scyllaclusters.scylla.scylladb.com/scylla-manager-cluster
kubectl -n=scylla-manager wait --timeout=5m --for='condition=Available=True' scyllaclusters.scylla.scylladb.com/scylla-manager-cluster
Expand Down
6 changes: 6 additions & 0 deletions hack/ci-deploy.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,12 @@ for f in $( find "${DEPLOY_DIR}"/ -type f -name '*.yaml' ); do
sed -i -E -e "s~docker\.io/scylladb/scylla-operator:[^ @]+$~${OPERATOR_IMAGE_REF}~" "${f}"
done

# TODO: Replace it with ScyllaOperatorConfig field when available.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could make this a GitHub issue?

if [[ -n "${SO_SCYLLA_OPERATOR_LOGLEVEL:-}" ]]; then
yq e --inplace '.spec.template.spec.containers[0].args += "--loglevel=" + env(SO_SCYLLA_OPERATOR_LOGLEVEL)' "${DEPLOY_DIR}/operator/50_operator.deployment.yaml"
yq e --inplace '.spec.template.spec.containers[0].args += "--loglevel=" + env(SO_SCYLLA_OPERATOR_LOGLEVEL)' "${DEPLOY_DIR}/manager/50_controller_deployment.yaml"
fi

yq e --inplace '.spec.template.spec.containers[0].args += ["--qps=200", "--burst=400"]' "${DEPLOY_DIR}/operator/50_operator.deployment.yaml"
yq e --inplace '.spec.template.spec.containers[0].args += ["--crypto-key-buffer-size-min=3", "--crypto-key-buffer-size-max=6", "--crypto-key-buffer-delay=2s"]' "${DEPLOY_DIR}/operator/50_operator.deployment.yaml"

Expand Down