Skip to content

Commit

Permalink
Add a chart test framework.
Browse files Browse the repository at this point in the history
Essentially, the idea is that we have a directory which contains
a set of values files: test/charts/*.in.  Corresponding to each chart
is the fully expanded chart that they generate.  This is stored in
the same directory with the .in replaced with a .expect.  They are
both checked into git.

We then provide a script called test-charts which uses "helm template"
to expand our charts using each values file and makes sure that it
matches the expected chart.  The script will error out if there are
any differences.  It will also emit the diffs.

If the diffs are acceptable, just:

	:; ./bin/test-charts -r <values_file_name>

will put the new expected chart output in place where you can add
it to your commit.
  • Loading branch information
elric1 committed Dec 4, 2021
1 parent d01449b commit 5d731b1
Show file tree
Hide file tree
Showing 8 changed files with 2,943 additions and 4 deletions.
21 changes: 17 additions & 4 deletions bin/helm-compare-versions
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
#!/bin/sh -e

copy_vals() {

NEWLOC="$(mktemp "$MYDIR/valuesXXXXXX")"
cp $1 $NEWLOC
echo $NEWLOC
}

usage() {

( echo "$1"
Expand All @@ -19,6 +26,8 @@ I_STASHED_IT=0
DIFF="diff -u"
DIFF_ARGS=

MYDIR="$(mktemp -d /tmp/helm-compare-versions-XXXXXX)"

while getopts SU:Yb:n:w f; do
case $f in
S) COMPARE_STASHED=1;;
Expand Down Expand Up @@ -47,6 +56,9 @@ if [ -z "$NEW_VALUES" ]; then
NEW_VALUES="$OLD_VALUES"
fi

[ ! -f "$NEW_VALUES" ] && NEW_VALUES="$(copy_vals test/charts/$NEW_VALUES.in)"
[ ! -f "$OLD_VALUES" ] && OLD_VALUES="$(copy_vals test/charts/$OLD_VALUES.in)"

CHART1=
CHART2=
CUR_BRANCH=
Expand All @@ -61,8 +73,9 @@ cleanup() {

trap '' PIPE INT

[ -f "$CHART1" ] && rm "$CHART1"
[ -f "$CHART2" ] && rm "$CHART2"
if [ -d "$MYDIR" ]; then
rm -rf "$MYDIR"
fi
if [ ! -z "$CUR_BRANCH" ]; then
git checkout "$CUR_BRANCH"
fi
Expand All @@ -86,7 +99,7 @@ if [ ! -z "$NEW_BRANCH" ]; then
git checkout "$NEW_BRANCH"
fi

CHART1=$(mktemp /tmp/chartXXXXXX)
CHART1="$MYDIR/chart1"
helm template charts/tezos/ --values "$NEW_VALUES" --namespace tez > "$CHART1"

if [ "$COMPARE_STASHED" = 1 ]; then
Expand All @@ -96,7 +109,7 @@ if [ "$COMPARE_STASHED" = 1 ]; then
fi
fi

CHART2=$(mktemp /tmp/chartXXXXXX)
CHART2="$MYDIR/chart2"
if [ ! -z "$OLD_BRANCH" ]; then
if [ -z "$CUR_BRANCH" ]; then
CUR_BRANCH=$(get_branch)
Expand Down
109 changes: 109 additions & 0 deletions bin/test-charts
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
#!/bin/sh -e

#
# test-charts runs "helm template" with a list of values files which
# are found in test/charts/*.in. Corresponding to each file, there is
# a file called test/charts/*.expect. This is the expected output of
# the template expansion.
#
# Running test-charts with no arguments will run all of the tests stopping
# on the first failure at which point it will emit a unified diff of the
# changes and exit nonzero.
#
# The -l flag will list all of the tests (without the .in suffix)
#
# The -r flag will regenerate the expect file corresponding to one or
# more of the tests. These must be provided as arguments on the command
# line and said arguments do not include the .in suffix.

OUT=

cleanup() {
rm -f $OUT
}
trap cleanup EXIT INT PIPE

run_test() {
IN="$1"
OUT="$(mktemp /tmp/chartXXXXXX)"
EXPECT="${1%.in}.expect"

echo "Testing $IN"

helm template --values "$IN" -n testing charts/tezos > $OUT

if ! cmp "$EXPECT" "$OUT"; then
echo "Test chart $IN failed"
echo
echo "Here are the differences from what we expect"
echo "What we expect will be -, what got +"
$DIFF "$EXPECT" "$OUT"
exit 1
fi
}

run_tests() {
for i in test/charts/*.in; do
run_test $i
done
}

list_tests() {
ls test/charts/*.in | sed 's,^test/charts/\(.*\).in,\1,'
}

regen() {
for i; do
IN="test/charts/$i.in"
if ! [ -f "$IN" ]; then
echo Test $i doesn\'t exist.
exit 1
fi
echo "Regenerating $IN"
EXPECT="${IN%.in}.expect"
helm template --values "$IN" -n testing charts/tezos > $EXPECT
done
}

usage() {
echo "$@" 1>&2
echo "Usage: $0 [-l]"
echo " $0 -r test_name [test_name ...]"
echo
echo " -l lists the tests"
echo " -r will cause the expected YAML files to be rebuilt for"
echo " the named tests"
exit 1
}

#
# First, get to the top level and ensure that we're tezos-k8s to
# some degree.

GIT_BASE="$(git rev-parse --show-toplevel)"
if [ -z "$GIT_BASE" -o ! -d "$GIT_BASE/charts/tezos" ]; then
usage "Not in a tezos-k8s git repo, failing"
fi
cd "$GIT_BASE"

#
# And now for main:

DIFF="diff -u"
JOB=run_tests

while getopts Ylr f; do
case $f in
Y) DIFF="dyff between";;
l) JOB=list;;
r) JOB=regen;;
\?) usage "Invalid option $f";;
esac
done
shift $(expr $OPTIND - 1)

case "$JOB" in
list) list_tests;;
run_tests) run_tests;;
regen) regen "$@";;
esac
Loading

0 comments on commit 5d731b1

Please sign in to comment.