Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cass graphite intg test #67

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
13 changes: 6 additions & 7 deletions test/cdr_cass_integration_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ grep -q "${cassandra_port}=\"${cassandra_it_port}\"" "$app_conf_path" && echo -n

# Finally, run the jar.
# Run with timeout
timeout $timeout java -Dnetty.epoll.enabled=false -Dconfig.file=$(basename "$app_conf_path") -jar "$jar_directory"
timeout $timeout java -Dnetty.epoll.enabled=false -Dconfig.file=$app_conf_path -Dtrends.dir=src/main/resoucers/trends/ -jar "$jar_directory"

# Return the config file as it was.
echo "Checking out $app_conf_path"
Expand All @@ -76,7 +76,6 @@ docker exec -i "$cass_container_name" cqlsh << EOF > "$temp_cassandra_result_fil
use $cassandra_keyspace;
select count(*) from $cassandra_cdr_table_name;
EOF

# If the query was successful, there should now be a file.
[ -f $temp_cassandra_result_file ] || { echo "No cassandra result file[no success]"; exit 1; }

Expand All @@ -85,18 +84,18 @@ has_records=$(cat "$temp_cassandra_result_file" | grep -oP '\s+[0-9]+')

# use head and tail to get the first and last.
has_records_cdr=$(echo "$has_records" | head -n+1)

#
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove

# cleanup
if [ -f "$temp_cassandra_result_file" ]
then
echo "Removed temp file"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why switch all 4 space identations to 3 space identations?

rm "$temp_cassandra_result_file"
echo "Removed temp file"
rm "$temp_cassandra_result_file"
fi

# Exit codes
if [ $has_records_cdr -eq 1 ]
then
echo '[success]'
echo '[success]'
else
echo '[no success]'
echo '[no success]'
fi
44 changes: 44 additions & 0 deletions test/dbcIntegrationTest.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
#!/bin/bash

app_conf_path="src/main/resources/application.conf"
limit="theLimit"
Copy link
Collaborator

@johan-bjareholt johan-bjareholt May 10, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rename

tempGraphiteFile="tempGraphiteData.txt"

#match "limit=number" and force number to be 1
pushd ../QvantelDBConnector/
sed -i "s#${limit}\ *\=\ *\-*[0-9]*#${limit}\=5#g" "$app_conf_path"

# now running the dbconnector
echo "running dbconnector using sbt run"
sbt run

# now change the app.config file to its origin
sed -i "s#${limit}\ *\=\ *\-*[0-9]*#${limit}\=-1#g" "$app_conf_path"

# back to the directory you were in at the beginning
popd

#now run curl to check if the data is at graphite. in this case we are looking for "callplanNormal"
curl '127.0.0.1:2090/render?target=qvantel.product.voice.CallPlanNormal&format=json' -o $tempGraphiteFile
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please set variables for IP and port


#check if the file contain data using grep
if grep -Fxq "[]" $tempGraphiteFile
then
#there are no records in this file
echo "the dbconnector integration test failed!"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indentation

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add identation

echo "[-failed-]"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to set a non-zero exit code upon fail, otherwise travis will not report this as a failed test.
If you do cleanup in the end of this file you most likely want to set a variable with the exit code and to "exit $myvar" in the end

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Saw that you later in integration.sh use grep on this text.
Sure it works but it is very bad and can easily break, so please fix anyway.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:( ok

else
#there are records in this file
echo "dbconnector integration test is done and successfull"
echo "[-success-]"
fi

# reset the "application.config" file to its origin values
pushd ../QvantelDBConnector/
#change the port for the graphite to a new port in config file
sed -i "s#c.port = [0-9]*#c.port = 9042#g" "$app_conf_path"
sed -i "s#g.port = [0-9]*#g.port = 2003#g" "$app_conf_path"
popd

#now remove the temp file
rm -f $tempGraphiteFile
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why -f? this can be abused. Remove -f.

77 changes: 69 additions & 8 deletions test/integration.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,13 @@ test -z "$(which git >/dev/null)" || { echo "Git not available"; exit 1; }

schema_file="/schema.cql"
container_name="int_cassandra"

container_image_name="cassandra_qvantel"
dbc_app_conf_path="src/main/resources/application.conf"
dbc_container_image="hopsoft/graphite-statsd"
dbc_container_name="dbc_test_container"
cass_port="9043"
dbc_graph_port="2025"

test -n "$(docker images | grep $container_image_name)" || { echo "Cassandra image not found.)"; exit 1; }

Expand All @@ -30,13 +35,51 @@ function provision_cass_container {
fi
}

function provision_dbc_container {

#provision container
if [ -n "$(docker images | grep $dbc_container_image)" ]
then
if [ -n "$(docker ps -a | grep $dbc_container_name)" ]
then
deallocat_dbc_container
fi

echo "running $dbc_container_name"
#change the port for the graphite to a new port in config file
sed -i "s#c.port = [0-9]*#c.port = $cass_port#g" $dbc_app_conf_path
sed -i "s#g.port = [0-9]*#g.port = $dbc_graph_port#g" $dbc_app_conf_path
# run docker to create new container with new port and
docker run \
--name $dbc_container_name \
-p 2025-2026:2003-2004 \
-p 2090:80 \
-p 8125:8125/udp \
-p 8126:8126 \
-p 2027-2028:2023-2024 \
-d hopsoft/graphite-statsd:latest
fi
}

function deallocat_dbc_container {

# deallocate the dbc container
docker stop "$dbc_container_name"
echo "dbc connector container stopped"
docker rm "$dbc_container_name"
echo "dbc connector removed"
}

function deallocate_cass_container {
# And we're back, now clean up the container
docker stop "$container_name" 2>&1 1>/dev/null && echo "Stopping $container_name"
docker rm "$container_name" 2>&1 1>/dev/null && echo "Stopping $container_name"
docker rm "$container_name" 2>&1 1>/dev/null && echo "removing $container_name"
}

provision_cass_container
pushd ../QvantelDBConnector/
provision_dbc_container
popd

# Wait until cqlsh is up and running
echo "Waiting for cqlsh to be up"
Expand All @@ -49,23 +92,41 @@ done
echo "Running schema"
docker exec -it "$container_name" cqlsh -f "$schema_file"

# It's up!
# Cqlsh is up!
# Run integration test
echo "Running integration test"
int_test=$(./cdr_cass_integration_test.sh "$container_name" 2>&1)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you change this?
On the lines above we create and start a cassandra container on port 9043 so we can run the tests without having to manually start cassandra as well as not having to shut down cassandra if it is already running on your local machine. This breaks it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree.

echo "Running cdr_cassandra integration test"
int_test=$(./cdr_cass_integration_test.sh "cassandra_qvantel" 2>&1)
code=$?
echo "Integration test done"
echo "cdr_cassandra Integration test done"

#run dbconnector integration test
echo "running dbconnector integration test"
dbc_inte_test=$(./dbcIntegrationTest.sh 2>&1)
result=$?
echo "dbconnector integration test done"

# Test is done
#remove cassandra container
deallocate_cass_container
deallocat_dbc_container

# Finally, report back
if [ -n "$(echo $int_test | grep '[SUCCESS]')" ] && [ $code -eq 0 ]
then
echo "Integration test successful"
exit 0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why remove exit 0?

echo "cdr_cass Integration test successful"

else
echo "Interation test failed"
echo "cdr_cass Integration test failed"
echo "$int_test"
exit 1
fi

# Finally, report back dbconnector
if [ -n "$(echo $dbc_inte_test | grep '[-success-]')" ] && [ $result -eq 0 ]
then
echo "dbConnector Integration test successful"
exit 0
else
echo "dbConnector Integration test failed"
exit 1
fi