Make Cassandra performance stats dashboard for Grafana

[genghisken]

There is little understanding of the performance bottlenecks in Cassandra. We need a Grafana dashboard that could potentially identify performance issues.

[genghisken]

I've added the following to each Cassandra node:

/etc/cassandra/prometheus/prometheus_cassandra.yml
/etc/cassandra/prometheus/jmx_prometheus_javaagent-1.0.1.jar

The JAR file was downloaded from here:
https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/1.0.1/jmx_prometheus_javaagent-1.0.1.jar

The /etc/cassandra/cassandra-env.sh file needs to be modified with the following line:

JVM_OPTS="$JVM_OPTS -javaagent:/etc/cassandra/prometheus/jmx_prometheus_javaagent-1.0.1.jar=7070:/etc/cassandra/prometheus/prometheus_cassandra.yml"

Now added to my deployment scripts so new Cassandra clusters will have the prometheus agent installed by default.

There is an extensive list of stats available. I think there are example Grafana dashboard templates we can download that will show us the best way to view the stats.