Inspect Grafana dashboards

[jacek-dudek]

Go through existing Grafana dashboards that are hosted on aaw dev and aaw prod clusters and confirm which are working correctly, which don't have incoming data, which are displaying something obviously wrong or unintelligible.

Some investigative steps for the broken ones:

• For the dashboards with no data showing, confirm what the data source is, look at the network activity to see if requests are being made and succeeding.
• Look at the data queries in the code-behind for each dashboard component. Identify the metric being polled. Confirm whether or not this metric is being served on the prometheus server component.
• For missing metrics we'll have to find out what endpoints (like cadvisor) can produce these metrics for prometheus to collect.

[Souheil-Yazji]

2 Point estimate for:

• Enumerating Broken Dashboards
• Brief summary on broken components in each dash

Next Steps:

• Report issues to CNS, go from there

[jacek-dudek]

I inspected the first 25 dashboards and I'm compiling a spreadsheet of observed issues.
I have another 25 dashboards to go...

[jacek-dudek]

Spreadsheet of all the issues I noticed in existing Grafana dashboards:
Grafana-dashboards-inspection.xlsx

[jacek-dudek]

Just need the contact info for CNS to be able to relay the information to them...