[baseserver] Emit metrics for logs produced by level ENG-349 #19063
Conversation
easyCZ
changed the title
easyCZ
force-pushed
the
mp/baseserver-log-metrics
branch
from
8bd046e to
7be67c3
Compare
There was a problem hiding this comment.
Not all components are using baseserver, do we need to register metric manually for them? Or improve it in follow-up PRs? (search github.com/gitpod-io/gitpod/common-go/baseserver in codebase)
Does it make sense to add service name as a label so that we know which components log errors more?
| } | ||
|
|
||
| func (h *LogHook) Fire(entry *logrus.Entry) error { | ||
| h.metrics.ReportLog(entry.Level) |
There was a problem hiding this comment.
log level here is uint32 type Level uint32
There was a problem hiding this comment.
Yes, but the method calls
func (m *Metrics) ReportLog(level logrus.Level) {
m.logEmitedCounter.WithLabelValues(level.String()).Inc()
}
which uses the string version
| function doLog(calledViaConsole: boolean, consoleLog: ConsoleLog, severity: GoogleLogSeverity, args: unknown[]): void { | ||
| logsCounter.labels(severity).inc(); |
There was a problem hiding this comment.
level here is type GoogleLogSeverity = "EMERGENCY" | "ALERT" | "CRITICAL" | "ERROR" | "WARNING" | "INFO" | "DEBUG";
| @@ -246,7 +247,16 @@ namespace GoogleLogSeverity { | |||
| }; | |||
| } | |||
|
|
|||
| const logsCounter = new prometheusClient.Counter({ | |||
There was a problem hiding this comment.
Can't find where we use (expose) this counter
easyCZ
force-pushed
the
mp/baseserver-log-metrics
branch
from
4484109 to
96ebcd2
Compare
easyCZ
changed the title
|
I've removed TS changes from this PR, I can't figure out how to make it work so will ask EXP |
|
Testing 👀 |
|
TS parts are here; will rebase once this is in. |
Description
To be able to monitor each cells error logs without having to export them immediately. First we can check volumes. As a side-effect, this also allows us to identify the most offending components in terms of log volume
Summary generated by Copilot
🤖 Generated by Copilot at 1a98bf1
This pull request adds log metrics to the common-go module, which can be used by the base server and other components to report the number of logs by level to prometheus. This enhances the monitoring and debugging capabilities of Gitpod services.
Related Issue(s)
Fixes #
How to test
Documentation
Preview status
Gitpod was successfully deployed to your preview environment.
Build Options
Build
Run the build with werft instead of GHA
Run Leeway with
--dont-testPublish
Installer
Add desired feature flags to the end of the line above, space separated
Preview Environment / Integration Tests
If enabled this will build
install/previewIf enabled this will create the environment on GCE infra
Valid options are
all,workspace,webapp,ide,jetbrains,vscode,ssh. If enabled,with-previewandwith-large-vmwill be enabled./hold