Add environment variables to support bundle (#2073)

* Add environment variables to support bundle * Update changelog * Update docs * Update documentation of environment variables and add windows service information * Update docs/sources/configure/windows.md Co-authored-by: Clayton Cornell <[email protected]> * Update docs/sources/configure/windows.md Co-authored-by: Clayton Cornell <[email protected]> * Update docs/sources/configure/windows.md Co-authored-by: Clayton Cornell <[email protected]> * Update docs/sources/configure/windows.md Co-authored-by: Clayton Cornell <[email protected]> * Remove windows specific details from environment variables page * Update docs/sources/reference/cli/environment-variables.md Co-authored-by: Clayton Cornell <[email protected]> * Update docs/sources/reference/cli/environment-variables.md Co-authored-by: Clayton Cornell <[email protected]> * Update docs/sources/reference/cli/environment-variables.md Co-authored-by: Clayton Cornell <[email protected]> * Update docs/sources/reference/cli/environment-variables.md Co-authored-by: Clayton Cornell <[email protected]> * Update docs/sources/reference/cli/environment-variables.md Co-authored-by: Clayton Cornell <[email protected]> * Update docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md Co-authored-by: Clayton Cornell <[email protected]> * Update docs/sources/reference/components/otelcol/otelcol.receiver.syslog.md Co-authored-by: Clayton Cornell <[email protected]> * Accidental commit --------- Co-authored-by: Clayton Cornell <[email protected]>
grafana · Nov 22, 2024 · 3b57159 · 3b57159
1 parent f04b78f
commit 3b57159
Show file tree

Hide file tree

Showing 5 changed files with 145 additions and 27 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -24,6 +24,8 @@ Main (unreleased)
 
 - Add second metrics sample to the support bundle to provide delta information (@dehaansa)
 
+- Add relevant golang environment variables to the support bundle (@dehaansa)
+
 ### Bugfixes
 
 - Fixed an issue in the `prometheus.exporter.postgres` component that would leak goroutines when the target was not reachable (@dehaansa)

diff --git a/docs/sources/configure/windows.md b/docs/sources/configure/windows.md
@@ -61,6 +61,37 @@ To change the set of command-line arguments passed to the {{< param "PRODUCT_NAM
 
    1. Click on **All Tasks > Restart**.
 
+## Change environment variable values
+
+The Go runtime provides several ways to modify the execution of a binary using [environment variables][environment].
+
+To change the environment variables used by {{< param "PRODUCT_NAME" >}}, perform the following steps.
+
+1. Open the Registry Editor:
+
+   1. Right click on the Start Menu and select **Run**.
+
+   1. Type `regedit` and click **OK**.
+
+1. Navigate to the key at the path `HKEY_LOCAL_MACHINE\SOFTWARE\GrafanaLabs\Alloy`.
+
+1. Double-click on the multi-string value called **Environment***.
+
+1. In the dialog box, enter the environment variable values to pass to the {{< param "PRODUCT_NAME" >}} binary.
+   Make sure that each variable is on its own line.
+
+1. Restart the {{< param "PRODUCT_NAME" >}} service:
+
+   1. Open the Windows Services manager (`services.msc`):
+
+      1. Right click on the Start Menu and select **Run**.
+
+      1. Type `services.msc` and click **OK**.
+
+   1. Right click on the service called **{{< param "PRODUCT_NAME" >}}**.
+
+   1. Click on **All Tasks > Restart**.
+
 ## Expose the UI to other machines
 
 By default, {{< param "PRODUCT_NAME" >}} listens on the local network for its HTTP
@@ -85,3 +116,4 @@ To expose the UI to other machines, complete the following steps:
      To listen on all interfaces, replace _`<LISTEN_ADDR>`_ with `0.0.0.0`.
 
 [UI]: ../../troubleshoot/debug/#alloy-ui
+[environment]: ../../reference/cli/environment-variables
diff --git a/docs/sources/reference/cli/environment-variables.md b/docs/sources/reference/cli/environment-variables.md
@@ -14,10 +14,15 @@ The following environment variables are supported:
 
 * `GODEBUG`
 * `HTTP_PROXY`
+* `HTTPS_PROXY`
+* `NO_PROXY`
 * `PPROF_MUTEX_PROFILING_PERCENT`
 * `PPROF_BLOCK_PROFILING_RATE`
 * `GOMEMLIMIT`
-
+* `AUTOMEMLIMIT`
+* `GOGC`
+* `GOMAXPROCS`
+* `GOTRACEBACK`
 
 Refer to the [Go runtime][runtime] documentation for more information about Go runtime environment variables.
 
@@ -31,9 +36,20 @@ You can use the `GODEBUG` environment variable to control the debugging variable
  `netdns`               | Force a resolver. Set to `go` for a pure Go resolver. Set to `cgo` or `win32` for a native resolver. |
  `netdns`               | Show resolver debugging information. Set to `1` for basic information. Set to `2` for verbose.       |
 
-## HTTP_PROXY
+## HTTP_PROXY, HTTPS_PROXY, NO_PROXY
+
+You can use the `HTTP_PROXY` environment variable to define the hostname or IP address of the proxy server for HTTP requests.  For example, you can set the proxy to `http://proxy.example.com`.
+
+You can use the `HTTPS_PROXY` environment variable to define the proxy server for HTTPS requests in the same manner as `HTTP_PROXY`.
 
-You can use the `HTTP_PROXY` environment variable to define the hostname or IP address of the proxy server.  For example, you can set the proxy to `http://proxy.example.com`.
+The `NO_PROXY` environment variable is used to define any hosts that should be excluded from proxying. `NO_PROXY` should contain a comma delimited list of any of the following options.
+
+ Option                 | Description                                                                                                    | Examples
+------------------------|----------------------------------------------------------------------------------------------------------------|---------
+ IP Address             | A single IP address (with optional port)                                                                       | `1.2.3.4` or `1.2.3.4:80`
+ CIDR Block             | A group of IP addresses that share a network prefix.                                                           | `1.2.3.4/8`
+ Domain                 | A domain name matches that name and all subdomains. A domain name with a leading "." matches subdomains only.  | `example.com` or `.example.com`
+ Asterisk               | A single asterisk indicates that no proxying should be done.                                                   | `*`
 
 ## PPROF_MUTEX_PROFILING_PERCENT
 
@@ -47,7 +63,7 @@ You can use the `PPROF_BLOCK_PROFILING_RATE` environment variable to define the
 * `1`: All mutexes are tracked.
 * A value greater than `1`: The number of nanoseconds to track mutexes.
 
-### GOMEMLIMIT
+## GOMEMLIMIT
 
 Usually, the [Go runtime][runtime] will release memory back to the operating system when requested.
 In some environments, this may cause issues such as Out Of Memory (OOM) errors.
@@ -61,7 +77,44 @@ For example, if you want to keep memory usage below `10GiB`, use `GOMEMLIMIT=9Gi
 
 #### Automatically set GOMEMLIMIT
 
-The `GOMEMLIMIT` environment variable is either automatically set to 90% of an available `cgroup` value, or you can explicitly set the  `GOMEMLIMIT` environment variable before you run  {{< param "PRODUCT_NAME" >}}.
+The `GOMEMLIMIT` environment variable is either automatically set to 90% of an available `cgroup` value using the [automemlimit] module, or you can explicitly set the  `GOMEMLIMIT` environment variable before you run  {{< param "PRODUCT_NAME" >}}.
+You can also change the 90% ratio by setting the `AUTOMEMLIMIT` environment variable to a float value between `0` and `1.0`.
 No changes will occur if the limit cannot be determined and you did not explicitly define a  `GOMEMLIMIT` value.
 
+## GOGC
+
+The `GOGC` environment variable controls the mechanism that triggers Go's garbage collection.
+It represents the garbage collection target percentage. A collection is triggered when the ratio 
+of freshly allocated data to live data remaining after the previous collection reaches this percentage.
+If you don't provide this variable, GOGC defaults to `100`.
+You can set `GOGC=off` to disable garbage collection.
+
+Configuring this value in conjunction with `GOMEMLIMIT` can help in situations where {{< param "PRODUCT_NAME" >}} 
+is consuming too much memory. Go provides a [very in-depth guide][gc_guide] to understanding `GOGC` and `GOMEMLIMIT`.
+
+## GOMAXPROCS
+
+The `GOMAXPROCS` environment variable defines the limit of OS threads that can simultaneously execute user-level Go code.
+This limit does not affect the number of threads that can be blocked in system calls on behalf of Go code and those
+threads are not counted against `GOMAXPROCS`.
+
+## GOTRACEBACK
+
+The `GOTRACEBACK` environment variable defines the behavior of the Go panic output.
+The standard panic output behavior is usually sufficient to debug and resolve an issue.
+If required, you can use this setting to collect additional information from the runtime.
+The following values are supported.
+
+Value            | Description                                                                     | Traces include runtime internal functions
+-----------------|---------------------------------------------------------------------------------|------------------------------------------
+ `none` or `0`   | Omit goroutine stack traces entirely from the panic output.                     | -
+ `single`        | Print the stack trace for the current goroutine.                                | No
+ `all` or `1`    | Print the stack traces for all user-created goroutines.                         | No
+ `system` or `2` | Print the stack traces for all user-created and runtime-created goroutines.     | Yes
+ `crash`         | Similar to `system`, but also triggers OS-specific additional behavior. For example, on Unix systems, this raises a SIGABRT to trigger a code dump.   | Yes
+ `wer`           | Similar to `crash`, but does not disable Windows Error Reporting.               | Yes
+
 [runtime]: https://pkg.go.dev/runtime
+[automemlimit]: https://github.com/KimMachineGun/automemlimit
+[gc_guide]: https://tip.golang.org/doc/gc-guide#GOGC
+[Windows]: ../../../configure/windows
diff --git a/docs/sources/troubleshoot/support_bundle.md b/docs/sources/troubleshoot/support_bundle.md
@@ -36,6 +36,7 @@ to verify the code used to generate these bundles.
 A support bundle contains the following data:
 * `alloy-components.json` contains information about the [components][components] running on this {{< param "PRODUCT_NAME" >}} instance, generated by the 
 `/api/v0/web/components` endpoint.
+* `alloy-environment.txt` contains the values of several environment variables relevant to the golang runtime.
 * `alloy-logs.txt` contains the logs during the bundle generation.
 * `alloy-metadata.yaml` contains the {{< param "PRODUCT_NAME" >}} build version and the installation's operating system, architecture, and uptime.
 * `alloy-metrics-sample-start.txt` contains a snapshot of the internal metrics for {{< param "PRODUCT_NAME" >}} at the start of the bundle collection.

diff --git a/internal/service/http/supportbundle.go b/internal/service/http/supportbundle.go
@@ -7,6 +7,7 @@ import (
 	"fmt"
 	"io"
 	"net/http"
+	"os"
 	"path/filepath"
 	"runtime"
 	"runtime/pprof"
@@ -28,17 +29,18 @@ type SupportBundleContext struct {
 
 // Bundle collects all the data that is exposed as a support bundle.
 type Bundle struct {
-	meta              []byte
-	alloyMetricsStart []byte
-	alloyMetricsEnd   []byte
-	components        []byte
-	peers             []byte
-	runtimeFlags      []byte
-	heapBuf           *bytes.Buffer
-	goroutineBuf      *bytes.Buffer
-	blockBuf          *bytes.Buffer
-	mutexBuf          *bytes.Buffer
-	cpuBuf            *bytes.Buffer
+	meta                 []byte
+	alloyMetricsStart    []byte
+	alloyMetricsEnd      []byte
+	components           []byte
+	peers                []byte
+	runtimeFlags         []byte
+	environmentVariables []byte
+	heapBuf              *bytes.Buffer
+	goroutineBuf         *bytes.Buffer
+	blockBuf             *bytes.Buffer
+	mutexBuf             *bytes.Buffer
+	cpuBuf               *bytes.Buffer
 }
 
 // Metadata contains general runtime information about the current Alloy environment.
@@ -141,17 +143,18 @@ func ExportSupportBundle(ctx context.Context, runtimeFlags []string, srvAddress
 	// Finally, bundle everything up to be served, either as a zip from
 	// memory, or exported to a directory.
 	bundle := &Bundle{
-		meta:              meta,
-		alloyMetricsStart: alloyMetricsStart,
-		alloyMetricsEnd:   alloyMetricsEnd,
-		components:        components,
-		peers:             peers,
-		runtimeFlags:      []byte(strings.Join(runtimeFlags, "\n")),
-		heapBuf:           &heapBuf,
-		goroutineBuf:      &goroutineBuf,
-		blockBuf:          &blockBuf,
-		mutexBuf:          &mutexBuf,
-		cpuBuf:            &cpuBuf,
+		meta:                 meta,
+		alloyMetricsStart:    alloyMetricsStart,
+		alloyMetricsEnd:      alloyMetricsEnd,
+		components:           components,
+		peers:                peers,
+		runtimeFlags:         []byte(strings.Join(runtimeFlags, "\n")),
+		environmentVariables: []byte(strings.Join(retrieveEnvironmentVariables(), "\n")),
+		heapBuf:              &heapBuf,
+		goroutineBuf:         &goroutineBuf,
+		blockBuf:             &blockBuf,
+		mutexBuf:             &mutexBuf,
+		cpuBuf:               &cpuBuf,
 	}
 
 	return bundle, nil
@@ -171,6 +174,32 @@ func retrieveAPIEndpoint(httpClient http.Client, srvAddress, endpoint string) ([
 	return res, nil
 }
 
+func retrieveEnvironmentVariables() []string {
+	relevantVariables := []string{
+		"AUTOMEMLIMIT",
+		"GODEBUG",
+		"GOGC",
+		"GOMAXPROCS",
+		"GOMEMLIMIT",
+		"HOSTNAME",
+		// the proxy variables can be provided either uppercased or lowercased
+		"HTTP_PROXY",
+		"http_proxy",
+		"HTTPS_PROXY",
+		"https_proxy",
+		"NO_PROXY",
+		"no_proxy",
+		"PPROF_BLOCK_PROFILING_RATE",
+		"PPROF_MUTEX_PROFILING_PERCENT",
+	}
+	values := []string{}
+	for _, v := range relevantVariables {
+		values = append(values, fmt.Sprintf("%s=%s", v, os.Getenv(v)))
+	}
+
+	return values
+}
+
 // ServeSupportBundle the collected data and logs as a zip file over the given
 // http.ResponseWriter.
 func ServeSupportBundle(rw http.ResponseWriter, b *Bundle, logsBuf *bytes.Buffer) error {
@@ -185,6 +214,7 @@ func ServeSupportBundle(rw http.ResponseWriter, b *Bundle, logsBuf *bytes.Buffer
 		"alloy-metrics-sample-start.txt": b.alloyMetricsStart,
 		"alloy-metrics-sample-end.txt":   b.alloyMetricsEnd,
 		"alloy-runtime-flags.txt":        b.runtimeFlags,
+		"alloy-environment.txt":          b.environmentVariables,
 		"alloy-logs.txt":                 logsBuf.Bytes(),
 		"pprof/cpu.pprof":                b.cpuBuf.Bytes(),
 		"pprof/heap.pprof":               b.heapBuf.Bytes(),