Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: switch to subsystem loggers for HTTP logging #739

Closed
wants to merge 1 commit into from

Conversation

ctreatma
Copy link
Contributor

@ctreatma ctreatma commented Jul 23, 2024

This addresses a linting issue that was bypassed in #736 to constrain the scope of that PR. The logging transport we were previously using for HTTP clients is deprecated, but the new Terraform logging plugin is more context-aware and requires--or at least enables--specifying separate subsystems for each provider component that will write logs.

This adds a new HTTPLoggingTransport type that encapsulates the logic of creating a subsystem HTTP logging transport, ensuring the necessary subsystem exists in any relevant HTTP request context, and masking any sensitive values in the request.

The Config type is updated to create a separate HTTPLoggingTransport for each API client. For now, the log level for all of the new transports is wired up to the TF_LOG environment variable. This matches the previous behavior of the deprecated HTTP logging transport.

// This is the initial env var for HTTP logging for backwards
// compatibility. We could now introduce service-specific
// variables, if desired, for more granular control of logging
HTTPLoggingEnvVar = "TF_LOG"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using TF_LOG to control the log level for all of the new subsystem loggers replicates prior behavior but may not be desirable. Now that we have these loggers in place, we could add support for one or more of TF_LOG_PROVIDER_EQUINIX (to specify log level for the whole provider) or TF_LOG_EQUINIX_<service> (to allow more granular configuration of provider logging).

@ctreatma
Copy link
Contributor Author

The log format for HTTP logs is different with the new loggers. Previously we would see logs like this:

---[ REQUEST ]---------------------------------------
GET /metal/v1//devices/5f286421-214e-4366-9358-f77a6d3ea4b0?include=project HTTP/1.1
Host: api.equinix.com
User-Agent: HashiCorp Terraform/1.8.1 (+https://www.terraform.io) Terraform Plugin SDK/unknown-version terraform-provider-equinix/dev equinix-sdk-go/0.42.0
Accept: application/json

-----------------------------------------------------
2024/07/23 10:45:45 [DEBUG] Equinix Metal (metal-go) API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Content-Length: 19139
Cache-Control: max-age=0, private, must-revalidate
Connection: keep-alive
Content-Type: application/json; charset=utf-8

{
 "id": "5f286421-214e-4366-9358-f77a6d3ea4b0",
 "short_id": "5f286421",
 "hostname": "tfacc-test-device-8858291862441735794",
 "description": "",
 "tags": [
  "8858291862441735794"
 ], ...
}

But the new loggers produce messages like this:

2024-07-23T14:47:34.535-0500 [DEBUG] equinix.Equinix Metal (metal-go): Sending HTTP Request: tf_http_trans_id=bce44d87-fd74-f11e-c4cd-2b90f42d73ab tf_http_req_body="" tf_http_req_version=HTTP/1.1 Host=api.equinix.com tf_http_op_type=request Accept=application/json X-Auth-Token="***" tf_http_req_method=DELETE User-Agent="HashiCorp Terraform/ (+https://www.terraform.io) Terraform Plugin Framework/unknown-version terraform-provider-equinix/dev equinix-sdk-go/0.42.0" tf_http_req_uri=/metal/v1//projects/701296d1-3de0-41fd-8f04-2142bbe1c733 Accept-Encoding=gzip
2024-07-23T14:47:35.160-0500 [DEBUG] equinix.Equinix Metal (metal-go): Received HTTP Response: tf_http_res_body="" X-Xss-Protection="1; mode=block" Cache-Control=no-cache X-Request-Id=3945d28e69a415cd76c5e5d172d75cc1 tf_http_res_status_code=204 X-Content-Type-Options=nosniff tf_http_trans_id=bce44d87-fd74-f11e-c4cd-2b90f42d73ab X-Permitted-Cross-Domain-Policies=none Referrer-Policy=strict-origin-when-cross-origin Date="Tue, 23 Jul 2024 19:47:35 GMT" tf_http_op_type=response tf_http_res_version=HTTP/1.1 X-Frame-Options=SAMEORIGIN X-Download-Options=noopen Connection=keep-alive Strict-Transport-Security="max-age=31536000; includeSubDomains" tf_http_res_status_reason="204 No Content"

@ctreatma
Copy link
Contributor Author

Another alternative would be to remove the HTTP logging transport entirely and update the Go SDK(s) to enable debug mode in the presence of specific environment variables instead.

// Subsystems for API client logging
packetClientSubsystem = "Equinix Metal (packngo)"
equinixClientSubsystem = "Equinix"
metalClientSubsystem = "Equinix Metal (metal-go)"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we missed a spot in the SDK conversion. This could either be (equinix-sdk-go) to reflect the previous convention of tagging with the SDK name or it could be (metalv1) to align with the fabricClientSubsystem format, in which case perhaps we could remove packetClientSubsystem, since the logs would already include User-Agent to see which requests came from which SDK.

Suggested change
metalClientSubsystem = "Equinix Metal (metal-go)"
metalClientSubsystem = "Equinix Metal (equinix-sdk-go)"

@ctreatma ctreatma force-pushed the subsystem-loggers branch from bd289d0 to a523401 Compare July 25, 2024 20:15

func (t *HTTPLoggingTransport) RoundTrip(r *http.Request) (*http.Response, error) {
ctx := tflog.NewSubsystem(r.Context(), t.subsystem, tflog.WithLevel(t.level))
ctx = tflog.SubsystemMaskFieldValuesWithFieldKeys(ctx, t.subsystem, t.sensitiveFields...)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new subsystem loggers add support for masking sensitive values, which is nice.

@ctreatma ctreatma closed this Nov 12, 2024
@ctreatma ctreatma deleted the subsystem-loggers branch November 12, 2024 19:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant