-
Notifications
You must be signed in to change notification settings - Fork 976
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
add troubleshooting snippet and reference it on three pages (#6028)
## What are you changing in this pull request and why? Created a snippet for a customer PrivateLink troubleshooting guide and added it to the bottom of three PL setup pages: Redshift, Postgres, and VCS. ## Checklist - [X] I have reviewed the [Content style guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md) so my content adheres to these guidelines. - [ ] The topic I'm writing about is for specific dbt version(s) and I have versioned it according to the [version a whole page](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version) and/or [version a block of content](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-blocks-of-content) guidelines. - [ ] I have added checklist item(s) to this list for anything anything that needs to happen before this PR is merged, such as "needs technical review" or "change base branch." --------- Co-authored-by: Matt Shaver <[email protected]>
- Loading branch information
1 parent
04bafea
commit 776a268
Showing
5 changed files
with
161 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,98 @@ | ||
| [ | ||
| { | ||
| "host": "jd43444.east-us-2.privatelink.snowflakecomputing.com", | ||
| "port": 443, | ||
| "type": "SNOWFLAKE_DEPLOYMENT" | ||
| }, | ||
| { | ||
| "host": "mgb-mgbdev.privatelink.snowflakecomputing.com", | ||
| "port": 443, | ||
| "type": "SNOWFLAKE_DEPLOYMENT_REGIONLESS" | ||
| }, | ||
| { | ||
| "host": "r3iqwjsfcb1stg.blob.core.windows.net", | ||
| "port": 443, | ||
| "type": "STAGE" | ||
| }, | ||
| { | ||
| "host": "sfc-repo.snowflakecomputing.com", | ||
| "port": 443, | ||
| "type": "SNOWSQL_REPO" | ||
| }, | ||
| { | ||
| "host": "client-telemetry.snowflakecomputing.com", | ||
| "port": 443, | ||
| "type": "OUT_OF_BAND_TELEMETRY" | ||
| }, | ||
| { | ||
| "host": "app-mgb-mgbdevenv.privatelink.snowflakecomputing.com", | ||
| "port": 443, | ||
| "type": "SNOWSIGHT_CLIENT_FAILOVER" | ||
| }, | ||
| { | ||
| "host": "mgb-mgbdevenv.privatelink.snowflakecomputing.com", | ||
| "port": 443, | ||
| "type": "CLIENT_FAILOVER" | ||
| }, | ||
| { | ||
| "host": "ocsp.mgb-mgbdevenv.privatelink.snowflakecomputing.com", | ||
| "port": 80, | ||
| "type": "OCSP_CLIENT_FAILOVER" | ||
| }, | ||
| { | ||
| "host": "ocsp.app-mgb-mgbdevenv.privatelink.snowflakecomputing.com", | ||
| "port": 80, | ||
| "type": "OCSP_CLIENT_FAILOVER" | ||
| }, | ||
| { | ||
| "host": "ocsp.jd43444.east-us-2.privatelink.snowflakecomputing.com", | ||
| "port": 80, | ||
| "type": "OCSP_CACHE" | ||
| }, | ||
| { | ||
| "host": "ocsp.mgb-mgbdev.privatelink.snowflakecomputing.com", | ||
| "port": 80, | ||
| "type": "OCSP_CACHE_REGIONLESS" | ||
| }, | ||
| { | ||
| "host": "api-73a252bb.duosecurity.com", | ||
| "port": 443, | ||
| "type": "DUO_SECURITY" | ||
| }, | ||
| { | ||
| "host": "app-mgb-mgbdev.privatelink.snowflakecomputing.com", | ||
| "port": 443, | ||
| "type": "SNOWSIGHT_DEPLOYMENT_REGIONLESS" | ||
| }, | ||
| { | ||
| "host": "app.east-us-2.privatelink.snowflakecomputing.com", | ||
| "port": 443, | ||
| "type": "SNOWSIGHT_DEPLOYMENT" | ||
| } | ||
| ] | ||
|
|
||
| # Current | ||
| { | ||
| "regionless-snowsight-privatelink-url": "app-mgb-mgbdev.privatelink.snowflakecomputing.com", | ||
| "privatelink-account-name": "jd43444.east-us-2.privatelink", | ||
| "snowsight-privatelink-url": "app.east-us-2.privatelink.snowflakecomputing.com", | ||
| "regionless-privatelink-ocsp-url": "ocsp.mgb-mgbdev.privatelink.snowflakecomputing.com", | ||
| "privatelink-account-url": "jd43444.east-us-2.privatelink.snowflakecomputing.com", | ||
| "privatelink-connection-ocsp-urls": "[ocsp.mgb-mgbdevenv.privatelink.snowflakecomputing.com, ocsp.app-mgb-mgbdevenv.privatelink.snowflakecomputing.com]", | ||
| "privatelink-pls-id": "sf-pvlinksvc-azeastus2.cf82bce2-be9d-4dc6-92ee-3de5fb04d191.eastus2.azure.privatelinkservice", | ||
| "regionless-privatelink-account-url": "mgb-mgbdev.privatelink.snowflakecomputing.com", | ||
| "privatelink_ocsp-url": "ocsp.jd43444.east-us-2.privatelink.snowflakecomputing.com", | ||
| "privatelink-connection-urls": "[mgb-mgbdevenv.privatelink.snowflakecomputing.com, app-mgb-mgbdevenv.privatelink.snowflakecomputing.com]" | ||
| } | ||
|
|
||
| # Received from customer 02/2023 | ||
| { | ||
| "regionless-snowsight-privatelink-url": "app-mgb-mgbdev.privatelink.snowflakecomputing.com", | ||
| "privatelink-account-name": "jd43444.east-us-2.privatelink", | ||
| "snowsight-privatelink-url": "app.east-us-2.privatelink.snowflakecomputing.com", | ||
| "privatelink-account-url": "jd43444.east-us-2.privatelink.snowflakecomputing.com", | ||
| "privatelink-pls-id": "sf-pvlinksvc-azeastus2.cf82bce2-be9d-4dc6-92ee-3de5fb04d191.eastus2.azure.privatelinkservice", | ||
| "regionless-privatelink-account-url": "mgb-mgbdev.privatelink.snowflakecomputing.com", | ||
| "privatelink_ocsp-url": "ocsp.jd43444.east-us-2.privatelink.snowflakecomputing.com", | ||
| "privatelink-connection-urls": "[CLIENT_FAILOVER,mgb-mgbdevenv.privatelink.snowflakecomputing.com]" | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| ## Troubleshooting | ||
|
|
||
| If the PrivateLink endpoint has been provisioned and configured in dbt Cloud, but connectivity is still failing, check the following in your networking setup to ensure requests and responses can be successfully routed between dbt Cloud and the backing service. | ||
|
|
||
| ### Configuration | ||
|
|
||
| Start with the configuration: | ||
|
|
||
| <Expandable alt_header="1. NLB Security Group"> | ||
|
|
||
| The Network Load Balancer (NLB) associated with the VPC Endpoint Service must either not have an associated Security Group or the Security Group must have a rule that allows requests from the appropriate dbt Cloud _private CIDR(s)_. Note that this differs from the static public IPs listed on the dbt Cloud Connection page. dbt Support can provide the correct private CIDR(s) upon request. | ||
| - **Note***: To test if this is the issue, temporarily adding an allow rule of `10.0.0.0/8` should allow connectivity until the rule can be refined to a smaller CIDR | ||
|
|
||
| </Expandable> | ||
|
|
||
| <Expandable alt_header="2. NLB Listener and Target Group"> | ||
|
|
||
| Check that there is a Listener connected to the NLB that matches the port that dbt Cloud is trying to connect to. This Listener must have a configured action to forward to a Target Group with targets that point to your backing service. At least one (but preferably all) of these targets must be **Healthy**. Unhealthy targets could suggest that the backing service is, in fact, unhealthy or that the service is protected by a Security Group that doesn't allow requests from the NLB. | ||
|
|
||
| </Expandable> | ||
|
|
||
| <Expandable alt_header="3. Cross-zone Load Balancing"> | ||
|
|
||
| Check that _Cross-zone load balancing_ is enabled for your NLB (check the **Attributes** tab of the NLB in the AWS console). If this is disabled, and the zones that dbt Cloud is connected to are misaligned with the zones where the service is running, requests may not be able to be routed correctly. Enabling cross-zone load balancing will also make the connection more resilient in the case of a failover in a zone outage scenario. Cross-zone connectivity may incur additional data transfer charges, though this should be minimal for requests from dbt Cloud. | ||
|
|
||
| </Expandable> | ||
|
|
||
| <Expandable alt_header="4. Routing tables and ACLs"> | ||
|
|
||
| If all the above check out, it may be possible that requests are not routing correctly within the private network. This could be due to a misconfiguration in the VPCs routing tables or access control lists. Review these settings with your network administrator to ensure that requests can be routed from the VPC Endpoint Service to the backing service and that the response can be returned to the VPC Endpoint Service. One way to test this is to create a VPC endpoint in another VPC in your network to test that connectivity is working independent of dbt's connection. | ||
|
|
||
| </Expandable> | ||
|
|
||
| ### Monitoring | ||
|
|
||
| To help isolate connection issues over a PrivateLink connection from dbt Cloud, there are a few monitoring sources that can be used to verify request activity. Requests must first be sent to the endpoint to see anything in the monitoring. Contact dbt Support to understand when connection testing occurred or request new connection attempts. Use these times to correlate with activity in the following monitoring sources. | ||
|
|
||
| <Expandable alt_header="VPC Endpoint Service Monitoring"> | ||
|
|
||
| In the AWS Console, navigate to VPC -> Endpoint Services. Select the Endpoint Service being tested and click the **Monitoring** tab. Update the time selection to include when test connection attempts were sent. If there is activity in the _New connections_ and _Bytes processed_ graphs, then requests have been received by the Endpoint Service, suggesting that the dbt endpoint is routing properly. | ||
|
|
||
| </Expandable> | ||
|
|
||
| <Expandable alt_header="NLB Monitoring"> | ||
|
|
||
| In the AWS Console, navigate to EC2 -> Load Balancers. Select the Network Load Balancer (NLB) being tested and click the **Monitoring** tab. Update the time selection to include when test connection attempts were sent. If there is activity in the _New flow count_ and _Processed bytes_ graphs, then requests have been received by the NLB from the Endpoint Service, suggesting the NLB Listener, Target Group, and Security Group are correctly configured. | ||
|
|
||
| </Expandable> | ||
|
|
||
| <Expandable alt_header="VPC Flow Logs"> | ||
|
|
||
| VPC Flow Logs can provide various helpful information for requests being routed through your VPCs, though they can sometimes be challenging to locate and interpret. Flow logs can be written to either S3 or CloudWatch Logs, so determine the availability of these logs for your VPC and query them accordingly. Flow logs record the Elastic Network Interface (ENI) ID, source and destination IP and port, and whether the request was accepted or rejected by the security group and/or network ACL. This can be useful in understanding if a request arrived at a certain network interface and whether that request was accepted, potentially illuminating overly restrictive rules. For more information on accessing and interpreting VPC Flow Logs, see the related [AWS documentation](https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs.html). | ||
|
|
||
| </Expandable> |