From 8c089e9f558909eb6dffe9cba195e9a080c64e4e Mon Sep 17 00:00:00 2001 From: Pat Kearns Date: Wed, 21 Jun 2023 14:37:43 +1000 Subject: [PATCH 1/5] Update connect-redshift-postgresql-alloydb.md --- .../connect-redshift-postgresql-alloydb.md | 34 +++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md b/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md index 72fe9e0449c..f288a3d9e30 100644 --- a/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md +++ b/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md @@ -65,6 +65,40 @@ To configure the SSH tunnel in dbt Cloud, you'll need to provide the hostname/IP The Bastion server should now be ready for dbt Cloud to use as a tunnel into the Redshift environment. +#### Intermittent Connection Issues + +
+ Database Error - could not connect to server: Connection timed out +
+
When you configure a connection to a database via an SSH tunnel -- typically you have the following components in play: + - An Elastic Load Balancer (ELB) or Network Load Balancing (NLB) instance. + - A bastion host (aka jump server) running the `sshd` process + - A Database (ex. Redshift cluster) +dbt Cloud establishes an SSH tunnel by connecting through the ELB/NLB to the `sshd` process which then is responsible for passing traffic to the database. +When dbt initiates a job run, it establishes an SSH tunnel at the beginning of the job run and if at any point the SSH tunnel fails, the job will fail. + +The most common causes of tunnel failures are: + - The SSH daemon terminates the session due to an idle timeout + - The connection is terminated by ELB or NLB due to an idle timeout + +dbt Cloud sets a value for its SSh tunnel called `ServerAliveInterval` and `ServerAliveCountMax` that polls the connection every 30 seconds and the underlying OS in our run "pods" will terminate the connection if the `sshd` process fails to respond after 300s. This will, in many cases, prevent an idle timeout entirely so longer as the customer is not using ELB with a firewall-level idle timeout of less than 30 seconds. However, if the customer is using ELB and is using an Idle Connection Timeout of less than 30s, this will be insufficient to prevent tunnels from being terminated. + +Additionally, some versions of Linux used on bastion hosts use a verison of `sshd` with additional idle timeout settings: +`ClientAliveCountMax` + This value sets the number of client alive messages which may be sent without `sshd` receiving any messages back from the client. If this threshold is reached while client alive messages are being sent, sshd will disconnect the client, terminating the session. The client alive mechanism is helpful when the client or server needs to know when a connection has become inactive. The default value is 3. +`ClientAliveInterval` + This value sets a timeout interval in seconds after which if no data has been received from the client, `sshd` will send a message through the encrypted channel to request a response from the client. The default is 0, indicating that these messages will not be sent to the client. + +Using default values, tunnels could be terminated prematurely by `sshd`. To solve this problem, the `/etc/ssh/sshd_config` file on the bastion host can be configured with the following values: +`ClientAliveCountMax` 10 +`ClientAliveInterval` 30 +where `ClientAliveCountMax` should be set to a non-zero value and `ClientAliveInterval` should be a value less than the ELB or NLB idle timeout value. +
+
+
+could not connect to server: Connection timed out + + ## Configuration To learn how to optimize performance with data platform-specific configurations in dbt Cloud, refer to [Redshift-specific configuration](/reference/resource-configs/redshift-configs). From eda6273126aa0e912e4bfeb11b833a1ff47361fc Mon Sep 17 00:00:00 2001 From: Pat Kearns Date: Wed, 21 Jun 2023 14:43:04 +1000 Subject: [PATCH 2/5] Update connect-redshift-postgresql-alloydb.md --- .../connect-redshift-postgresql-alloydb.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md b/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md index f288a3d9e30..aff58319123 100644 --- a/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md +++ b/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md @@ -81,9 +81,9 @@ The most common causes of tunnel failures are: - The SSH daemon terminates the session due to an idle timeout - The connection is terminated by ELB or NLB due to an idle timeout -dbt Cloud sets a value for its SSh tunnel called `ServerAliveInterval` and `ServerAliveCountMax` that polls the connection every 30 seconds and the underlying OS in our run "pods" will terminate the connection if the `sshd` process fails to respond after 300s. This will, in many cases, prevent an idle timeout entirely so longer as the customer is not using ELB with a firewall-level idle timeout of less than 30 seconds. However, if the customer is using ELB and is using an Idle Connection Timeout of less than 30s, this will be insufficient to prevent tunnels from being terminated. +dbt Cloud sets a value for its SSH tunnel called `ServerAliveInterval` and `ServerAliveCountMax` that polls the connection every 30 seconds and the underlying OS in our run "pods" will terminate the connection if the `sshd` process fails to respond after 300s. This will, in many cases, prevent an idle timeout entirely so longer as the customer is not using ELB with a firewall-level idle timeout of less than 30 seconds. However, if the customer is using ELB and is using an Idle Connection Timeout of less than 30s, this will be insufficient to prevent tunnels from being terminated. -Additionally, some versions of Linux used on bastion hosts use a verison of `sshd` with additional idle timeout settings: +Some versions of Linux used on bastion hosts use a verison of `sshd` with additional idle timeout settings: `ClientAliveCountMax` This value sets the number of client alive messages which may be sent without `sshd` receiving any messages back from the client. If this threshold is reached while client alive messages are being sent, sshd will disconnect the client, terminating the session. The client alive mechanism is helpful when the client or server needs to know when a connection has become inactive. The default value is 3. `ClientAliveInterval` @@ -92,7 +92,7 @@ Additionally, some versions of Linux used on bastion hosts use a verison of `ssh Using default values, tunnels could be terminated prematurely by `sshd`. To solve this problem, the `/etc/ssh/sshd_config` file on the bastion host can be configured with the following values: `ClientAliveCountMax` 10 `ClientAliveInterval` 30 -where `ClientAliveCountMax` should be set to a non-zero value and `ClientAliveInterval` should be a value less than the ELB or NLB idle timeout value. +where `ClientAliveCountMax` should be set to a non-zero value and `ClientAliveInterval` should be a value less than the ELB or NLB idle timeout value. Using the suggested values, unresponsive SSH clients will be disconnected after approximately 300 seconds. From d02c71fdc19e9f175a32926a76dc7c7b89fa8ec1 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 7 Nov 2023 10:05:52 +0000 Subject: [PATCH 3/5] Update connect-redshift-postgresql-alloydb.md --- .../connect-redshift-postgresql-alloydb.md | 22 ++++++++----------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md b/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md index ffe8eef36cc..0b89003bed5 100644 --- a/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md +++ b/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md @@ -61,27 +61,26 @@ chmod 600 ~/.ssh/authorized_keys The Bastion server should now be ready for dbt Cloud to use as a tunnel into the Redshift environment. -#### Intermittent Connection Issues +#### Intermittent connection issues
Database Error - could not connect to server: Connection timed out -
-
When you configure a connection to a database via an SSH tunnel -- typically you have the following components in play: + When you configure a connection to a database via an SSH tunnel -- typically you have the following components in play: - An Elastic Load Balancer (ELB) or Network Load Balancing (NLB) instance. - - A bastion host (aka jump server) running the `sshd` process - - A Database (ex. Redshift cluster) -dbt Cloud establishes an SSH tunnel by connecting through the ELB/NLB to the `sshd` process which then is responsible for passing traffic to the database. -When dbt initiates a job run, it establishes an SSH tunnel at the beginning of the job run and if at any point the SSH tunnel fails, the job will fail. + - A bastion host (aka jump server) running the sshd process + - A Database (such as Redshift cluster) +dbt Cloud establishes an SSH tunnel by connecting through the ELB/NLB to the sshd process which then is responsible for passing traffic to the database. +When dbt initiates a job run, it establishes an SSH tunnel at the beginning of the job run, and if at any point the SSH tunnel fails, the job will fail. The most common causes of tunnel failures are: - The SSH daemon terminates the session due to an idle timeout - The connection is terminated by ELB or NLB due to an idle timeout -dbt Cloud sets a value for its SSH tunnel called `ServerAliveInterval` and `ServerAliveCountMax` that polls the connection every 30 seconds and the underlying OS in our run "pods" will terminate the connection if the `sshd` process fails to respond after 300s. This will, in many cases, prevent an idle timeout entirely so longer as the customer is not using ELB with a firewall-level idle timeout of less than 30 seconds. However, if the customer is using ELB and is using an Idle Connection Timeout of less than 30s, this will be insufficient to prevent tunnels from being terminated. +dbt Cloud sets a value for its SSH tunnel called `ServerAliveInterval` and `ServerAliveCountMax` that polls the connection every 30 seconds and the underlying OS in our run "pods" will terminate the connection if the `sshd` process fails to respond after 300s. This will, in many cases, prevent an idle timeout entirely so long as the customer is not using ELB with a firewall-level idle timeout of less than 30 seconds. However, if the customer is using ELB and is using an Idle Connection Timeout of less than 30s, this will be insufficient to prevent tunnels from being terminated. -Some versions of Linux used on bastion hosts use a verison of `sshd` with additional idle timeout settings: +Some versions of Linux used on bastion hosts use a version of `sshd` with additional idle timeout settings: `ClientAliveCountMax` - This value sets the number of client alive messages which may be sent without `sshd` receiving any messages back from the client. If this threshold is reached while client alive messages are being sent, sshd will disconnect the client, terminating the session. The client alive mechanism is helpful when the client or server needs to know when a connection has become inactive. The default value is 3. + This value sets the number of client alive messages that may be sent without `sshd` receiving any messages back from the client. If this threshold is reached while client alive messages are being sent, `sshd` will disconnect the client, terminating the session. The client-alive mechanism is helpful when the client or server needs to know when a connection has become inactive. The default value is 3. `ClientAliveInterval` This value sets a timeout interval in seconds after which if no data has been received from the client, `sshd` will send a message through the encrypted channel to request a response from the client. The default is 0, indicating that these messages will not be sent to the client. @@ -89,10 +88,7 @@ Using default values, tunnels could be terminated prematurely by `sshd`. To solv `ClientAliveCountMax` 10 `ClientAliveInterval` 30 where `ClientAliveCountMax` should be set to a non-zero value and `ClientAliveInterval` should be a value less than the ELB or NLB idle timeout value. Using the suggested values, unresponsive SSH clients will be disconnected after approximately 300 seconds. -
-
-could not connect to server: Connection timed out ## Configuration From 122a20657b9b26ea3d233ea32ec8e1c89561774d Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 7 Nov 2023 11:40:50 +0000 Subject: [PATCH 4/5] Update connect-redshift-postgresql-alloydb.md --- .../connect-redshift-postgresql-alloydb.md | 47 ++++++++++--------- 1 file changed, 24 insertions(+), 23 deletions(-) diff --git a/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md b/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md index 0b89003bed5..c6caeca638b 100644 --- a/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md +++ b/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md @@ -65,29 +65,30 @@ The Bastion server should now be ready for dbt Cloud to use as a tunnel into the
Database Error - could not connect to server: Connection timed out - When you configure a connection to a database via an SSH tunnel -- typically you have the following components in play: - - An Elastic Load Balancer (ELB) or Network Load Balancing (NLB) instance. - - A bastion host (aka jump server) running the sshd process - - A Database (such as Redshift cluster) -dbt Cloud establishes an SSH tunnel by connecting through the ELB/NLB to the sshd process which then is responsible for passing traffic to the database. -When dbt initiates a job run, it establishes an SSH tunnel at the beginning of the job run, and if at any point the SSH tunnel fails, the job will fail. - -The most common causes of tunnel failures are: - - The SSH daemon terminates the session due to an idle timeout - - The connection is terminated by ELB or NLB due to an idle timeout - -dbt Cloud sets a value for its SSH tunnel called `ServerAliveInterval` and `ServerAliveCountMax` that polls the connection every 30 seconds and the underlying OS in our run "pods" will terminate the connection if the `sshd` process fails to respond after 300s. This will, in many cases, prevent an idle timeout entirely so long as the customer is not using ELB with a firewall-level idle timeout of less than 30 seconds. However, if the customer is using ELB and is using an Idle Connection Timeout of less than 30s, this will be insufficient to prevent tunnels from being terminated. - -Some versions of Linux used on bastion hosts use a version of `sshd` with additional idle timeout settings: -`ClientAliveCountMax` - This value sets the number of client alive messages that may be sent without `sshd` receiving any messages back from the client. If this threshold is reached while client alive messages are being sent, `sshd` will disconnect the client, terminating the session. The client-alive mechanism is helpful when the client or server needs to know when a connection has become inactive. The default value is 3. -`ClientAliveInterval` - This value sets a timeout interval in seconds after which if no data has been received from the client, `sshd` will send a message through the encrypted channel to request a response from the client. The default is 0, indicating that these messages will not be sent to the client. - -Using default values, tunnels could be terminated prematurely by `sshd`. To solve this problem, the `/etc/ssh/sshd_config` file on the bastion host can be configured with the following values: -`ClientAliveCountMax` 10 -`ClientAliveInterval` 30 -where `ClientAliveCountMax` should be set to a non-zero value and `ClientAliveInterval` should be a value less than the ELB or NLB idle timeout value. Using the suggested values, unresponsive SSH clients will be disconnected after approximately 300 seconds. + You will have the following components when you configure a connection to a database using an SSH tunnel:
+ - An Elastic Load Balancer (ELB) or Network Load Balancing (NLB) instance.
+ - A bastion host (or jump server) running the sshd process.
+ - A Database (such as Redshift cluster)

+ +dbt Cloud establishes an SSH tunnel connection through the ELB or NLB to the sshd process. This is responsible for routing traffic to the database. When dbt initiates a job run, an SSH tunnel is created at the start of the run. If this SSH tunnel fails at any point, the job will also fail.
+ + The most common causes of tunnel failures are:
+ - The SSH daemon terminates the session due to an idle timeout.
+ - The ELB or NLB terminates the connection when it's idle.

+ + dbt Cloud sets a value for its SSH tunnel called `ServerAliveInterval` and `ServerAliveCountMax` that polls the connection every 30 seconds and the underlying OS in our run "pods" will terminate the connection if the `sshd` process fails to respond after 300s. This will, in many cases, prevent an idle timeout entirely so long as the customer is not using ELB with a firewall-level idle timeout of less than 30 seconds. However, if the customer is using ELB and is using an Idle Connection Timeout of less than 30s, this will be insufficient to prevent tunnels from being terminated.

+ + Some versions of Linux used on bastion hosts use a version of `sshd` with additional idle timeout settings `ClientAliveCountMax`. This value sets the number of client alive messages that may be sent without `sshd` receiving any messages back from the client. If this threshold is reached while client alive messages are being sent, `sshd` will disconnect the client, terminating the session. The client-alive mechanism is helpful when the client or server needs to know when a connection has become inactive. The default value is 3.

+ `ClientAliveInterval`

+ + This value sets a timeout interval in seconds after which if no data has been received from the client, `sshd` will send a message through the encrypted channel to request a response from the client. The default is 0, indicating that these messages will not be sent to the client.

+ +Using default values, tunnels could be terminated prematurely by `sshd`. To solve this problem, the `/etc/ssh/sshd_config` file on the bastion host can be configured with the following values:
+`ClientAliveCountMax` 10
+`ClientAliveInterval` 30
+where `ClientAliveCountMax` should be set to a non-zero value and `ClientAliveInterval` should be a value less than the ELB or NLB idle timeout value.

+ +With these settings, unresponsive SSH clients will be disconnected after approximately 300 seconds, helping to prevent tunnel failures.
From c52fac0a4d2d1f07a5a9e10db996a616f603d84c Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 7 Nov 2023 11:51:27 +0000 Subject: [PATCH 5/5] Update connect-redshift-postgresql-alloydb.md --- .../connect-redshift-postgresql-alloydb.md | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md b/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md index c6caeca638b..486aa787936 100644 --- a/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md +++ b/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md @@ -15,7 +15,7 @@ The following fields are required when creating a Postgres, Redshift, or AlloyDB **Note**: When you set up a Redshift or Postgres connection in dbt Cloud, SSL-related parameters aren't available as inputs. - + ### Connecting via an SSH Tunnel @@ -23,7 +23,7 @@ To connect to a Postgres, Redshift, or AlloyDB instance via an SSH tunnel, selec Once the connection is saved, a public key will be generated and displayed for the Connection. You can copy this public key to the bastion server to authorize dbt Cloud to connect to your database via the bastion server. - + #### About the Bastion server in AWS @@ -74,19 +74,19 @@ dbt Cloud establishes an SSH tunnel connection through the ELB or NLB to the - The SSH daemon terminates the session due to an idle timeout.
- - The ELB or NLB terminates the connection when it's idle.

+ - The ELB or NLB terminates the connection when it's idle.
- dbt Cloud sets a value for its SSH tunnel called `ServerAliveInterval` and `ServerAliveCountMax` that polls the connection every 30 seconds and the underlying OS in our run "pods" will terminate the connection if the `sshd` process fails to respond after 300s. This will, in many cases, prevent an idle timeout entirely so long as the customer is not using ELB with a firewall-level idle timeout of less than 30 seconds. However, if the customer is using ELB and is using an Idle Connection Timeout of less than 30s, this will be insufficient to prevent tunnels from being terminated.

+dbt Cloud sets a value for its SSH tunnel called `ServerAliveInterval` and `ServerAliveCountMax` that polls the connection every 30 seconds and the underlying OS in our run "pods" will terminate the connection if the `sshd` process fails to respond after 300s. This will, in many cases, prevent an idle timeout entirely so long as the customer is not using ELB with a firewall-level idle timeout of less than 30 seconds. However, if the customer is using ELB and is using an Idle Connection Timeout of less than 30s, this will be insufficient to prevent tunnels from being terminated.
- Some versions of Linux used on bastion hosts use a version of `sshd` with additional idle timeout settings `ClientAliveCountMax`. This value sets the number of client alive messages that may be sent without `sshd` receiving any messages back from the client. If this threshold is reached while client alive messages are being sent, `sshd` will disconnect the client, terminating the session. The client-alive mechanism is helpful when the client or server needs to know when a connection has become inactive. The default value is 3.

- `ClientAliveInterval`

- - This value sets a timeout interval in seconds after which if no data has been received from the client, `sshd` will send a message through the encrypted channel to request a response from the client. The default is 0, indicating that these messages will not be sent to the client.

+Some versions of Linux used on bastion hosts use a version of `sshd` with additional idle timeout settings `ClientAliveCountMax`. This value sets the number of client alive messages that may be sent without `sshd` receiving any messages back from the client. If this threshold is reached while client alive messages are being sent, `sshd` will disconnect the client, terminating the session. The client-alive mechanism is helpful when the client or server needs to know when a connection has become inactive. The default value is 3.

+ +`ClientAliveInterval`:
+This value sets a timeout interval in seconds after which if no data has been received from the client, `sshd` will send a message through the encrypted channel to request a response from the client. The default is 0, indicating that these messages will not be sent to the client.
-Using default values, tunnels could be terminated prematurely by `sshd`. To solve this problem, the `/etc/ssh/sshd_config` file on the bastion host can be configured with the following values:
-`ClientAliveCountMax` 10
-`ClientAliveInterval` 30
-where `ClientAliveCountMax` should be set to a non-zero value and `ClientAliveInterval` should be a value less than the ELB or NLB idle timeout value.

+Using default values, tunnels could be terminated prematurely by `sshd`. To solve this problem, the `/etc/ssh/sshd_config` file on the bastion host can be configured with the following values:

+- `ClientAliveCountMax` 10
+- `ClientAliveInterval` 30
+where `ClientAliveCountMax` should be set to a non-zero value and `ClientAliveInterval` should be a value less than the ELB or NLB idle timeout value.
With these settings, unresponsive SSH clients will be disconnected after approximately 300 seconds, helping to prevent tunnel failures.