NOTE: This module has been adopted by the official Snowplow DevOps organization and is now archived. Please use the official module.
A Terraform module which deploys the Snowplow BigQuery Loader, Mutator and Repeater apps on Google Cloud Compute Engine. If you want to use a custom image for this deployment you will need to ensure it is based on top of Ubuntu 20.04.
Name | Version |
---|---|
terraform | >= 0.15 |
>= 3.50.0 |
Name | Version |
---|---|
>= 3.50.0 | |
local | n/a |
Name | Source | Version |
---|---|---|
telemetry | snowplow-devops/telemetry/snowplow | 0.2.0 |
Name | Description | Type | Default | Required |
---|---|---|---|---|
associate_public_ip_address | Whether to assign a public ip address to this instance; if false this instance must be behind a Cloud NAT to connect to the internet | bool |
true |
no |
dataset_config | The dataset in which to load the Snowplow events. Created by default. | object({ |
{ |
no |
enriched_topic_id | The pubsub topic to read enriched messages from. | string |
n/a | yes |
gcp_logs_enabled | Whether application logs should be reported to GCP Logging | bool |
true |
no |
images | The docker images with version tag to deploy on Compute Engine's instances. See here for details: https://docs.snowplowanalytics.com/docs/pipeline-components-and-applications/loaders-storage-targets/bigquery-loader/ The default is to launch all three apps: 'Stream Loader', 'Mutator' and 'Repeater'. |
list(string) |
[ |
no |
labels | The labels to append to this resource | map(string) |
{} |
no |
machine_type | The machine type to use | string |
"e2-micro" |
no |
name | Will be prefixed to all resource names. Use to easily identify the resources created. | string |
"loader" |
no |
network | The name of the network to deploy within. | string |
n/a | yes |
project_id | The GCP project ID. | string |
n/a | yes |
region | The name of the region to deploy within. | string |
n/a | yes |
ssh_block_project_keys | Whether to block project wide SSH keys | bool |
true |
no |
ssh_ip_allowlist | The list of CIDR ranges to allow SSH traffic from | list(any) |
[ |
no |
ssh_key_pairs | The list of SSH key-pairs to add to the servers | list(object({ |
[] |
no |
subnetwork | The name of the sub-network to deploy within; if populated will override the 'network' setting. | string |
"" |
no |
table_config | The table in which to load the Snowplow events. Created by default. | object({ |
{ |
no |
tags | The tags to apply to the created resources. | list(string) |
[] |
no |
telemetry_enabled | Whether or not to send telemetry information back to Snowplow Analytics Ltd | bool |
true |
no |
ubuntu_20_04_source_image | The source image to use which must be based of of Ubuntu 20.04; by default the latest community version is used | string |
"" |
no |
user_provided_id | An optional unique identifier to identify the telemetry events emitted by this stack | string |
"" |
no |
zone | The zone in which to deploy the instances. | any |
n/a | yes |
Name | Description |
---|---|
bad_types_topic_name | n/a |
bq_loader_apps | The compute instances created. |
created_or_provided_dataset_id | n/a |
created_or_provided_table_id | n/a |
enriched_topic_name | n/a |
failed_inserts_sub_name | n/a |
failed_inserts_topic_name | n/a |
input_sub_name | n/a |
types_sub_name | n/a |
types_topic_name | n/a |
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this software except in compliance with the License.
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.