Skip to content

Latest commit

 

History

History
180 lines (141 loc) · 17 KB

README.md

File metadata and controls

180 lines (141 loc) · 17 KB
Raw

Weights & Biases AWS Module

IMPORTANT: You are viewing a beta version of the official module to install Weights & Biases. This new version is incompatible with earlier versions, and it is not currently meant for production use. Please contact your Customer Success Manager for details before using.

This is a Terraform module for provisioning a Weights & Biases Cluster on AWS. Weights & Biases Local is our self-hosted distribution of wandb.ai. It offers enterprises a private instance of the Weights & Biases application, with no resource limits and with additional enterprise-grade architectural features like audit logging and SAML single sign-on.

About This Module

Pre-requisites

This module is intended to run in an AWS account with minimal preparation, however it does have the following pre-requisites:

Terrafom version >= 1

Credentials / Permissions

AWS Services Used

  • AWS Identity & Access Management (IAM)
  • AWS Key Management System (KMS)
  • Amazon Aurora MySQL
  • Amazon VPC
  • Amazon S3
  • Amazon Route53
  • Amazon Certificate Manager (ACM)
  • Amazon Elastic Loadbalancing (ALB)
  • Amazon Secrets Manager

Public Hosted Zone

If you are managing DNS via AWS Route53 the hosted zone entry is created automatically as part of your domain management.

If you're managing DNS outside of Route53, you will need to:

  1. Create a Route53 zone name {subdomain}.{domain} (e.g test.wandb.ai)
  2. Create a NS record in your parent system and point it to the newly created Route53
  3. Enable the external_dns option in this module

You can learn more about creating a hosted zone for a subdomain, which you will need to do for the subdomain you are planning to use for your Weights & Biases installation. To create this hosted zone with Terraform, use the aws_route53_zone resource.

ACM Certificate

While this is not required, it is recommend to already have an existing ACM certification. Certificate validation can take up two hours, causing timeouts during module apply if the cert is generated as one of the resources contained in the module.

How to Use This Module

  • Ensure account meets module pre-requisites from above.

  • Please note that while some resources are individually and uniquely tagged, all common tags are expected to be configured within the AWS provider as shown in the example code snippet below.

  • Create a Terraform configuration that pulls in this module and specifies values of the required variables:

provider "aws" {
  region = "<your AWS region>"
  default_tags {
    tags = var.common_tags
  }
}

module "wandb" {
  source    = "<filepath to cloned module directory>"
  namespace = "<prefix for naming AWS resources>"
}
  • Run terraform init and terraform apply

Examples

We have included documentation and reference examples for additional common installation scenarios for Weights & Biases, as well as examples for supporting resources that lack official modules.

Requirements

Name Version
terraform ~> 1.0
aws ~> 3.60
kubernetes ~> 2.6

Providers

Name Version
aws 3.61.0

Modules

Name Source Version
acm terraform-aws-modules/acm/aws ~> 3.0
app_eks ./modules/app_eks n/a
app_lb ./modules/app_lb n/a
database ./modules/database n/a
file_storage ./modules/file_storage n/a
kms ./modules/kms n/a
networking ./modules/networking n/a
redis ./modules/redis n/a

Resources

Name Type

Inputs

Name Description Type Default Required
acm_certificate_arn The ARN of an existing ACM certificate. string null no
allowed_inbound_cidr Allow HTTP(S) traffic to W&B. Defaults to no connections. list(string) [] no
allowed_inbound_ipv6_cidr Allow HTTP(S) traffic to W&B. Defaults to no connections. list(string) [] no
create_elasticache Boolean indicating whether to create an elasticache instance (true) or not (false). bool false no
create_vpc Boolean indicating whether to deploy a VPC (true) or not (false). bool true no
deletion_protection If the instance should have deletion protection enabled. The database / S3 can't be deleted when this value is set to true. bool true no
domain_name Domain for accessing the Weights & Biases UI. string n/a yes
external_dns Using external DNS. A subdomain must also be specified if this value is true. bool false no
kms_key_alias KMS key alias for AWS KMS Customer managed key. string null no
kms_key_deletion_window Duration in days to destroy the key after it is deleted. Must be between 7 and 30 days. number 7 no
kubernetes_public_access Indicates whether or not the Amazon EKS public API server endpoint is enabled. bool false no
kubernetes_public_access_cidrs List of CIDR blocks which can access the Amazon EKS public API server endpoint. list(string) [] no
namespace String used for prefix resources. string n/a yes
network_cidr CIDR block for VPC. string "10.10.0.0/16" no
network_database_subnet_cidrs List of private subnet CIDR ranges to create in VPC. list(string) 
[
 "10.10.20.0/24",
 "10.10.21.0/24"
]
 no
network_database_subnets A list of the identities of the database subnetworks in which resources will be deployed. list(string) [] no
network_id The identity of the VPC in which resources will be deployed. string "" no
network_private_subnet_cidrs List of private subnet CIDR ranges to create in VPC. list(string) 
[
 "10.10.10.0/24",
 "10.10.11.0/24"
]
 no
network_private_subnets A list of the identities of the private subnetworks in which resources will be deployed. list(string) [] no
network_public_subnet_cidrs List of private subnet CIDR ranges to create in VPC. list(string) 
[
 "10.10.0.0/24",
 "10.10.1.0/24"
]
 no
network_public_subnets A list of the identities of the public subnetworks in which resources will be deployed. list(string) [] no
public_access Is this instance accessable a public domain. bool false no
ssl_policy SSL policy to use on ALB listener string "ELBSecurityPolicy-FS-1-2-Res-2020-10" no
subdomain Subdomain for accessing the Weights & Biases UI. Default creates record at Route53 Route. string null no
zone_id Domain for creating the Weights & Biases subdomain on. string n/a yes

Outputs

Name Description
bucket_name n/a
bucket_queue_name n/a
bucket_region n/a
cluster_id n/a
database_connection_string n/a
internal_app_port n/a
kms_key_arn The Amazon Resource Name of the KMS key used to encrypt data at rest.
network_id The identity of the VPC in which resources are deployed.
network_private_subnets The identities of the private subnetworks deployed within the VPC.
network_public_subnets The identities of the public subnetworks deployed within the VPC.
url The URL to the W&B application