Added mini variant

[rschmied]

This adds a AWS-mini variant. It's not nicely integrated with the existing code and requires a manual change in the deploy module to activate it. The current branch has it activated, though. But that will be removed eventually.

What it does:

• it takes a subnet ID and a security group ID in the configuration file
• it only creates an EC2 instance, adds a network interface to it which is attached to the given subnet with the given security ID
• it installs an all-in-one CML instance (no clustering)
• it currently does allocate an elastic IP for the interface. If this is not desireable then see the below text

Configuration

I've added a minimized config-aws-mini.yml which has just the minimum required attributes in the aws and common block.

Elastic IP

If no EIP is needed/wanted, then

• change public_ip to private_ip in output.tf
• delete the EIP resource block so that no EIP is allocated

In this case, the machine that runs Terraform / the provisioning must be able to reach the private IP address and the security group must permit HTTPS to the controller.

[rschmied]

This is the manual change I mentioned in the PR description. Setting the source here to aws-mini will take the HCL from that directory and not from the original one (which has all the cluster / VPC / subnet / ... creation bits).

[BobbyGR]

Reviewing.

[orndor]

I receive this error under terraform plan:

Error: Unsupported attribute

  on modules/deploy/aws-mini/main.tf line 50, in data "aws_subnet" "selected_subnet":
  50:   id = var.options.cfg.aws.subnet_id
    ├────────────────
    │ var.options.cfg.aws is object with 11 attributes

This object does not have an attribute named "subnet_id".
Error: Unsupported attribute

  on modules/deploy/aws-mini/main.tf line 54, in data "aws_security_group" "selected_security_group":
  54:   id = var.options.cfg.aws.sg_id
    ├────────────────
    │ var.options.cfg.aws is object with 11 attributes

This object does not have an attribute named "sg_id".

[rschmied]

you need to put those values into your config.yml:

aws:
  region: us-east-1
  bucket: your-bucket-here
  flavor: c5.2xlarge
  profile: your-profile-here
  enable_ebs_encryption: false
  subnet_id: "subnet-1234567890"
  sg_id: "sg-1234567890"

[orndor]

Perhaps you mean the config-aws-mini.yml (config.yml in the aws-mini variant doesn't have subnet_id nor sg_id defined)? Yes, I've done that.

[rschmied]

The config-aws-mini.yml configuration example is not used by default, it just serves as an example what to put into the actual configuration... You have two options:

• copy the content of config-aws-mini.yml into config.yml -or-
• define the export TF_VAR_cfg_file="config-aws-mini.yml" environment variable that specifies which configuration file to use.

[rschmied]

But re-reading your comment makes me think that you did add the attributes to the aws object in your config.yml. Not sure in this case. We might need to get on a call to look into this together. Otherwise this is going to be difficult.

[orndor]

I cloned the latest DEV branch, including your edits from last night, made the appropriate updates as per the AWS Mini instructions, and all is working now. Looks like for at least @BobbyGR and I, this is a good variant.

[CML-TAC]

@BobbyGR, May I know if you got it working or are you still stuck?

[BobbyGR]

it looks like it wants to delete the security group defined first?

module.deploy.module.aws[0].aws_security_group.sg_tf: Still destroying... [id=sg-xyz123, 15m0s elapsed]
╷
│ Error: deleting Security Group (sg-xyz123): DependencyViolation: resource sg-xyz123has a dependent object
│       status code: 400, request id: abc321

[rschmied]

it looks like it wants to delete the security group defined first?

module.deploy.module.aws[0].aws_security_group.sg_tf: Still destroying... [id=sg-xyz123, 15m0s elapsed]
╷
│ Error: deleting Security Group (sg-xyz123): DependencyViolation: resource sg-xyz123has a dependent object
│       status code: 400, request id: abc321

hard to say what's going on here without additional context.

[BobbyGR]

aws:
  region: us-west-2
  bucket: cml-broberts4-s3
  flavor: c5.2xlarge
  profile: s3-access-ec2
  enable_ebs_encryption: false
  subnet_id: "subnet-xyz"
  sg_id: "sg-abc

module.deploy.module.aws[0].aws_instance.cml_controller: Creating...
module.deploy.module.aws[0].aws_security_group.sg_tf: Still destroying... [id=sg-abc, 10s elapsed]
module.deploy.module.aws[0].aws_security_group.sg_tf: Still destroying... [id=sg-abc, 20s elapsed]
module.deploy.module.aws[0].aws_security_group.sg_tf: Still destroying... [id=sg-abc, 30s elapsed]
module.deploy.module.aws[0].aws_security_group.sg_tf: Still destroying... [id=sg-abc, 40s elapsed]
module.deploy.module.aws[0].aws_security_group.sg_tf: Still destroying... [id=sg-abc, 50s elapsed]
module.deploy.module.aws[0].aws_security_group.sg_tf: Still destroying... [id=sg-abc, 1m0s elapsed]

[rschmied]

@BobbyGR there's something wrong with your configuration. The aws-mini HCL does not create any security group or subnet -- they are both references / used as "data sources" (e.g. read-only, not managed). A data source is only read and thus will never be created or destroyed. So, either your state is messed up or, as stated, your configuration is somehow wrong.