README.md.handlebars

# CloudControl ☁️ 🧰

The cloud engineer's toolbox.

## Introduction

*CloudControl* is a [Docker](https://docker.com) based configuration environment containing all the tools
required and configured to manage modern cloud infrastructures.

The toolbox comes in different "flavours" depending on what cloud you are working in.
Currently supported cloud flavours are:

{{#each docObjects.flavour}}
* {{ this.title }}
{{/each}}

Following features and tools are supported:

{{#each docObjects.feature}}
* {{ this.title }}
{{/each}}

## Table of contents

* [Usage](#usage)
* [Using Kubernetes](#using-kubernetes)
* [FAQ](#faq)
* [Flavours](#flavours)
{{#each docObjects.flavour}}
    * [{{ @key }}](#{{ @key }})
{{/each}}
* [Features](#features)
{{#each docObjects.feature}}
    * [{{ @key }}](#{{ @key }})
{{/each}}
* [Development](#development)
* [Building](#building)

## Usage

*CloudControl* can be used best with docker-compose. Check out the `sample` directory in a flavour for a sample
compose file and to convenience scripts. It includes a small web server written in Go and Vuejs-client dubbed
"CloudControlCenter", which is used as a status screen. It listens to port 8080 inside the container.

Copy the compose file and configure it to your needs. Check below for configuration options per flavour and feature.

Run `init.sh`. This script basically just runs `docker-compose up -d` and tells you the URL for CloudControlCenter.
Open it and wait for CloudControl to finish initializing.

The initialization process will download and configure the additional tools and completes with a message when its done.
It will run each time when the stack is recreated.

After the initialization process you can simply run `docker-compose exec cli /usr/local/bin/cloudcontrol run` to jump
into the running container and work with the installed features.

If you need to change any of the configuration environment variables, rerun the init script afterwards to apply
the changes. Remember, that CloudControl needs to reininitialize for this.

## Using Kubernetes

*CloudControl* is targeted to run on a local machine. It requires the following features to work:

* host path volumes
* host based networking

Some Kubernetes distributions such as [Rancher desktop](https://rancherdesktop.io/) support this and can be used to
run *CloudControl*.

The `sample` directories of each flavour provide an example Kubernetes configuration based on a deployment and a
service. They were preliminary tested on Rancher desktop.

Modify them to your local requirements and then run

    kubectl apply -f k8s.yaml

to apply them.

This will create a new namespace for your project and a deployment and a service in that. Check
`kubectl get -n <project> pod` to watch the progress until a cli pod has been created.

Use `kubectl get -n <project> svc cli` to see the bound ports for the cli service and use your browser to connect
to the *CloudControlCenter* instance.

After the initialization is done, use `kubectl -n <project> exec -it deployment/cli -- /usr/local/bin/cloudcontrol run`
to enter *CloudControl*.

*Warning*: This implementation is currently a preview feature and hasn't been tested thoroughly. It highly depends on
the proper support for host based volumes and networking of the Kubernetes distribution. Please refer to the
documentation and support of your Kubernetes distribution if something isn't working.

## FAQ

### How can I add an informational text for users of CloudControl?

If you want to display a *custom login message* when users enter the container, set environment variable `MOTD`
to that message. If you want to display the default login message as well, also
set the environment variable `MOTD_DISPLAY_DEFAULT` to *yes*.

### How can I forward ports to the host?

If you'd like to forward traffic into a cluster using `kubectl port-forward` you can do the following:

* Add a ports key to the cli-service in your docker-compose file to forward a free port on your host to a defined
port in your container. The docker-compose-files in the sample directories already use port 8081 for this.

* Inside *CloudControl*, check the IP of the container:

```
bash-5.0$ ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 02:42:AC:15:00:02
          inet addr:172.21.0.2  Bcast:172.21.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:53813 errors:0 dropped:0 overruns:0 frame:0
          TX packets:20900 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:75260363 (71.7 MiB)  TX bytes:2691219 (2.5 MiB)
```

* Use the IP address used by the container as the bind address for port-forward to forward traffic
to the previously defined container port to a service on its port (e.g. port 8081 to the
service my-service listening on port 8080):

```
kubectl port-forward --address 172.21.0.2 svc/my-service 8081:8080
```

* Check out, which host port docker bound to the private port you set up (e.g. 8081)

```
docker-compose port cli 8081
```

* Connect to localhost:<host port> on your host

### How to set up command aliases

If you'd like to set up aliases to save some typing, you can use the *run* feature. Run your container with these
environment variables:

* `USE_run=yes`: Set up the run feature
* `RUN_COMMANDS=alias firstalias=command;alias secondalias=command`: Set up some aliases

### How can I share my SSH-keys with the CloudControl container

First, mount your .ssh directory into the container at /home/cloudcontrol/.ssh.

Also, to not enter your passphrase every time you use the key, you should mount the ssh agent socket into the
container and set the environment variable SSH_AUTH_SOCK to that path. CloudControl will automatically fix the
permissions of that file so the CloudControl user can use it.

Here are snippets for your docker-compose file for convenience:

    (...)
    volumes:
        - "<Path to .ssh directory>:/home/cloudcontrol/.ssh"
        # for Linux:
        - "${SSH_AUTH_SOCK}:/ssh-agent"
        # for macOS:
        - "/run/host-services/ssh-auth.sock:/ssh-agent"
    environment:
        - "SSH_AUTH_SOCK=/ssh-agent"

### How to identify Terraform state locks by other CloudControl-users

Because of how CloudControl is designed it uses a defined user named "cloudcontrol", so Terraform state lock
messages look like this:

> Error: Error locking state: Error acquiring the state lock: storage: service returned error: StatusCode=409, ErrorCode=LeaseAlreadyPresent, ErrorMessage=There is already a lease present.
  RequestId:56c21b95-501e-0096-7082-41fa0d000000
  Time:2021-05-05T07:41:25.9164547Z, RequestInitiated=Wed, 05 May 2021 07:41:25 GMT, RequestId=56c21b95-501e-0096-7082-41fa0d000000, API Version=2018-03-28, QueryParameterName=, QueryParameterValue=
  Lock Info:
  ID:        a1cef2cc-fec4-1765-4da8-d068a729ba7e
  Path:      path/terraform.tfstate
  Operation: OperationTypeApply
  Who:       cloudcontrol@5c47a37f920b
  Version:   0.12.17
  Created:   2021-05-05 07:38:01.188897776 +0000 UTC
  Info:

It's hard to identify from that who the other CloudControl user is, that may have opened a lock. The system
user can't be changed, but it's possible to set a better hostname than the one Docker autogenerated.

See this docker-compose snippet on how to set a better hostname:

    version: "3"
    services:
    cli:
    image: "dodevops/cloudcontrol-azure:latest"
    hostname: "<TODO yourname>"
    volumes:
    (...)

If you set hostname in that snippet to "alice", the state lock will look like this now:

> Error: Error locking state: Error acquiring the state lock: storage: service returned error: StatusCode=409, ErrorCode=LeaseAlreadyPresent, ErrorMessage=There is already a lease present.
  RequestId:56c21b95-501e-0096-7082-41fa0d000000
  Time:2021-05-05T07:41:25.9164547Z, RequestInitiated=Wed, 05 May 2021 07:41:25 GMT, RequestId=56c21b95-501e-0096-7082-41fa0d000000, API Version=2018-03-28, QueryParameterName=, QueryParameterValue=
  Lock Info:
  ID:        a1cef2cc-fec4-1765-4da8-d068a729ba7e
  Path:      path/terraform.tfstate
  Operation: OperationTypeApply
  Who:       cloudcontrol@alice
  Version:   0.12.17
  Created:   2021-05-05 07:38:01.188897776 +0000 UTC
  Info:

## Flavours

{{#each docObjects.flavour}}
### {{ @key }}

{{ this.description }}

#### Configuration

{{#each this.configuration}}
* {{ this }}
{{/each}}
{{/each}}

## Features

{{#each docObjects.feature}}
### {{ @key }}

{{ this.description }}

#### Configuration

* USE_{{ @key }}: Enable this feature
* DEBUG_{{ @key }}: Debug this feature
{{#each this.configuration }}
* {{ this }}
{{/each}}

{{/each}}

## Development

*CloudControl* supports a decoupled development of features and flavours. If you're missing something, just fork this
repository, create a subfolder for your new feature under "features" and add these files:

* feature.yaml: A descriptor for your feature with a title, a description and configuration notes
* install.sh: A shell script that is run by CloudControlCenter and should install everything you need
  for your new feature
* motd.sh: (optional) If you want to show some information to the users upon login, put them here.

If you need another flavour (aka cloud provider), add a new subdirectory under "flavour" and add a flavour.yaml describing
your flavour the same way as a feature. For the rest of the files, please check out existing flavours for details. Please,
include a sample configuration for your flavour to make it easier for other people to work with it.

### Installer utilities

In your install script, you can source the utils library

    . /feature-installer-utils.sh

#### execHandle

Installation scripts usually echo out some kind of progress, execute something and have to check for errors. The command
`execHandle` does all this in a one liner:

    execHandle "Progress message" command

This will print out "Progress message...", run the command and if it exits with a non-zero status code, it will print
the output of the command and exit with status code 1.

Using this makes installer script way shorter and easier to maintain.

## Building

Build a flavor container image with the base of the repository as the build context like this:

    build.sh <tag> <flavour>

To build all flavours with the same tag, use

    build.sh <tag>