Skip to content

Commit

Permalink
Merge branch 'main' of github.com:kubefirst/kubefirst
Browse files Browse the repository at this point in the history
  • Loading branch information
johndietz committed Jul 29, 2022
2 parents 5167be9 + c2f4432 commit ad06edc
Show file tree
Hide file tree
Showing 13 changed files with 205 additions and 144 deletions.
60 changes: 40 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,26 +7,45 @@
Kubefirst CLI is a cloud provisioning tool. With simple setup and few CLI calls, we spin up a full AWS cluster with full
GitOps integration, secrets management, production and development Kubernetes environments ready to be consumed.

- [Setup](#setup)
- [Environment Variables](#environment-variables)
- [DNS setup](#dns-setup)
- [Start the container](#start-the-container)
- [Initialization](#initialization)
- [Creation](#creation)
- [Access ArgoCD](#access-argocd)
- [Destroy](#destroy)
- [Available Commands]()
- [Available Commands](#available-commands)

![kubefirst architecture diagram](/images/kubefirst-arch.png)

## Setup
## Environment Variables

The setup is extremely simple, create a `.env` file in the root folder, and add the following variables:

| Variable | example |
|--------------------|------------------|
| AWS_PROFILE | default |
| CLOUD_PROVIDER=aws | aws |
| HOSTED_ZONE_NAME | example.com |
| ADMIN_EMAIL | [email protected] |
| Variable | example |
|-------------|--------------|
| AWS_PROFILE | default |
| AWS_REGION | eu-central-1 |

## DNS Setup

In order to install Kubefirst it's required to have a public domain. For root domains, setting the `--hosted-zone-name`
is enough, in case you want to use subdomains, and the domain is hosted on AWS, please follow the
[AWS documentation](https://aws.amazon.com/premiumsupport/knowledge-center/create-subdomain-route-53/).

Provisioned services on root domain will be hosted as:
```
argocd.example.com
gitlab.example.com
...
```

Provisioned services on subdomains will be hosted as:
```
argocd.subdomain.example.com
gitlab.subdomain.example.com
...
```

## Start the container

Expand All @@ -41,8 +60,12 @@ docker-compose up kubefirst-dev
Some process requires previous initialization, for that, run:

```bash
mkdir -p ~/.kubefirst
go run . init --admin-email $ADMIN_EMAIL --cloud $CLOUD_PROVIDER --hosted-zone-name $HOSTED_ZONE_NAME --region $AWS_REGION
go run . init \
--cloud aws \
--region eu-central-1 \
--admin-email [email protected] \
--cluster-name your_cluster_name \
--hosted-zone-name domain.example
```

## Creation
Expand All @@ -66,26 +89,23 @@ kubectl -n argocd port-forward svc/argocd-server 8080:80
It will destroy the kubefirst management cluster, and clean up every change made in the cloud.

```bash

go run . destroy
rm -rf ~/.kubefirst
rm ~/.flare
```

## Available Commands

Kubefirst provides extra tooling for handling the provisioning work.

| Command | Description |
|:------------|:-----------------------------------------------------------|
| Command | Description |
|:---------------|:----------------------------------------------------------|
| argocdSync | Request ArgoCD to synchronize applications |
| checktools | use to check compatibility of .kubefirst/tools |
| clean | removes all kubefirst resources locally for new execution |
| cluster create | create a kubefirst management cluster |
| destroy | destroy the kubefirst management cluster |
| info | provides general Kubefirst setup data |
| init | initialize your local machine to execute `create` |
| version | print the version number for kubefirst-cli" |
| destroy | destroy the kubefirst management cluster |
| info | provides general Kubefirst setup data |
| init | initialize your local machine to execute `create` |
| version | print the version number for kubefirst-cli" |

---
## The provisioning process
Expand Down
24 changes: 12 additions & 12 deletions build/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,15 +1,11 @@
FROM --platform=linux/amd64 golang:1.18

WORKDIR /opt/kubefirst
WORKDIR /opt/kubefirst-install

RUN apt-get update && \
apt-get install -y unzip curl jq vim unzip less \
apt-get install -y unzip curl jq vim unzip less \
&& rm -rf /var/lib/apt/lists/*


# enable terminal vi mode
RUN set -o vi

# Kubernetes client
RUN curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.21.3/bin/$(uname -s)/amd64/kubectl && \
chmod +x ./kubectl && \
Expand All @@ -18,23 +14,27 @@ RUN curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.21.3/b
# AWS cli
RUN curl -LO https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip && \
unzip awscli-exe-linux-x86_64.zip && \
./aws/install
./aws/install && \
rm -r aws && \
rm awscli-exe-linux-x86_64.zip

# AWS EKS cli
RUN curl -LO https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_linux_amd64.tar.gz && \
tar -xvzf eksctl_linux_amd64.tar.gz -C /usr/local/bin/
tar -xvzf eksctl_linux_amd64.tar.gz -C /usr/local/bin/ && \
rm eksctl_linux_amd64.tar.gz

# AWS IAM Authenticator tool
RUN curl -LO https://s3.us-west-2.amazonaws.com/amazon-eks/1.21.2/2021-07-05/bin/linux/amd64/aws-iam-authenticator && \
chmod +x aws-iam-authenticator && \
mv aws-iam-authenticator /usr/local/bin/

RUN go install github.com/spf13/cobra-cli@latest
RUN go install golang.org/x/tools/cmd/godoc@latest
RUN go install golang.org/x/lint/golint@latest


# setup user
RUN useradd -ms /bin/bash developer
USER developer
WORKDIR /home/developer/kubefirst

COPY --chown=developer:developer . .

# download dependencies and prepare fresh installation
RUN go mod download && go run . clean
31 changes: 2 additions & 29 deletions cmd/create.go
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ to quickly create a Cobra application.`,
progressPrinter.IncrementTracker("step-argo", 1)

informUser("Getting an argocd auth token")
token := argocd.GetArgocdAuthToken(dryRun)

progressPrinter.IncrementTracker("step-argo", 1)
if !dryRun {
_, _, err = pkg.ExecShellReturnStrings(config.KubectlClientPath, "--kubeconfig", config.KubeConfigPath, "-n", "argocd", "apply", "-f", fmt.Sprintf("%s/gitops/components/helpers/registry.yaml", config.K1FolderPath))
Expand All @@ -164,34 +164,7 @@ to quickly create a Cobra application.`,
}
time.Sleep(45 * time.Second)
}

informUser("Syncing the registry application")

if dryRun {
log.Printf("[#99] Dry-run mode, Sync ArgoCD skipped")
} else {
// todo: create ArgoCD struct, and host dependencies (like http client)
customTransport := http.DefaultTransport.(*http.Transport).Clone()
customTransport.TLSClientConfig = &tls.Config{InsecureSkipVerify: true}
httpClient := http.Client{Transport: customTransport}

// retry to sync ArgoCD application until reaches the maximum attempts
argoCDIsReady, err := argocd.SyncRetry(&httpClient, 60, 5, "registry", token)
if err != nil {
log.Printf("something went wrong during ArgoCD sync step, error is: %v", err)
}

if !argoCDIsReady {
log.Println("unable to sync ArgoCD application, continuing...")
}
}

progressPrinter.IncrementTracker("step-argo", 1)
// todo, need to stall until the registry has synced, then get to ui asap

//! skip this if syncing from argocd and not helm installing
// log.Printf("sleeping for 30 seconds, hurry up jared sign into argocd %s", viper.GetString("argocd.admin.password"))
// time.Sleep(30 * time.Second)

//!
//* we need to stop here and wait for the vault namespace to exist and the vault pod to be ready
Expand Down Expand Up @@ -368,7 +341,7 @@ to quickly create a Cobra application.`,
httpClient := http.Client{Transport: customTransport}

// retry to sync ArgoCD application until reaches the maximum attempts
argoCDIsReady, err := argocd.SyncRetry(&httpClient, 60, 5, "registry", token)
argoCDIsReady, err := argocd.SyncRetry(&httpClient, 120, 5, "registry", token)
if err != nil {
log.Printf("something went wrong during ArgoCD sync step, error is: %v", err)
}
Expand Down
81 changes: 81 additions & 0 deletions cmd/k1-state.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
package cmd

import (
"fmt"
"github.com/kubefirst/kubefirst/configs"
"github.com/kubefirst/kubefirst/internal/aws"
"github.com/kubefirst/kubefirst/internal/reports"
"github.com/spf13/cobra"
"log"
"os"
)

var k1state = &cobra.Command{
Use: "state",
Short: "push and pull Kubefirst configuration to S3 bucket",
Long: `Kubefirst configuration can be handed over to another user by pushing the Kubefirst config files to a S3 bucket.`,
Run: func(cmd *cobra.Command, args []string) {

push, err := cmd.Flags().GetBool("push")
if err != nil {
log.Println(err)
}
pull, err := cmd.Flags().GetBool("pull")
if err != nil {
log.Println(err)
}

bucketName, err := cmd.Flags().GetString("bucket-name")
if err != nil {
log.Println(err)
}

if !push && !pull {
fmt.Println(cmd.Help())
return
}

config := configs.ReadConfig()
if push {
err = aws.UploadFile(bucketName, config.KubefirstConfigFileName, config.KubefirstConfigFilePath)
if err != nil {
fmt.Println(err)
return
}
finalMsg := fmt.Sprintf("Kubefirst configuration file was upload to AWS S3 at %q bucket name", bucketName)

log.Printf(finalMsg)
fmt.Println(reports.StyleMessage(finalMsg))
}

if pull {
err := aws.DownloadS3File(bucketName, config.KubefirstConfigFileName)
if err != nil {
fmt.Println(err)
return
}
currentFolder, err := os.Getwd()
finalMsg := fmt.Sprintf("Kubefirst configuration file was downloaded to %q/, and is now available to be copied to %q/",
currentFolder,
config.K1FolderPath,
)

log.Printf(finalMsg)
fmt.Println(reports.StyleMessage(finalMsg))
}
},
}

func init() {
rootCmd.AddCommand(k1state)

k1state.Flags().Bool("push", false, "push Kubefirst config file to the S3 bucket")
k1state.Flags().Bool("pull", false, "pull Kubefirst config file to the S3 bucket")
k1state.Flags().String("bucket-name", "", "set the bucket name to store the Kubefirst config file")
err := k1state.MarkFlagRequired("bucket-name")
if err != nil {
log.Println(err)
return
}

}
2 changes: 2 additions & 0 deletions configs/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ type Config struct {
InstallerEmail string

KubefirstLogPath string `env:"KUBEFIRST_LOG_PATH" envDefault:"logs"`
KubefirstConfigFileName string
KubefirstConfigFilePath string
K1FolderPath string
KubectlClientPath string
Expand Down Expand Up @@ -58,6 +59,7 @@ func ReadConfig() *Config {
log.Panic(err)
}

config.KubefirstConfigFileName = ".kubefirst"
config.KubefirstConfigFilePath = fmt.Sprintf("%s/.kubefirst", homePath)

config.LocalOs = runtime.GOOS
Expand Down
9 changes: 7 additions & 2 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,19 @@ services:
kubefirst-dev:
platform: linux/amd64
build:
context: ./build
context: .
dockerfile: ./build/Dockerfile
container_name: kubefirst-dev
env_file:
- .env
environment:
TERM: xterm-256color
ports:
- "8080:8080" # ArgoCD
- "8888:8888" # GitLab
- "8200:8200" # Vault
volumes:
- ./:/home/developer/kubefirst
- ./:/opt/kubefirst
# AWS credentials are strictly used to provision the Kubefirst in your AWS account
- $HOME/.aws:/home/developer/.aws
command: sh -c "./scripts/kubefirst-dev.sh"
4 changes: 2 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ require (
github.com/acomagu/bufpipe v1.0.3 // indirect
github.com/armon/go-metrics v0.3.10 // indirect
github.com/armon/go-radix v1.0.0 // indirect
github.com/aws/aws-sdk-go-v2 v1.16.7 // indirect
github.com/aws/aws-sdk-go-v2 v1.16.7
github.com/aws/aws-sdk-go-v2/credentials v1.12.2 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.5 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.14 // indirect
Expand Down Expand Up @@ -141,7 +141,7 @@ require (
gopkg.in/ini.v1 v1.66.4 // indirect
gopkg.in/square/go-jose.v2 v2.5.1 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v2 v2.4.0
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/klog/v2 v2.60.1 // indirect
k8s.io/utils v0.0.0-20220210201930-3a6ce19ff2f9 // indirect
Expand Down
9 changes: 0 additions & 9 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -96,8 +96,6 @@ github.com/atotto/clipboard v0.1.4/go.mod h1:ZY9tmq7sm5xIbd9bOK4onWV4S6X0u6GY7Vn
github.com/aws/aws-sdk-go v1.44.23 h1:oFvpKJk5qdprnCcuCWk2/CADdvfYtyduQ392bMXjlYI=
github.com/aws/aws-sdk-go v1.44.23/go.mod h1:y4AeaBuwd2Lk+GepC1E9v0qOiTws0MIWAX4oIKwKHZo=
github.com/aws/aws-sdk-go-v2 v1.16.4/go.mod h1:ytwTPBG6fXTZLxxeeCCWj2/EMYp/xDUgX+OET6TLNNU=
github.com/aws/aws-sdk-go-v2 v1.16.5 h1:Ah9h1TZD9E2S1LzHpViBO3Jz9FPL5+rmflmb8hXirtI=
github.com/aws/aws-sdk-go-v2 v1.16.5/go.mod h1:Wh7MEsmEApyL5hrWzpDkba4gwAPc5/piwLVLFnCxp48=
github.com/aws/aws-sdk-go-v2 v1.16.7 h1:zfBwXus3u14OszRxGcqCDS4MfMCv10e8SMJ2r8Xm0Ns=
github.com/aws/aws-sdk-go-v2 v1.16.7/go.mod h1:6CpKuLXg2w7If3ABZCl/qZ6rEgwtjZTn4eAf4RcEyuw=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.3 h1:S/ZBwevQkr7gv5YxONYpGQxlMFFYSRfz3RMcjsC9Qhk=
Expand All @@ -109,13 +107,9 @@ github.com/aws/aws-sdk-go-v2/credentials v1.12.2/go.mod h1:/XWqDVuzclEKvzileqtD7
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.5 h1:YPxclBeE07HsLQE8vtjC8T2emcTjM9nzqsnDi2fv5UM=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.5/go.mod h1:WAPnuhG5IQ/i6DETFl5NmX3kKqCzw7aau9NHAGcm4QE=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.11/go.mod h1:tmUB6jakq5DFNcXsXOA/ZQ7/C8VnSKYkx58OI7Fh79g=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.12 h1:Zt7DDk5V7SyQULUUwIKzsROtVzp/kVvcz15uQx/Tkow=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.12/go.mod h1:Afj/U8svX6sJ77Q+FPWMzabJ9QjbwP32YlopgKALUpg=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.14 h1:2C0pYHcUBmdzPj+EKNC4qj97oK6yjrUhc1KoSodglvk=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.14/go.mod h1:kdjrMwHwrC3+FsKhNcCMJ7tUVj/8uSD5CZXeQ4wV6fM=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.5/go.mod h1:fV1AaS2gFc1tM0RCb015FJ0pvWVUfJZANzjwoO4YakM=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.6 h1:eeXdGVtXEe+2Jc49+/vAzna3FAQnUD4AagAw8tzbmfc=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.6/go.mod h1:FwpAKI+FBPIELJIdmQzlLtRe8LQSOreMcM2wBsPMvvc=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.8 h1:2J+jdlBJWEmTyAwC82Ym68xCykIvnSnIN18b8xHGlcc=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.8/go.mod h1:ZIV8GYoC6WLBW5KGs+o4rsc65/ozd+eQ0L31XF5VDwk=
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.12 h1:j0VqrjtgsY1Bx27tD0ysay36/K4kFMWRp9K3ieO9nLU=
Expand All @@ -126,7 +120,6 @@ github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.9.3 h1:4n4KCtv5
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.9.3/go.mod h1:gkb2qADY+OHaGLKNTYxMaQNacfeyQpZ4csDTQMeFmcw=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.1.9 h1:gVv2vXOMqJeR4ZHHV32K7LElIJIIzyw/RU1b0lSfWTQ=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.1.9/go.mod h1:EF5RLnD9l0xvEWwMRcktIS/dI6lF8lU5eV3B13k6sWo=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.5 h1:gRW1ZisKc93EWEORNJRvy/ZydF3o6xLSveJHdi1Oa0U=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.5/go.mod h1:ZbkttHXaVn3bBo/wpJbQGiiIWR90eTBUVBrEHUEQlho=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.8 h1:oKnAXxSF2FUvfgw8uzU/v9OTYorJJZ8eBmWhr9TWVVQ=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.8/go.mod h1:rDVhIMAX9N2r8nWxDUlbubvvaFMnfsm+3jAV7q+rpM4=
Expand All @@ -141,8 +134,6 @@ github.com/aws/aws-sdk-go-v2/service/sso v1.11.5/go.mod h1:TFVe6Rr2joVLsYQ1ABACX
github.com/aws/aws-sdk-go-v2/service/sts v1.16.6 h1:aYToU0/iazkMY67/BYLt3r6/LT/mUtarLAF5mGof1Kg=
github.com/aws/aws-sdk-go-v2/service/sts v1.16.6/go.mod h1:rP1rEOKAGZoXp4iGDxSXFvODAtXpm34Egf0lL0eshaQ=
github.com/aws/smithy-go v1.11.2/go.mod h1:3xHYmszWVx2c0kIwQeEVf9uSm4fYZt67FBJnwub1bgM=
github.com/aws/smithy-go v1.11.3 h1:DQixirEFM9IaKxX1olZ3ke3nvxRS2xMDteKIDWxozW8=
github.com/aws/smithy-go v1.11.3/go.mod h1:Tg+OJXh4MB2R/uN61Ko2f6hTZwB/ZYGOtib8J3gBHzA=
github.com/aws/smithy-go v1.12.0 h1:gXpeZel/jPoWQ7OEmLIgCUnhkFftqNfwWUwAHSlp1v0=
github.com/aws/smithy-go v1.12.0/go.mod h1:Tg+OJXh4MB2R/uN61Ko2f6hTZwB/ZYGOtib8J3gBHzA=
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
Expand Down
Loading

0 comments on commit ad06edc

Please sign in to comment.