adding hashiqube resources for HashiTalks2020

.gitignore

@@ -0,0 +1,28 @@
+**.vagrant
+*.log
+**.DS_Store
+*.zip
+**.terraform
+**.tfstate
+**.backup
+**.rli
+SUMMARY.md
+sonarqube/logs**
+sonarqube/data/es6**
+sonarqube/data/sonar.lock.db
+jenkins/jenkins_home/jobs/*/builds*
+jenkins/jenkins_home/workspace/*/.git*
+jenkins/jenkins_home/logs*
+jenkins/jenkins_home/updates/*
+jenkins/jenkins_home/.java/fonts/*
+jenkins/jenkins_home/queue.xml
+jenkins/jenkins_home/war**
+jenkins/jenkins_home/nodes**
+jenkins/jenkins_home/jobs**
+jenkins/jenkins_home/secrets**
+jenkins/jenkins_home/.owner
+**packer_cache*
+hashicorp/packer/windows/output**
+hashicorp/packer/windows/ansible/connection_plugins/__pycache__**
+**.box
+**.iso

README.md

@@ -1,15 +1,96 @@
 # HashiQube Overview
 HashiQube is a VM with a Docker daemon inside. It runs all HashiCorp products. __Vault, Terraform, Nomad, Consul, Vagrant, Packer and Sentinel.__
-
+It also runs a host of other popular Open Source DevOps / DevSecOps applications showcasing how simple integration with HashiCorp products can result in tangible learnings and benefits for all its users
+Once the Qube is up an internet connection is no longer needed meaning sales pitches and demos for potential and existing customers is greatly aided.
 
 ## HashiQube runs all HashiCorp's products
 ![HashiQube](images/thestack.png?raw=true "HashiQube")
 
+## Purpose
+HashiQube has been created to enable anyone who is interested in secure automation pipelines the ability to run a suite of ‘best in class’ tools their local machines at the cost of a small amount of system resources.
+The Qube gives all interested parties the empowerment  to  deploy these tools in a way covers multiple use cases effectively providing a ‘concept to completion’ test bed using open source HashiCorp products.
+The original use case was born the desire to demystify DevSecOps utilising Terraform, Vault, Consul, Sentinel and Nomad as well as some other well know open source CI/CD tools by providing a ‘hands-on’ environment that demonstrates the value of secret and credential management in standard software development pipeline.
+
+Thanks to the flexibility of the HashiCorp products there is no need wonder how to achieve the goals of bringing software to market in a more secure and timely fashion, just Vagrant up!
+
+## Instructions
+* Please download __Virtualbox__ from https://www.virtualbox.org/wiki/Downloads and __Vagrant__ from https://www.vagrantup.com/downloads.html and install
+* Using `git` - clone this repo `git clone $repo .` [__What is Git?__](git/#git)
+* Inside the local repo folder, do `vagrant up --provision` - This will setup, Vault, Nomad, Consul, Terraform, Localstack and Docker as well as giving you access the docsify website at http://localhost:3333
+* To run a specific service you want to use run the declarative command for it, for example, `vagrant up --provision-with minikube`
+* Open in your browser http://localhost:3333 for Documentation
+
+## Pre-requisites
+* 10GB of disk space
+* 4GB RAM
+* Admin rights / sudo (you will be asked to update ETC Host file)
+* Virtualbox
+* Vagrant
+* `vagrant up --provision`
+
+## Additional Information
+This repository is designed to provide you with a stack that demonstrates the power of HashiCorp's product suite with non-enterprise editions of the following software;
+* [__Vagrant__](hashicorp/#vagrant) - Development Environments Made Easy
+* [__Vault__](hashicorp/#vault) - Manage Secrets and Protect Sensitive Data
+* [__Consul__](hashicorp/#consul) - Secure Service Networking
+* [__Nomad__](hashicorp/#nomad) - Deploy and Manage Any Containerized, Legacy, or Batch Application
+* [__Terraform__](hashicorp/#terraform) - Use Infrastructure as Code to provision and manage any cloud, infrastructure, or service
+* [__Packer__](hashicorp/#packer) - Build Automated Machine Images
+* [__Sentinel__](hashicorp/#sentinel) - Sentinel is an embedded policy-as-code framework
+* [__Fabio__](hashicorp/#fabio-load-balancer) - Fabio is an HTTP and TCP reverse proxy that configures itself with data from Consul
+* [__Docker__](docker/#docker) - Securely build, share and run any application, anywhere
+* [__Localstack__](localstack/#localstack) - A fully functional local AWS cloud stack
+* [__LDAP__](ldap/#ldap) - Lightweight Directory Access Protocol
+* [__Jenkins__](jenkins/#jenkins) - The leading open source automation server
+* [__Oracle MySQL__](database/#oracle-mysql) - MySQL is an open-source relational database management system (RDBMS)
+* [__Microsoft MSSQL__](database/#microsoft-sql-mssql-express) - Microsoft SQL Server is a relational database management system developed by Microsoft
+* [__Docsify__](docsify/#docsify) - A magical documentation site generator
+
+Once the stack is up you will have a large number of services running and available on `localhost` <br />
+For Documentation please open http://localhost:3333 in your browser
+
+## Hashicorp basic usage
+* Vault http://localhost:8200
+* Nomad http://localhost:4646
+* Consul http://localhost:8500
+* Localstack http://localhost:8080
+* Terraform Enterprise (enterprise needs a licence) http://localhost:8800
+
+## HashiQube runs all HashiCorp's products
+![HashiQube](images/hashicorp_products.png?raw=true "HashiQube")
+
+## Other
+* LDAP can be accessed on ldap://localhost:389
+* Localstack web http://localhost:8080
+* Jenkins http://localhost:8088
+* Oracle MySQL localhost:3306
+* Microsoft SQL localhost:1433
+
+### Vagrant Basic Usage
+* vagrant up --provision OR vagrant up --provision-with bootstrap|nomad|consul|vault|docker|ldap
+* vagrant global-status # to see which VMs are active
+* vagrant global-status --prune # to remove stale VMs from Vagrant cache
+* vagrant status # vagrant status
+* vagrant reload
+* vagrant up
+* vagrant destroy
+* vagrant provision
+* vagrant plugin list
+
+### Docker Basic Usage
+* docker image ls
+* docker ps
+* docker stop
+
 ## Support & Feedback
 For suggestions, feedback and queries please branch or and submit a Pull Request or directly contact the architects of the HashiQube via email:
 
 Lead Automation Architect [[email protected]](mailto:[email protected]) | Lead Security Architect [[email protected]](mailto:[email protected])    
 
+## Contributors and Special mentions
+A Very special mention to HashiQube's contributors, Thank You All for your help, suggestions and contributions no matter how small <3
+ - Konstantin Vanyushov
+ - Tristan Morgan
 
-#### License
-HashiQube is available as open-source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+## License
+HashiQube is available as open-source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

Vagrantfile

@@ -0,0 +1,204 @@
+# -*- mode: ruby -*-
+# vi: set ft=ruby :
+
+# create local domain name e.g user.local.dev
+user = ENV["USER"].downcase
+fqdn = ENV["fqdn"] || "#{user}.local.dev"
+
+# https://www.virtualbox.org/manual/ch08.html
+vbox_config = [
+  { '--memory' => '4096' },
+  { '--cpus' => '2' },
+  { '--cpuexecutioncap' => '100' },
+  { '--biosapic' => 'x2apic' },
+  { '--ioapic' => 'on' },
+  { '--largepages' => 'on' },
+  { '--natdnshostresolver1' => 'on' },
+  { '--natdnsproxy1' => 'on' },
+  { '--nictype1' => 'virtio' },
+  { '--audio' => 'none' },
+]
+
+# machine(s) hash
+machines = [
+  {
+    :name => "#{fqdn}",
+    :ip => '10.9.99.10',
+    :ssh_port => '2255',
+    :disksize => '10GB',
+    :vbox_config => vbox_config,
+    :synced_folders => [
+      { :vm_path => '/data', :ext_rel_path => '../../', :vm_owner => 'ubuntu' }
+    ],
+  }
+]
+
+Vagrant::configure("2") do |config|
+
+  # check for vagrant version
+  Vagrant.require_version ">= 1.9.7"
+
+  if Vagrant::Util::Platform.windows?
+    COMMAND_SEPARATOR = "&"
+  else
+    COMMAND_SEPARATOR = ";"
+  end
+
+  # auto install plugins, will prompt for admin password on 1st vagrant up
+  required_plugins = %w( vagrant-disksize )
+  required_plugins.each do |plugin|
+    exec "vagrant plugin install #{plugin}#{COMMAND_SEPARATOR}vagrant #{ARGV.join(" ")}" unless Vagrant.has_plugin? plugin || ARGV[0] == 'plugin'
+  end
+
+  machines.each do |machine|
+
+    config.vm.box = "ubuntu/bionic64"
+    config.vm.define machine[:name] do |host|
+
+      config.disksize.size = machine[:disksize]
+      config.ssh.forward_agent = true
+      config.ssh.insert_key = true
+      config.vm.network "private_network", ip: machine[:ip]
+      config.vm.network "forwarded_port", guest: 22, host: machine[:ssh_port], id: 'ssh', auto_correct: true
+      config.vm.network "forwarded_port", guest: 8200, host: 8200 # vault
+      config.vm.network "forwarded_port", guest: 4646, host: 4646 # nomad
+      config.vm.network "forwarded_port", guest: 8500, host: 8500 # consul
+      config.vm.network "forwarded_port", guest: 8600, host: 8600, protocol: 'udp' # consul dns
+      config.vm.network "forwarded_port", guest: 8800, host: 8800 # terraform-enterprise
+      config.vm.network "forwarded_port", guest: 443, host: 4443 # terraform-enterprise
+      config.vm.network "forwarded_port", guest: 8888, host: 8888 # ansible/roles/www
+      config.vm.network "forwarded_port", guest: 8889, host: 8889 # docker/apache2
+      config.vm.network "forwarded_port", guest: 389, host: 3389 # ldap
+      config.vm.network "forwarded_port", guest: 8080, host: 8080 # localstack web
+      config.vm.network "forwarded_port", guest: 8088, host: 8088 # jenkins
+      config.vm.network "forwarded_port", guest: 9002, host: 9002 # consul counter-dashboard
+      config.vm.network "forwarded_port", guest: 9001, host: 9001 # consul counter-api
+      config.vm.network "forwarded_port", guest: 9022, host: 9022 # consul counter-dashboard-test
+      config.vm.network "forwarded_port", guest: 9011, host: 9011 # consul counter-api-test
+      config.vm.network "forwarded_port", guest: 3306, host: 3306 # mysql
+      config.vm.network "forwarded_port", guest: 1433, host: 1433 # mssql
+      config.vm.network "forwarded_port", guest: 9998, host: 9998 # fabio-dashboard
+      config.vm.network "forwarded_port", guest: 9999, host: 9999 # fabiolb
+      config.vm.network "forwarded_port", guest: 3333, host: 3333 # docsify
+    # localstack
+    for port in 4567..4597 do
+      config.vm.network "forwarded_port", guest: "#{port}", host: "#{port}" # localstack
+    end
+
+      config.vm.hostname = "#{fqdn}"
+
+      unless machine[:vbox_config].nil?
+        config.vm.provider :virtualbox do |vb|
+          machine[:vbox_config].each do |hash|
+            hash.each do |key, value|
+              vb.customize ['modifyvm', :id, "#{key}", "#{value}"]
+            end
+          end
+        end
+      end
+
+      # mount the shared folder inside the VM
+      unless machine[:synced_folders].nil?
+        machine[:synced_folders].each do |folder|
+          config.vm.synced_folder "#{folder[:ext_rel_path]}", "#{folder[:vm_path]}", owner: "#{folder[:vm_owner]}", mount_options: ["dmode=777,fmode=777"]
+          # below will mount shared folder via NFS
+          # config.vm.synced_folder "#{folder[:ext_rel_path]}", "#{folder[:vm_path]}", nfs: true, nfs_udp: false, mount_options: ['nolock', 'noatime', 'lookupcache=none', 'async'], linux__nfs_options: ['rw','no_subtree_check','all_squash','async']
+        end
+      end
+
+      # vagrant up --provision-with bootstrap to only run this on vagrant up
+      config.vm.provision "bootstrap", preserve_order: true, type: "shell", privileged: true, inline: <<-SHELL
+        echo -e '\e[38;5;198m'"BEGIN BOOTSTRAP $(date '+%Y-%m-%d %H:%M:%S')"
+        echo -e '\e[38;5;198m'"running vagrant as #{user}"
+        cd ~\n
+        # install applications
+        export DEBIAN_FRONTEND=noninteractive
+        export PATH=$PATH:/root/.local/bin
+        sudo DEBIAN_FRONTEND=noninteractive apt-get --assume-yes update -o Acquire::CompressionTypes::Order::=gz
+        sudo DEBIAN_FRONTEND=noninteractive apt-get --assume-yes upgrade
+        sudo DEBIAN_FRONTEND=noninteractive apt-get --assume-yes install swapspace jq curl unzip software-properties-common bzip2 git make python3-pip python3-dev python3-virtualenv golang-go apt-utils
+        sudo -E -H pip3 install pip --upgrade
+        sudo DEBIAN_FRONTEND=noninteractive apt-get --assume-yes autoremove
+        sudo DEBIAN_FRONTEND=noninteractive apt-get --assume-yes clean
+        sudo rm -rf /var/lib/apt/lists/partial
+
+        # if the user IS jenkins, the we are running this from a Jenkinsfile (Scripted Pipelines)
+        if [ "#{user}" != "jenkins" ]; then
+          cd "#{machine[:synced_folders][0][:vm_path]}"
+          # printenv
+          # below is run from the Makefile, shorthand commands to run composer, gulp, database importer
+          # make bootstrap
+        fi
+        echo -e '\e[38;5;198m'"END BOOTSTRAP $(date '+%Y-%m-%d %H:%M:%S')"
+      SHELL
+
+      # install docker
+      # vagrant up --provision-with docker to only run this on vagrant up
+      config.vm.provision "docker", preserve_order: true, type: "shell", path: "docker/docker.sh"
+
+      # install terraform
+      # vagrant up --provision-with terraform to only run this on vagrant up
+      config.vm.provision "terraform", preserve_order: true, type: "shell", privileged: true, path: "hashicorp/terraform.sh"
+
+      # install terraform-enterprise
+      # vagrant up --provision-with terraform-enterprise to only run this on vagrant up
+      config.vm.provision "terraform-enterprise", run: "never", preserve_order: true, type: "shell", privileged: true, path: "hashicorp/terraform-enterprise.sh"
+
+      # install vault
+      # vagrant up --provision-with vault to only run this on vagrant up
+      config.vm.provision "vault", type: "shell", preserve_order: true, privileged: true, path: "hashicorp/vault.sh"
+
+      # install consul
+      # vagrant up --provision-with consul to only run this on vagrant up
+      config.vm.provision "consul", type: "shell", preserve_order: true, privileged: true, path: "hashicorp/consul.sh"
+
+      # install nomad
+      # vagrant up --provision-with nomad to only run this on vagrant up
+      config.vm.provision "nomad", type: "shell", preserve_order: true, privileged: true, path: "hashicorp/nomad.sh"
+
+      # install packer
+      # vagrant up --provision-with packer to only run this on vagrant up
+      config.vm.provision "packer", type: "shell", preserve_order: true, privileged: true, path: "hashicorp/packer.sh"
+
+      # install sentinel
+      # vagrant up --provision-with sentinel to only run this on vagrant up
+      config.vm.provision "sentinel", type: "shell", preserve_order: true, privileged: true, path: "hashicorp/sentinel.sh"
+
+      # install localstack
+      # vagrant up --provision-with localstack to only run this on vagrant up
+      config.vm.provision "localstack", type: "shell", preserve_order: true, privileged: false, path: "localstack/localstack.sh"
+
+      # vagrant up --provision-with ldap to only run this on vagrant up
+      # run ldap docker container for testing with vault (for example) ldap login
+      config.vm.provision "ldap", run: "never", type: "shell", preserve_order: true, privileged: true, path: "ldap/ldap.sh"
+
+      # vagrant up --provision-with mysql to only run this on vagrant up
+      # run mysql docker container for testing with vault
+      config.vm.provision "mysql", run: "never", type: "shell", preserve_order: true, privileged: false, path: "database/mysql.sh"
+
+      # vagrant up --provision-with mssql to only run this on vagrant up
+      # run mssql docker container for testing with vault
+      config.vm.provision "mssql", run: "never", type: "shell", preserve_order: true, privileged: false, path: "database/mssql.sh"
+
+      # install jenkins
+      # vagrant up --provision-with jenkins to only run this on vagrant up
+      config.vm.provision "jenkins", run: "never", type: "shell", preserve_order: true, privileged: false, path: "jenkins/jenkins.sh"
+
+      # docsify
+      # vagrant up --provision-with docsify to only run this on vagrant up
+      config.vm.provision "docsify", type: "shell", preserve_order: true, privileged: false, path: "docsify/docsify.sh"
+
+      # vagrant up --provision-with bootstrap to only run this on vagrant up
+      config.vm.provision "welcome", preserve_order: true, type: "shell", privileged: true, inline: <<-SHELL
+        echo -e '\e[38;5;198m'"HashiQube has now been provisioned, and your services should be running."
+        echo -e '\e[38;5;198m'"Below are some links for you to get started."
+        echo -e '\e[38;5;198m'"Main documentation http://localhost:3333 Open this first."
+        echo -e '\e[38;5;198m'"Vault http://localhost:8200"
+        echo -e '\e[38;5;198m'"Consul http://localhost:8500"
+        echo -e '\e[38;5;198m'"Nomad http://localhost:4646"
+        echo -e '\e[38;5;198m'"Fabio http://localhost:9998"
+      SHELL
+
+    end
+  end
+end