Skip to content

Latest commit

 

History

History
244 lines (196 loc) · 9.22 KB

Home.md

File metadata and controls

244 lines (196 loc) · 9.22 KB
Raw

Google Kubernetes Engine Plugin Documentation

The Google Kubernetes Engine (GKE) Plugin allows you to publish deployments built within Jenkins to your Kubernetes clusters running within GKE.

Prerequisites

  • Minimum Jenkins version: 2.164.2

  • Jenkins plugin dependencies:

    NOTE: Unless otherwise specified, pre-installation of these plugins aren't required. Just note that if a conflicting version is present a class-loading error could occur.

    • google-oauth-plugin: 0.7 (pre-installation required)
    • workflow-step-api: 2.19
    • pipeline-model-definition: 1.3.8 (pre-installation required for Pipeline DSL support)
    • git: 3.9.3
    • junit: 1.3
    • structs: 1.17
    • credentials: 2.1.16

Setup

Setup the necessary environment variables

export PROJECT=$(gcloud info --format='value(config.project)')
export CLUSTER=<YOUR_CLUSTER_NAME>
export ZONE=<YOUR_PROJECTS_ZONE>
export SA=<YOUR_GCP_SA_NAME>
export SA_EMAIL=${SA}@${PROJECT}.iam.gserviceaccount.com

Enable Required GCP APIs

gcloud services enable compute.googleapis.com \
container.googleapis.com \
servicemanagement.googleapis.com \
cloudresourcemanager.googleapis.com \
    --project $PROJECT

Configure target GKE cluster

  1. If necessary, create your GKE cluster:

    gcloud container clusters create $CLUSTER --zone $ZONE

  2. Retrieve the KubeConfig for your cluster:

    gcloud container clusters get-credentials $CLUSTER --zone $ZONE

  3. If necessary, grant your GCP login account cluster-admin permissions necessary for creating cluster role bindings:

    kubectl create clusterrolebinding cluster-admin-binding --clusterrole=cluster-admin \
     --user=$(gcloud config get-value account)

Manually Configure GCP Service Account Permissions

This section describes the manual procedures for configuring the permissions necessary for your GCP service account to deploy to your GKE cluster. Automation is also provided for executing these procedures using Terraform and Helm in the Automated Permissions Configuration section.

GCP IAM Permissions

  1. Create a service account using the Google Cloud SDK:

    gcloud iam service-accounts create $SA

  2. Create custom GCP IAM Role with minimal permissions using the custom role defined within rbac/IAMrole.yaml:

    gcloud iam roles create gke_deployer --project $PROJECT --file \
rbac/IAMrole.yaml

  3. Grant the IAM role to your GCP service account:

    gcloud projects add-iam-policy-binding $PROJECT \
--member serviceAccount:$SA_EMAIL \
--role projects/$PROJECT/roles/gke_deployer

  4. Download a JSON Service Account key for your newly created service account. Take note of where the file was created, you will upload it to Jenkins in a subsequent step:

    gcloud iam service-accounts keys create ~/jenkins-gke-key.json --iam-account $SA_EMAIL
    • If using cloud shell, click the 3 vertical dots and Download file, then enter "jenkins-gke-key.json".

  5. In Jenkins on the left side of the screen, click on Credentials, then System.

  6. Click Global credentials then Add credentials on the left.

  7. In the Kind dropdown, select Google Service Account from private key.

  8. Enter your project name, then select your JSON key that was created in the preceding steps.

  9. Click OK.

GKE Cluster RBAC Permissions

Grant your GCP service account a restricted set of RBAC permissions allowing it to deploy to your GKE cluster.

  1. Create the custom robot-deployer cluster role defined within rbac/robot-deployer.yaml:

    kubectl create -f rbac/robot-deployer.yaml

  2. Grant your GCP service account the robot-deployer role binding using rbac/robot-deployer-bindings.yaml:

    envsubst < rbac/robot-deployer-bindings.yaml | kubectl create -f -

Automated Permissions Configuration

This section demonstrates using provided automation for configuring the necessary GCP service account permissions for deploying to your GKE cluster.

GCP IAM Permissions

  1. Navigate to the rbac directory:

    pushd rbac/

  2. The gcp-sa-setup.tf Terraform plan will create a custom GCP IAM role with restricted permissions, create a GCP service account, and grant said service account the custom role. (NOTE: This only needs to be done once).

    export TF_VAR_PROJECT=${PROJECT}
export TF_VAR_REGION=${REGION}
export TF_VAR_SA_NAME=${SA}
terraform init
terraform plan -out /tmp/tf.plan
terraform apply /tmp/tf.plan
rm /tmp/tf.plan

GKE Cluster RBAC Permissions

  1. Navigate to the helm directory

    popd
pushd helm/

  2. Execute the helm chart provided within this directory to create the robot-deployer cluster role, and grant your GCP service account the robot-deployer role binding. This will need to be executed on each cluster being deployed to by the plugin. If necessary follow the instructions provided by the helm project for configuring helm/tiller on your GKE cluster.

    export TARGET_NAMESPACE=<YOUR_TARGET_NAMESPACE>
envsubst < gke-robot-deployer/values.yaml | helm install ./gke-robot-deployer --name gke-robot-deployer -f -
References:

Usage

Google Kubernetes Engine Build Step Configuration

Each GKE Build Step configuration can point to a different GKE cluster. Follow the steps below to create one.

GKE Build Step Parameters

The GKE Build Step has the following parameters:

  1. credentialsId(string): The ID of the credentials that you uploaded earlier.
  2. projectId(string): The Project ID housing the GKE cluster to be published to.
  3. zone(string): [Deprecated] The Zone housing the GKE cluster to be published to.
  4. location(string): The Zone or Region housing the GKE cluster to be published to.
  5. clusterName(string): The name of the Cluster to be published to.
  6. manifestPattern(string): The file pattern of the Kubernetes manifest to be deployed.
  7. verifyDeployments(boolean): [Optional] Whether the plugin will verify deployments.

Jenkins Web UI

  1. On the Jenkins home page, select the project to be published to GKE.
  2. Click Configure from the left nav-bar.
  3. At the bottom of the page there will be a button labeled Add build step, click the button then select Deploy to Google Kubernetes Engine.
  4. In the Service Account Credentials dropdown, select the credentials that you uploaded earlier. This should autopopulate Project ID and Cluster, if not:
  • Select the Project ID housing the GKE cluster to be published to.
  • Select the Cluster to be published to.
  1. Enter the file path of the Kubernetes manifest within your project to be used for deployment.

Jenkins Declarative Pipeline

  1. Create a file named "Jenkinsfile" in the root of your project.
  2. Within your Jenkinsfile add a step which invokes the GKE plugin's build step class: "KubernetesEngineBuilder". See the example code below:
pipeline {
    agent any
    environment {
        PROJECT_ID = '<YOUR_PROJECT_ID>'
        CLUSTER_NAME = '<YOUR_CLUSTER_NAME>'
        LOCATION = '<YOUR_CLUSTER_LOCATION>'
        CREDENTIALS_ID = '<YOUR_CREDENTIAS_ID>'
    }
    stages {
        stage('Deploy to GKE') {
            steps{
                step([
                $class: 'KubernetesEngineBuilder',
                projectId: env.PROJECT_ID,
                clusterName: env.CLUSTER_NAME,
                location: env.LOCATION,
                manifestPattern: 'manifest.yaml',
                credentialsId: env.CREDENTIALS_ID,
                verifyDeployments: true])
            }
        }
    }
}

Jenkins Environment Configuration

The GKE Jenkins plugin requires the kubectl binary to be installed within the Jenkins agent environment.