Add support for off-cluster postgres, kafka and zookeeper to helm chart

CHANGELOG.md

@@ -1,3 +1,17 @@
+## Development
+
+### Kubernetes
+* Fix usage of 'pullPolicy' values in deployments
+* Change env vars setup for deployments. Setup only necessary vars for choosen bus type
+* Documentation updates
+* Add CoAP-WebSockets proxy deployment and service in chart
+* Remove separate switch for deploying external WebSocket proxy. It is requred for Plugin management service and must be deployed if it is enabled.
+* Add top level Ingress in chart Notes
+
+### Docker Compose
+* Add compose file for CoAP-WebSockets proxy
+* Add `DEBUG_RMI_HOSTNAME` variable for setting up JMX debug access via env
+
 ## 3.5.0 / 2018-06-04
 
 * k8s: add parameters for log level configuration in Java Server services

README.md

@@ -28,10 +28,14 @@ More details in the [rdbms-image](rdbms-image/) subdirectory.
 Installation was tested on machine with CentOS 7 distribution.
 
 ## Kubernetes installation
-DeviceHive can be installed on Kubernetes with provided [Helm chart](k8s/). This chart also installs PostgreSQL chart and Kafka chart from [Kubeapps](https://kubeapps.com) repositories. External installations of PostgreSQL and Kafka are not supported at the moment.
+### DeviceHive
+DeviceHive can be installed on Kubernetes with provided [devicehive Helm chart](k8s/devicehive). This chart also installs PostgreSQL chart and Kafka chart from [Kubeapps](https://kubeapps.com) repositories. External installations of PostgreSQL and Kafka are not supported at the moment.
 
 Previous installation method on Kubernetes using a `kubectl` utility and a plain YAML files are deprecated now. Please [issue a ticket](https://github.com/devicehive/devicehive-docker/issues/new) in our [GitHub repository](https://github.com/devicehive/devicehive-docker/) if you have questions about mirgating such environment to the one deployed with Helm chart.
 
+### Cassandra storage plugin
+DeviceHive Cassandra storage plugin can be installed on Kubernetes with provided [devicehive-cassandra-plugin Helm chart](k8s/devicehive-cassandra-plugin). It requires already running cassandra cluster. README file contains [example installation of Cassandra](k8s/devicehive-cassandra-plugin/README.md#example-installation-with-cassandra-cluster-installed-via-helm) with Helm for tests.
+
 ## Installation on Docker for Windows or Docker for Mac
 If you like to try DeviceHive using Docker for Windows or Docker for Mac, please note that this software runs Docker in special Virtual Machine (that got automaticaly created for you by installer). By default these Virtual Machines with much lower parameters that required for DeviceHive, 2GB of RAM and 2 vCPU. Here is example of how to change parameters in Docker for Windows, on Macs this should be similar:
 

k8s/README.md → k8s/devicehive/README.md

@@ -38,17 +38,18 @@ The command deploys DeviceHive on the Kubernetes cluster in the default configur
 Default DeviceHive admin user has name `dhadmin` and password `dhadmin_#911`.
 
 ### Service endpoints
-Table below lists endpoints where you can find various DeviceHive services. If `proxy.ingress` set to `true`, replace *localhost* with hostname(s) used in `proxy.ingress.hosts` parameter.
-
-| Service              | URL                               | Notes                        |
-|----------------------|-----------------------------------|------------------------------|
-| Admin Console        | http://*localhost*/admin          |                              |
-| Frontend service API | http://*localhost*/api/rest       |                              |
-| Auth service API     | http://*localhost*/auth/rest      |                              |
-| Plugin service API   | http://*localhost*/plugin/rest    | If enabled, see [Run with DeviceHive Plugin Service](#run-with-devicehive-plugin-service) section below |
-| Frontend Swagger     | http://*localhost*/api/swagger    |                              |
-| Auth Swagger         | http://*localhost*/auth/swagger   |                              |
-| Plugin Swagger       | http://*localhost*/plugin/swagger | If Plugin service is enabled |
+Table below lists endpoints where you can find various DeviceHive services. If `ingress` set to `true`, replace *localhost* with hostname(s) used in `ingress.hosts` parameter.
+
+| Service                       | URL                               | Notes                        |
+|-------------------------------|-----------------------------------|------------------------------|
+| Admin Console                 | http://*localhost*/admin          |                              |
+| Frontend service API          | http://*localhost*/api/rest       |                              |
+| Auth service API              | http://*localhost*/auth/rest      |                              |
+| Plugin management service API | http://*localhost*/plugin/rest    | If enabled, see [Install with DeviceHive Plugin Management Service](#install-with-devicehive-plugin-management-service) section below |
+| External WS Proxy for plugins | http://*localhost*/plugin/proxy   | If Plugin service is enabled |
+| Frontend Swagger              | http://*localhost*/api/swagger    |                              |
+| Auth Swagger                  | http://*localhost*/auth/swagger   |                              |
+| Plugin Swagger                | http://*localhost*/plugin/swagger | If Plugin service is enabled |
 
 ## Uninstalling the Chart
 
@@ -62,7 +63,7 @@ The command removes all the Kubernetes components associated with the chart and
 
 ## Configuration
 
-The following tables lists the configurable parameters of the DeviceHive chart and their default values.
+The following table lists the configurable parameters of the DeviceHive chart and their default values.
 
 Parameter | Description | Default
 --------- | ----------- | -------
@@ -102,6 +103,13 @@ Parameter | Description | Default
 `backendNode.loggerLevel` | Node backend logger level (levels: debug, info, warn, error ) | `info`
 `backendNode.replicaCount` | Desired number of Node backend pods | `1`
 `backendNode.resources` | Node backend  resource requests and limits | `{}`
+`coapProxy.enabled` | If true, CoAP-WebSockets proxy will be deployed | `false`
+`coapProxy.image` | CoAP-WebSockets proxy image and tag | `devicehive/devicehive-coap-proxy:1.0.0`
+`coapProxy.pullPolicy`| CoAP-WebSockets proxy image pull policy | `IfNotPresent`
+`coapProxy.replicaCount` | Desired number of CoAP-WebSockets proxy pods | `1`
+`coapProxy.resources` | CoAP-WebSockets proxy deployment resource requests and limits | `{}`
+`coapProxy.service.type` | Type of CoAP-WebSockets proxy service to create | `ClusterIP`
+`coapProxy.service.port` | CoAP-WebSockets proxy service port | `5683`
 `mqttBroker.enabled` | If true, DH MQTT broker will be deployed | `false`
 `mqttBroker.appLogLevel` | Application logger level (levels: debug, info, warn, error) | `info`
 `mqttBroker.image` | MQTT broker image and tag | `devicehive/devicehive-mqtt:1.1.0`
@@ -121,7 +129,6 @@ Parameter | Description | Default
 `wsProxy.pullPolicy` | DH WS Proxy image pull policy | `IfNotPresent`
 `wsProxy.internal.replicaCount` | Desired number of internal WS Proxy service pods | `1`
 `wsProxy.internal.resources` | Internal WS Proxy service resource requests and limits  | `{}`
-`wsProxy.external.enabled` | If true, External WS Proxy deployment will be created. Requires `javaServer.plugin.enabled` set to `true` | `false`
 `wsProxy.external.replicaCount` | Desired number of external WS Proxy service pods | `1`
 `wsProxy.external.resources` | External WS Proxy service resource requests and limits | `{}`
 `nodeSelector` | Node labels for DeviceHive pods assignment | `{}`
@@ -151,6 +158,26 @@ $ helm install ./devicehive --name my-release -f values.yaml
 
 > **Tip**: You can use the default [values.yaml](devicehive/values.yaml)
 
+### Install with DeviceHive Plugin Management Service
+
+Plugin management service disabled by default. To enable it you need to pass several values to `helm`.
+Change <external_hostname> to hostname pointing to your cluster. For example, if you setup Ingress resource with host 'devicehive.example.com' then pluginConnectUrl will be 'ws://devicehive.example.com/plugin/proxy':
+``` console
+$ helm install \
+  --name my-release
+  --set javaServer.plugin.enabled=true \
+  --set javaServer.plugin.pluginConnectUrl=ws://<external_hostname>/plugin/proxy \
+  ./devicehive
+```
+or with following parameters in values file:
+``` yaml
+javaServer:
+  plugin:
+    enabled: true
+    pluginConnectUrl: ws://<external_hostname>/plugin/proxy
+```
+Enabling Plugin management service automaticaly enables external WebSocket proxy for plugins.
+
 ### RBAC Configuration
 First, Helm itself requires additional configuration to use on Kubernetes clusters where RBAC enabled. Follow instructions in [Helm documentation](https://docs.helm.sh/using_helm/#role-based-access-control).
 
@@ -160,3 +187,31 @@ To manually setup RBAC you need to set the parameter rbac.create=false and speci
 
 ### Ingress TLS
 Ingress TLS doesn't supported yet by this Helm chart.
+
+### Setting up horizontal autoscaling for services
+
+Autoscaling DeviceHive in Kubernetes relies on Horizontal Pod Authoscaler in your cluster. DeviceHive Helm chart provides ability to set resources for pods and cluster administrator have to create HPA manualy.
+
+When deploying application specify .resource.requests values, see [Configuration section](#configuration) for available values. Here is example from `values.yaml` file used by `helm install --name test ./devicehive -f values.yaml`:
+```yaml
+javaServer:
+  backend:
+    resources:
+      requests:
+        cpu: 2
+        memory: 1536Mi
+  frontend:
+    resources:
+      requests:
+        cpu: 2
+        memory: 1536Mi
+```
+
+When resources.requests for pods are set create hpa by issuing follwing commands:
+```console
+$ kubectl autoscale deployment test-devicehive-backend --cpu-percent=70 --min=1 --max=3
+$ kubectl autoscale deployment test-devicehive-frontend --cpu-percent=70 --min=1 --max=3
+$ kubectl get hpa
+```
+
+> **Note**: resources.requests values and HPA configuration provided above had to be tweaked for your deployment. Please consult [HPA walkthrough](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/) in Kubernetes documentation for more details.

k8s/devicehive/templates/NOTES.txt

@@ -1,8 +1,13 @@
-Thank you for installing {{ .Chart.Name }}.
+Thank you for installing {{ .Chart.Name }} chart.
 
 Your release is named {{ .Release.Name }}.
 
-{{ if .Values.proxy.ingress.enabled -}}
+{{ if .Values.ingress.enabled -}}
+From outside the cluster, DeviceHive Admin Console URL(s) are:
+{{- range .Values.ingress.hosts }}
+http://{{ . }}/admin/
+{{- end }}
+{{- else if .Values.proxy.ingress.enabled -}}
 From outside the cluster, DeviceHive Admin Console URL(s) are:
 {{- range .Values.proxy.ingress.hosts }}
 http://{{ . }}/admin/

k8s/devicehive/templates/coap-proxy-deployment.yaml

@@ -0,0 +1,42 @@
+{{- if .Values.coapProxy.enabled }}
+apiVersion: apps/v1beta1
+kind: Deployment
+metadata:
+  name: {{ .Release.Name }}-devicehive-coap-proxy
+  labels:
+    chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
+    component: "coap-proxy"
+    heritage: "{{ .Release.Service }}"
+    release: "{{ .Release.Name }}"
+spec:
+  replicas: {{ .Values.coapProxy.replicaCount }}
+  template:
+    metadata:
+      labels:
+        app: {{ .Release.Name }}-devicehive-coap-proxy
+    spec:
+      serviceAccountName: {{ if .Values.rbac.create }}{{ template "devicehive.fullname" . }}{{ else }}"{{ .Values.rbac.serviceAccountName }}"{{ end }}
+      containers:
+      - name: coap-proxy
+        image: {{ .Values.coapProxy.image | quote }}
+        imagePullPolicy: {{ .Values.coapProxy.pullPolicy }}
+        env:
+        - name: ENVSEPARATOR
+          value: "_"
+        - name: PROXY_HOST
+          value: 0.0.0.0
+        - name: PROXY_PORT
+          value: "5683"
+        - name: PROXY_TARGET
+          value: "ws://{{ .Release.Name }}-devicehive-frontend:8080/api/websocket"
+        ports:
+        - name: coap
+          protocol: UDP
+          containerPort: 5683
+        resources:
+{{ toYaml .Values.coapProxy.resources | indent 10 }}
+    {{- with .Values.nodeSelector }}
+      nodeSelector:
+{{ toYaml . | indent 8 }}
+    {{- end }}
+{{- end }}

k8s/devicehive/templates/coap-proxy-service.yaml

@@ -0,0 +1,14 @@
+{{- if .Values.coapProxy.enabled }}
+kind: Service
+apiVersion: v1
+metadata:
+  name: {{ .Release.Name }}-devicehive-coap-proxy
+spec:
+  selector:
+    app: {{ .Release.Name }}-devicehive-coap-proxy
+  type: {{ .Values.coapProxy.service.type }}
+  ports:
+    - protocol: UDP
+      port: {{ .Values.coapProxy.service.port | int }}
+      targetPort: coap
+{{- end }}

k8s/devicehive/templates/dh-auth-deployment.yaml

@@ -18,21 +18,21 @@ spec:
       containers:
       - name: devicehive-auth
         image: "{{ .Values.javaServer.repository }}/devicehive-auth:{{ .Values.javaServer.tag }}"
-        imagePullPolicy: {{ .Values.javaServer.PullPolicy }}
+        imagePullPolicy: {{ .Values.javaServer.pullPolicy }}
         env:
         {{- if eq .Values.javaServer.bus "rpc" }}
         - name: SPRING_PROFILES_ACTIVE
           value: "rpc-client"
-        {{- else }}
-        - name: DH_WS_PROXY
-          value: "{{ .Release.Name }}-devicehive-ws-proxy-internal:3000"
-        {{- end }}
         - name: DH_KAFKA_BOOTSTRAP_SERVERS
           value: "{{ .Release.Name }}-kafka:9092"
         - name: DH_ZK_ADDRESS
           value: "{{ .Release.Name }}-zookeeper"
         - name: DH_ZK_PORT
           value: "2181"
+        {{- else }}
+        - name: DH_WS_PROXY
+          value: "{{ .Release.Name }}-devicehive-ws-proxy-internal:3000"
+        {{- end }}
         - name: DH_POSTGRES_ADDRESS
           value: "{{ .Release.Name }}-postgresql"
         - name: DH_POSTGRES_DB

k8s/devicehive/templates/dh-backend-deployment.yaml

@@ -19,21 +19,21 @@ spec:
       containers:
       - name: devicehive-backend
         image: "{{ .Values.javaServer.repository }}/devicehive-backend:{{ .Values.javaServer.tag }}"
-        imagePullPolicy: {{ .Values.javaServer.PullPolicy }}
+        imagePullPolicy: {{ .Values.javaServer.pullPolicy }}
         env:
         {{- if eq .Values.javaServer.bus "rpc" }}
         - name: SPRING_PROFILES_ACTIVE
           value: "rpc-server"
-        {{- else }}
-        - name: DH_WS_PROXY
-          value: "{{ .Release.Name }}-devicehive-ws-proxy-internal:3000"
-        {{- end }}
         - name: DH_KAFKA_BOOTSTRAP_SERVERS
           value: "{{ .Release.Name }}-kafka:9092"
         - name: DH_ZK_ADDRESS
           value: "{{ .Release.Name }}-zookeeper"
         - name: DH_ZK_PORT
           value: "2181"
+        {{- else }}
+        - name: DH_WS_PROXY
+          value: "{{ .Release.Name }}-devicehive-ws-proxy-internal:3000"
+        {{- end }}
         - name: DH_POSTGRES_ADDRESS
           value: "{{ .Release.Name }}-postgresql"
         - name: DH_POSTGRES_DB

k8s/devicehive/templates/dh-backend-node-deployment.yaml

@@ -19,7 +19,7 @@ spec:
       containers:
       - name: devicehive-backend-node
         image: {{ .Values.backendNode.image | quote }}
-        imagePullPolicy: {{ .Values.backendNode.PullPolicy }}
+        imagePullPolicy: {{ .Values.backendNode.pullPolicy }}
         env:
         - name: ENVSEPARATOR
           value: '_'

k8s/devicehive/templates/dh-frontend-deployment.yaml

@@ -18,21 +18,21 @@ spec:
       containers:
       - name: devicehive-frontend
         image: "{{ .Values.javaServer.repository }}/devicehive-frontend:{{ .Values.javaServer.tag }}"
-        imagePullPolicy: {{ .Values.javaServer.PullPolicy }}
+        imagePullPolicy: {{ .Values.javaServer.pullPolicy }}
         env:
         {{- if eq .Values.javaServer.bus "rpc" }}
-        - name: SPRING_PROFILES_ACTIVE
-          value: "rpc-client"
-        {{- else }}
-        - name: DH_WS_PROXY
-          value: "{{ .Release.Name }}-devicehive-ws-proxy-internal:3000"
-        {{- end }}
         - name: DH_KAFKA_BOOTSTRAP_SERVERS
           value: "{{ .Release.Name }}-kafka:9092"
         - name: DH_ZK_ADDRESS
           value: "{{ .Release.Name }}-zookeeper"
         - name: DH_ZK_PORT
           value: "2181"
+        - name: SPRING_PROFILES_ACTIVE
+          value: "rpc-client"
+        {{- else }}
+        - name: DH_WS_PROXY
+          value: "{{ .Release.Name }}-devicehive-ws-proxy-internal:3000"
+        {{- end }}
         - name: DH_AUTH_URL
           value: "http://{{ .Release.Name }}-devicehive-auth:8090/auth/rest"
         - name: DH_POSTGRES_ADDRESS

k8s/devicehive/templates/dh-hazelcast-deployment.yaml

@@ -18,7 +18,7 @@ spec:
       containers:
       - name: devicehive-hazelcast
         image: "{{ .Values.javaServer.repository }}/devicehive-hazelcast:{{ .Values.javaServer.tag }}"
-        imagePullPolicy: {{ .Values.javaServer.PullPolicy }}
+        imagePullPolicy: {{ .Values.javaServer.pullPolicy }}
         env:
         - name: MIN_HEAP_SIZE
           value: {{ .Values.javaServer.hazelcast.minHeapSize | quote }}

k8s/devicehive/templates/dh-plugin-deployment.yaml

@@ -19,15 +19,11 @@ spec:
       containers:
       - name: devicehive-plugin
         image: "{{ .Values.javaServer.repository }}/devicehive-plugin:{{ .Values.javaServer.tag }}"
-        imagePullPolicy: {{ .Values.javaServer.PullPolicy }}
+        imagePullPolicy: {{ .Values.javaServer.pullPolicy }}
         env:
         {{- if eq .Values.javaServer.bus "rpc" }}
         - name: SPRING_PROFILES_ACTIVE
           value: "rpc-client"
-        {{- else }}
-        - name: DH_WS_PROXY
-          value: "{{ .Release.Name }}-devicehive-ws-proxy-internal:3000"
-        {{- end }}
         - name: DH_KAFKA_BOOTSTRAP_SERVERS
           value: "{{ .Release.Name }}-kafka:9092"
         - name: DH_ZK_ADDRESS
@@ -36,6 +32,10 @@ spec:
           value: "2181"
         - name: DH_RPC_CLIENT_RES_CONS_THREADS
           value: "3"
+        {{- else }}
+        - name: DH_WS_PROXY
+          value: "{{ .Release.Name }}-devicehive-ws-proxy-internal:3000"
+        {{- end }}
         - name: DH_AUTH_URL
           value: "http://{{ .Release.Name }}-devicehive-auth:8090/auth/rest"
         - name: DH_POSTGRES_ADDRESS
@@ -48,10 +48,8 @@ spec:
           value: "{{ .Values.postgresql.postgresUser }}"
         - name: DH_POSTGRES_PASSWORD
           value: "{{ .Values.postgresql.postgresPassword }}"
-{{- if .Values.wsProxy.external.enabled }}
         - name: DH_PROXY_PLUGIN_CONNECT
           value: {{ .Values.javaServer.plugin.pluginConnectUrl | default "ws://localhost/plugin/proxy" | quote }}
-{{- end }}
         - name: DH_ZK_ADDRESS
           value: "{{ .Release.Name }}-zookeeper"
         - name: DH_ZK_PORT

k8s/devicehive/templates/dh-proxy-deployment.yaml

@@ -19,7 +19,7 @@ spec:
       containers:
       - name: devicehive-proxy
         image: {{ .Values.proxy.image | quote }}
-        imagePullPolicy: {{ .Values.proxy.PullPolicy }}
+        imagePullPolicy: {{ .Values.proxy.pullPolicy }}
         ports:
         - name: http
           containerPort: 8080

k8s/devicehive/templates/dh-proxy-locations-configmap.yaml

@@ -62,8 +62,6 @@ data:
         proxy_set_header X-Forwarded-Proto $scheme;
         proxy_set_header X-Real-IP $remote_addr;
     }
-{{- end }}
-{{- if .Values.wsProxy.external.enabled }}
     location /plugin/proxy {
         proxy_redirect off;
         proxy_pass http://wsproxyext/;

k8s/devicehive/templates/dh-proxy-upstreams-configmap.yaml

@@ -14,8 +14,6 @@ data:
     upstream plugin_upstream {
         server {{ .Release.Name }}-devicehive-plugin:8110;
     }
-{{- end }}
-{{- if .Values.wsProxy.external.enabled }}
     upstream wsproxyext {
         server {{ .Release.Name }}-devicehive-ws-proxy-external:3000;
     }