Skip to content

Commit

Permalink
OPSEXP-2298: refactor enterprise search deployments and jobs (#132)
Browse files Browse the repository at this point in the history
  • Loading branch information
alxgomz authored Oct 19, 2023
1 parent aab2403 commit 41e9f85
Show file tree
Hide file tree
Showing 27 changed files with 419 additions and 114 deletions.
6 changes: 3 additions & 3 deletions charts/alfresco-repository/Chart.lock
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
dependencies:
- name: alfresco-common
repository: https://alfresco.github.io/alfresco-helm-charts/
version: 3.0.0-alpha.2
version: 3.0.0-alpha.3
- name: postgresql
repository: oci://registry-1.docker.io/bitnamicharts
version: 12.5.6
digest: sha256:8b29bc4958ca18fce2e9b0fe2d8d21b4628cb3e1c13f90226378fe6ac3fbd9ff
generated: "2023-09-25T15:51:04.023194318Z"
digest: sha256:fc5f240540fd10560dca1e88b13601b18128e2fbf0ed0b52ab9294ad58dff037
generated: "2023-10-18T14:43:10.28712+02:00"
4 changes: 2 additions & 2 deletions charts/alfresco-repository/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@ apiVersion: v2
name: alfresco-repository
description: Alfresco content repository Helm chart
type: application
version: 0.1.0-alpha.18
version: 0.1.0-alpha.19
appVersion: 23.1.0-A21
dependencies:
- name: alfresco-common
version: 3.0.0-alpha.2
version: 3.0.0-alpha.3
repository: https://alfresco.github.io/alfresco-helm-charts/
- name: postgresql
version: 12.5.6
Expand Down
4 changes: 2 additions & 2 deletions charts/alfresco-repository/README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# alfresco-repository

![Version: 0.1.0-alpha.18](https://img.shields.io/badge/Version-0.1.0--alpha.18-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 23.1.0-A21](https://img.shields.io/badge/AppVersion-23.1.0--A21-informational?style=flat-square)
![Version: 0.1.0-alpha.19](https://img.shields.io/badge/Version-0.1.0--alpha.19-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 23.1.0-A21](https://img.shields.io/badge/AppVersion-23.1.0--A21-informational?style=flat-square)

Alfresco content repository Helm chart

## Requirements

| Repository | Name | Version |
|------------|------|---------|
| https://alfresco.github.io/alfresco-helm-charts/ | alfresco-common | 3.0.0-alpha.2 |
| https://alfresco.github.io/alfresco-helm-charts/ | alfresco-common | 3.0.0-alpha.3 |
| oci://registry-1.docker.io/bitnamicharts | postgresql | 12.5.6 |

## Values
Expand Down
23 changes: 0 additions & 23 deletions charts/alfresco-repository/templates/_helpers-message-broker.tpl

This file was deleted.

2 changes: 1 addition & 1 deletion charts/alfresco-repository/templates/configmap-mq.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,6 @@ metadata:
{{- include "alfresco-repository.labels" . | nindent 4 }}
data:
{{- with .Values.configuration.messageBroker }}
{{ template "alfresco-repository.mq.cm" .url }}
{{ template "alfresco-common.activemq.cm" .url }}
{{- end }}
{{- end }}
6 changes: 3 additions & 3 deletions charts/alfresco-search-enterprise/Chart.lock
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
dependencies:
- name: alfresco-common
repository: https://alfresco.github.io/alfresco-helm-charts/
version: 3.0.0-alpha.1
version: 3.0.0-alpha.3
- name: activemq
repository: https://alfresco.github.io/alfresco-helm-charts/
version: 3.3.0
- name: elasticsearch
repository: https://helm.elastic.co
version: 7.17.3
digest: sha256:3217395d08bca2dca3ada26c8045f92409e4173deb0e881dd0d020c975b6217c
generated: "2023-09-21T19:55:50.962494+02:00"
digest: sha256:32410a9edb8cebdc613c15998ebbb9f38d1e13c815bf5270ff03b709ed6524fc
generated: "2023-10-18T14:38:30.777957+02:00"
4 changes: 2 additions & 2 deletions charts/alfresco-search-enterprise/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@ apiVersion: v2
name: alfresco-search-enterprise
description: A Helm chart for deploying Alfresco Elasticsearch connector
type: application
version: 3.0.0-alpha.1
version: 3.0.0-alpha.2
appVersion: 3.3.1
dependencies:
- name: alfresco-common
version: 3.0.0-alpha.1
version: 3.0.0-alpha.3
repository: https://alfresco.github.io/alfresco-helm-charts/
- name: activemq
version: 3.3.0
Expand Down
12 changes: 10 additions & 2 deletions charts/alfresco-search-enterprise/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# alfresco-search-enterprise

![Version: 3.0.0-alpha.1](https://img.shields.io/badge/Version-3.0.0--alpha.1-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 3.3.1](https://img.shields.io/badge/AppVersion-3.3.1-informational?style=flat-square)
![Version: 3.0.0-alpha.2](https://img.shields.io/badge/Version-3.0.0--alpha.2-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 3.3.1](https://img.shields.io/badge/AppVersion-3.3.1-informational?style=flat-square)

A Helm chart for deploying Alfresco Elasticsearch connector

Expand All @@ -11,7 +11,7 @@ Please refer to the [documentation](https://github.com/Alfresco/acs-deployment/b
| Repository | Name | Version |
|------------|------|---------|
| https://alfresco.github.io/alfresco-helm-charts/ | activemq | 3.3.0 |
| https://alfresco.github.io/alfresco-helm-charts/ | alfresco-common | 3.0.0-alpha.1 |
| https://alfresco.github.io/alfresco-helm-charts/ | alfresco-common | 3.0.0-alpha.3 |
| https://helm.elastic.co | elasticsearch | 7.17.3 |

## Values
Expand Down Expand Up @@ -56,6 +56,8 @@ Please refer to the [documentation](https://github.com/Alfresco/acs-deployment/b
| liveIndexing.path.image.repository | string | `"quay.io/alfresco/alfresco-elasticsearch-live-indexing-path"` | |
| liveIndexing.path.image.tag | string | `"3.3.1"` | |
| liveIndexing.path.replicaCount | int | `1` | |
| messageBroker.existingConfigMap | object | `{"keys":{"url":"BROKER_URL"},"name":null}` | Alternatively, provide message broker connection details via an existing configmap |
| messageBroker.existingConfigMap.keys.url | string | `"BROKER_URL"` | Key within the configmap holding the URL of the message broker |
| messageBroker.existingSecretName | string | `nil` | Provide connection details alternatively via an existing secret that contains BROKER_URL, BROKER_USERNAME and BROKER_PASSWORD keys |
| messageBroker.password | string | `nil` | Broker password |
| messageBroker.url | string | `nil` | Broker URL formatted as per: https://activemq.apache.org/failover-transport-reference |
Expand All @@ -80,6 +82,9 @@ Please refer to the [documentation](https://github.com/Alfresco/acs-deployment/b
| reindexing.initcontainers.waitForRepository.resources.limits.cpu | string | `"0.25"` | |
| reindexing.initcontainers.waitForRepository.resources.limits.memory | string | `"10Mi"` | |
| reindexing.pathIndexingEnabled | bool | `true` | |
| reindexing.repository.existingConfigMap.keys.url | string | `"REPOSITORY_URL"` | Key within the configmap holding the full url to connect to the alfresco repository |
| reindexing.repository.existingConfigMap.name | string | `nil` | Alternatively, provide repository connection details via an existing configmap |
| reindexing.repository.url | string | `nil` | URL of the Alfresco repository |
| reindexing.resources.limits.cpu | string | `"2"` | |
| reindexing.resources.limits.memory | string | `"512Mi"` | |
| reindexing.resources.requests.cpu | string | `"0.5"` | |
Expand All @@ -96,4 +101,7 @@ Please refer to the [documentation](https://github.com/Alfresco/acs-deployment/b
| searchIndex.protocol | string | `nil` | Valid values are http or https |
| searchIndex.user | string | `nil` | The username required to access the service, if any |
| securityContext | object | `{}` | |
| serviceAccount.annotations | object | `{}` | |
| serviceAccount.create | bool | `true` | |
| serviceAccount.name | string | `"alfresco-search-enterprise-sa"` | |
| tolerations | list | `[]` | |
2 changes: 2 additions & 0 deletions charts/alfresco-search-enterprise/ci/default-values.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# avoid too long resource names being truncated and conflicting
nameOverride: aesc
reindexing:
# requires postgres database at startup
enabled: false
Expand Down
30 changes: 30 additions & 0 deletions charts/alfresco-search-enterprise/templates/_helpers-activemq.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{{/*
Usage: include "alfresco-search-enterprise.activemq.url" $
*/}}
{{- define "alfresco-search-enterprise.activemq.url" -}}
{{- with .Values.activemq }}
{{- if .enabled }}
{{- $mqCtx := dict "Values" (dict "nameOverride" (.nameOverride | default "")) "Chart" (dict "Name" "activemq") "Release" $.Release }}
{{- printf "failover:(nio://%s-broker:61616)?timeout=3000&jms.useCompression=true" (include "activemq.fullname" $mqCtx) }}
{{- else }}
{{- required "Disabling in-cluster ActiveMQ requires passing (at least) messageBroker.url" $.Values.messageBroker.url }}
{{- end }}
{{- end }}
{{- end -}}

{{/*
Usage: include "alfresco-search-enterprise.activemq.cm.env" $
*/}}
{{- define "alfresco-search-enterprise.activemq.cm.env" -}}
{{- $mqCtx := dict "Values" (dict "nameOverride" (printf "%s-%s" ($.Values.nameOverride | default $.Chart.Name) "mq")) "Chart" $.Chart "Release" $.Release }}
{{- $mqCm := coalesce $.Values.messageBroker.existingConfigMap.name (include "alfresco-search-enterprise.fullname" $mqCtx) }}
- name: BROKER_URL
valueFrom:
configMapKeyRef:
name: {{ $mqCm }}
key: {{ $.Values.messageBroker.existingConfigMap.keys.url }}
{{- end -}}
11 changes: 11 additions & 0 deletions charts/alfresco-search-enterprise/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -49,3 +49,14 @@ Selector labels
app.kubernetes.io/name: {{ include "alfresco-search-enterprise.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}

{{/*
Create the name of the service account to use
*/}}
{{- define "alfresco-search-enterprise.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "alfresco-search-enterprise.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
{{- end }}
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,10 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "alfresco-search-enterprise.fullname" . }}-ats
{{- $ctx := dict "Values" (dict "nameOverride" (printf "%s-%s" ($.Values.nameOverride | default $.Chart.Name) "ats")) "Chart" $.Chart "Release" $.Release }}
name: {{ template "alfresco-search-enterprise.fullname" $ctx }}
labels:
{{- include "alfresco-search-enterprise.labels" $ | nindent 4 }}
{{- include "alfresco-search-enterprise.labels" . | nindent 4 }}
data:
{{ template "alfresco-search-enterprise.ats.fullurl" (required "You need to provide the ATS URL using either ats.transform_url value or a configmap" .Values.ats.transform_url) }}
{{ template "alfresco-search-enterprise.sfs.fullurl" (required "You need to provide the SFS URL using either ats.sfs_url value or a configmap" .Values.ats.sfs_url) }}
Expand Down
12 changes: 12 additions & 0 deletions charts/alfresco-search-enterprise/templates/configmap-mq.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{{- if not .Values.messageBroker.existingConfigMap.name -}}
apiVersion: v1
kind: ConfigMap
metadata:
name: >-
{{- $ctx := dict "Values" (dict "nameOverride" (printf "%s-%s" (.Values.nameOverride | default .Chart.Name) "mq")) "Chart" .Chart "Release" .Release }}
{{ template "alfresco-search-enterprise.fullname" $ctx }}
labels:
{{- include "alfresco-search-enterprise.labels" . | nindent 4 }}
data:
{{ template "alfresco-common.activemq.cm" (include "alfresco-search-enterprise.activemq.url" .) }}
{{- end -}}
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{{- with .Values.reindexing }}
{{- if and .enabled (not .repository.existingConfigMap.name) -}}
---
apiVersion: v1
kind: ConfigMap
metadata:
name: >-
{{- $ctx := dict "Values" (dict "nameOverride" (printf "%s-%s" ($.Values.nameOverride | default $.Chart.Name) "repository")) "Chart" $.Chart "Release" $.Release }}
{{ template "alfresco-search-enterprise.fullname" $ctx }}
labels:
{{- include "alfresco-search-enterprise.labels" $ | nindent 4 }}
data:
{{- $failmsg := ".reindexing.repository.url is mandatory when not using existingConfigMap" }}
REPOSITORY_URL: {{ required $failmsg .repository.url | quote }}
{{- end -}}
{{- end -}}
Original file line number Diff line number Diff line change
@@ -1,19 +1,18 @@
{{- range $serviceName, $service := .Values.liveIndexing }}
{{- range $serviceName, $service := omit .Values.liveIndexing "mediation" }}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "alfresco-search-enterprise.fullname" $ }}-{{ $serviceName }}
labels:
{{- include "alfresco-search-enterprise.labels" $ | nindent 4 }}
spec:
{{- if eq $serviceName "mediation"}}
replicas: 1
{{- else }}
app.kubernetes.io/component: {{ printf "%s-indexing" $serviceName }}
spec :
replicas: {{ $service.replicaCount | default 1 }}
{{- end }}
selector:
matchLabels:
{{- include "alfresco-search-enterprise.selectorLabels" $ | nindent 6 }}
app.kubernetes.io/component: {{ printf "%s-indexing" $serviceName }}
template:
metadata:
annotations:
Expand All @@ -25,22 +24,26 @@ spec:
{{- end }}
labels:
{{- include "alfresco-search-enterprise.selectorLabels" $ | nindent 8 }}
app.kubernetes.io/component: {{ printf "%s-indexing" $serviceName }}
spec:
{{- include "component-pod-security-context" .Values | indent 4 }}
{{- include "alfresco-content-services.imagePullSecrets" $ | indent 6 }}
serviceAccountName: {{ include "alfresco-search-enterprise.serviceAccountName" $ }}
{{- include "alfresco-common.component-pod-security-context" .Values | indent 4 }}
{{- include "alfresco-common.imagePullSecrets" $ | indent 6 }}
containers:
- name: {{ $.Chart.Name }}-{{ $serviceName }}
image: "{{ index $.Values "liveIndexing" (printf "%s" $serviceName) "image" "repository" }}:\
{{ index $.Values "liveIndexing" (printf "%s" $serviceName) "image" "tag" }}"
imagePullPolicy: {{ index $.Values "liveIndexing" (printf "%s" $serviceName) "image" "pullPolicy" }}
{{- include "component-security-context" .Values | indent 8 }}
{{- include "alfresco-common.component-security-context" .Values | indent 8 }}
envFrom:
- secretRef:
name: {{ default (printf "%s-messagebroker-secret" (include "alfresco-search-enterprise.fullname" $)) $.Values.messageBroker.existingSecretName }}
- configMapRef:
name: {{ template "alfresco-search-enterprise.fullname" $ }}-es
env:
{{- $ats_cm := $.Values.ats.existingConfigMap.name | default (printf "%s-ats" (include "alfresco-search-enterprise.fullname" $)) }}
{{- include "alfresco-search-enterprise.activemq.cm.env" $ | nindent 12 }}
{{- $atsCtx := dict "Values" (dict "nameOverride" (printf "%s-%s" ($.Values.nameOverride | default $.Chart.Name) "ats")) "Chart" $.Chart "Release" $.Release }}
{{- $ats_cm := coalesce $.Values.ats.existingConfigMap.name (include "alfresco-search-enterprise.fullname" $atsCtx) }}
- name: ALFRESCO_ACCEPTEDCONTENTMEDIATYPESCACHE_BASEURL
valueFrom:
configMapKeyRef:
Expand All @@ -51,7 +54,7 @@ spec:
configMapKeyRef:
name: {{ $ats_cm }}
key: {{ $.Values.ats.existingConfigMap.keys.sfs_url }}
{{- include "spring.activemq.env" . | nindent 12 }}
{{- include "alfresco-common.spring.activemq.env" . | nindent 12 }}
{{- include "alfresco-search-enterprise.config.spring.envCredentials" $ | nindent 12 }}
{{- include "alfresco-search-enterprise.env" $ | nindent 12 }}
{{- range $key, $val := $.Values.liveIndexing.environment }}
Expand Down Expand Up @@ -87,5 +90,4 @@ spec:
tolerations:
{{- toYaml $ | nindent 8 }}
{{- end }}
---
{{- end }}
31 changes: 23 additions & 8 deletions charts/alfresco-search-enterprise/templates/reindexing-job.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,15 @@ spec:
labels:
{{- include "alfresco-search-enterprise.selectorLabels" . | nindent 8 }}
spec:
{{- include "component-pod-security-context" .Values | indent 4 }}
{{- include "alfresco-content-services.imagePullSecrets" . | indent 6 }}
serviceAccountName: {{ include "alfresco-search-enterprise.serviceAccountName" . }}
{{- include "alfresco-common.component-pod-security-context" .Values | indent 4 }}
{{- include "alfresco-common.imagePullSecrets" . | indent 6 }}
restartPolicy: Never
containers:
- name: {{ .Chart.Name }}-reindexing
image: "{{ .Values.reindexing.image.repository }}:{{ .Values.reindexing.image.tag }}"
imagePullPolicy: {{ .Values.reindexing.image.pullPolicy }}
{{- include "component-security-context" .Values | indent 8 }}
{{- include "alfresco-common.component-security-context" .Values | indent 8 }}
resources: {{- toYaml .Values.reindexing.resources | nindent 12 }}
envFrom:
- configMapRef:
Expand Down Expand Up @@ -65,7 +66,8 @@ spec:
configMapKeyRef:
name: {{ .Values.reindexing.db.existingConfigMap.name | default $dbFullName }}
key: {{ .Values.reindexing.db.existingConfigMap.keys.url }}
{{- include "spring.activemq.env" . | nindent 12 }}
{{- include "alfresco-search-enterprise.activemq.cm.env" $ | nindent 12 }}
{{- include "alfresco-common.spring.activemq.env" . | nindent 12 }}
{{- include "alfresco-search-enterprise.config.spring.envCredentials" $ | nindent 12 }}
ports:
- name: http
Expand All @@ -74,12 +76,25 @@ spec:
initContainers:
- name: wait-for-repository
image: curlimages/curl:7.79.1
{{- include "component-security-context" .Values | indent 8 }}
{{- include "alfresco-common.component-security-context" .Values | indent 8 }}
resources: {{- toYaml .Values.reindexing.initcontainers.waitForRepository.resources | nindent 12 }}
env:
- name: ALFRESCO_REPOSITORY_URL
value: http://{{ template "content-services.shortname" . }}-repository/alfresco/api/-default-/public/alfresco/versions/1/probes/-ready-
{{- $repoCtx := dict "Values" (dict "nameOverride" (printf "%s-%s" ($.Values.nameOverride | default $.Chart.Name) "repository")) "Chart" $.Chart "Release" $.Release }}
{{- $repoCm := coalesce .Values.reindexing.repository.existingConfigMap.name (include "alfresco-search-enterprise.fullname" $repoCtx) }}
- name: REPOSITORY_URL
valueFrom:
configMapKeyRef:
name: {{ $repoCm }}
key: {{ .Values.reindexing.repository.existingConfigMap.keys.url }}
command: [ "/bin/sh","-c" ]
# Delay running the reindexing to give Alfresco Repository a chance to fully initialise
args: [ "while [ $(curl -sw '%{http_code}' $ALFRESCO_REPOSITORY_URL -o /dev/null) -ne 200 ]; do sleep 5; echo 'Waiting for the Alfresco Repository...'; done; echo 'Alfresco is ready, delay reindexing to give a chance to fully initialise.'; sleep 30; echo 'Reindexing started!'" ]
args:
- |
while [ $(curl -sw '%{http_code}' $(ALFRESCO_REPOSITORY_URL)/api/-default-/public/alfresco/versions/1/probes/-ready- -o /dev/null) -ne 200 ]
do echo 'Waiting for the Alfresco Repository...'
sleep 5
done
echo 'Alfresco is ready, delay reindexing to give a chance to fully initialise.'
sleep 30
echo 'Reindexing started!'
{{ end }}
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,10 @@ metadata:
type: Opaque
data:
{{- if .Values.activemq.enabled }}
BROKER_URL: {{ printf "failover:(nio://%s-activemq-broker:61616)?timeout=3000&jms.useCompression=true" (include "alfresco-search-enterprise.fullname" .) | b64enc | quote }}
BROKER_USERNAME: {{ .Values.activemq.adminUser.user | b64enc | quote }}
BROKER_PASSWORD: {{ .Values.activemq.adminUser.password | b64enc | quote }}
BROKER_USERNAME: {{ .Values.activemq.adminUser.user | default "admin" | b64enc | quote }}
BROKER_PASSWORD: {{ .Values.activemq.adminUser.password | default "admin" | b64enc | quote }}
{{- else }}
BROKER_URL: {{ required "Disabling in-cluster ActiveMQ requires passing (at least) messageBroker.url" .Values.messageBroker.url | b64enc | quote }}
BROKER_USERNAME: {{ .Values.messageBroker.user | b64enc | quote }}
BROKER_PASSWORD: {{ .Values.messageBroker.password | b64enc | quote }}
BROKER_USERNAME: {{ .Values.messageBroker.user | default "" | b64enc | quote }}
BROKER_PASSWORD: {{ .Values.messageBroker.password | default "" | b64enc | quote }}
{{- end }}
{{- end }}
Loading

0 comments on commit 41e9f85

Please sign in to comment.