Lua in ingress-nginx

[wkloucek]

I saw that you're making use of lua plugins in:

Kubernetes-Docs-Shards/sources/ingress_values.yaml

Lines 7 to 9 in 0297dcb

	- name: balancer-lua
	mountPath: /etc/nginx/lua/balancer.lua
	subPath: balancer.lua

According to kubernetes/ingress-nginx#10186 / kubernetes/ingress-nginx#11821 third party support for Lua plugins is gonna be removed from ingress-nginx.

Does affect this chart and how can it be mitigated? Maybe use a standalone nginx and not a ingress-nginx based nginx?

[alexanderonlyoffice]

Hello @wkloucek, mentioned PRs are disabling the plugins, but we are just mounting the slightly refined balancer, so that it reverts back the endpoint which we actually use afterwards in our load balancing. These changes in the controller will not affect our chart functioning.

[wkloucek]

but we are just mounting the slightly refined balancer,

OK, that looks like it's gonna continue to work.

But actually because of this mechanism I see problems that we can actually use this setup.
We have a ingress controller already on our clusters that we must not change that invasive for the OnlyOffice installation, since it's also used for other applications.

Would it be possible to maybe have a "regular" nginx as a Deployment which receives the traffic from the "untouched" ingress controller (which could be nginx-ingress, HA proxy, ...)

For example Grafana Mimir also ships a NGINX deployment for similar functionality like you have it for OnlyOffice, see https://github.com/grafana/mimir/tree/main/operations/helm/charts/mimir-distributed/templates/gateway

[rikatz]

sorry for be intrusive, but I would love to know how kubernetes/ingress-nginx#11819 will impact you folks :)

Can you please provide some feedback on some features we are marking to deprecate as well?

thanks

[alexanderonlyoffice]

@wkloucek,

"We have a ingress controller already on our clusters that we must not change that invasive for the OnlyOffice installation, since it's also used for other applications."

Please specify if you have already looked through this section of the article where the usage of already set nginx-ingress controller is mentioned. Please let me know what you think about it.

"Would it be possible to maybe have a "regular" nginx as a Deployment which receives the traffic from the "untouched" ingress controller (which could be nginx-ingress, HA proxy, ...)"

Our Nginx Ingress Controller based load balancing uses the built-in mechanism for obtaining DS replica endpoints and our lua proxy script based on a request argument. We require to consider and implement such a mechanism when we are talking about "regular" nginx.

@rikatz,

"sorry for be intrusive, but I would love to know how kubernetes/ingress-nginx#11819 will impact you folks :)"

We checked the deployment with enabled controller.enableAnnotationValidations and this parameter did not affect chart functioning. Do you have a test build with the changes added by this PR somewhere, we would like to do more testing?

"Can you please provide some feedback on some features we are marking to deprecate as well?"

Unfortunately, we saw only few of them, do you have a list of the features you are going to mark to deprecate in the future releases, so to say "a to-do" list?

[rikatz]

kubernetes/ingress-nginx#10186

[alexanderonlyoffice]

@rikatz, thanks for pointing it out for us again, we can confirm that the changes to be added in the next release will not affect our chart. We will track the future changes as well to make sure everything will works from our end.

[wkloucek]

Please specify if you have already looked through this section of the article where the usage of already set nginx-ingress controller is mentioned. Please let me know what you think about it.

Honestly I'd still prefer a more self contained solution like described in #13 (comment):

Would it be possible to maybe have a "regular" nginx as a Deployment which receives the traffic from the "untouched" ingress controller (which could be nginx-ingress, HA proxy, ...)
For example Grafana Mimir also ships a NGINX deployment for similar functionality like you have it for OnlyOffice, see https://github.com/grafana/mimir/tree/main/operations/helm/charts/mimir-distributed/templates/gateway

With the current solution, how can I deploy OnlyOffice on a cluster:

• where I don't manage the ingress controller, like in multi tenant clusters?
• where I don't have nginx-ingress as ingress controller? Eg. OpenShift has HAproxy by default
• where I want to deploy OnlyOffice multiple times in different namespaces?

[alexonlyoffice]

@wkloucek, thank you for the feedback.

Honestly I'd still prefer a more self contained solution like described in #13 (comment):

We discussed your suggestion and we are considering adding some autonomous load balancing service behind the Ingress Controller. As of now I can say for sure that the changes made to the balancer should not affect access via the Ingress to other applications.

Regarding your questions, please find the answers below:

where I don't manage the ingress controller, like in multi tenant clusters?

The currently installed version will not work without the access to management (update) of the already installed Ingress Controller.

where I don't have nginx-ingress as ingress controller? Eg. OpenShift has HAproxy by default

Right now current iteration supports only Nginx Ingress Controller, please refer to this link https://github.com/ONLYOFFICE/Kubernetes-Docs-Shards?tab=readme-ov-file#4-configure-dependent-charts

where I want to deploy OnlyOffice multiple times in different namespaces?

There are might be problems with non-WOPI (standard API) integration on the current deployment as of now. We will check how they can be overcome.

[wkloucek]

The error handling in one of the lua scripts seems to be odd:

onlyoffice-ingress-nginx-controller-6699bdfb96-wg5v8 controller 10.244.0.59 - - [11/Oct/2024:07:25:25 +0000] "GET /hosting/wopi/word/edit?WOPISrc=http%3A%2F%2Fcollaboration-onlyoffice.ocis.svc.cluster.local%3A9300%2Fwopi%2Ffiles%2F931b364de4ceb7f9a10ea954aa844b5495b8023e3d9da46c7e0c4fa4bee12edb&ui=en-US HTTP/2.0" 200 77 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36" 156 0.000 [onlyoffice-documentserver-8888] [] - - - - 510715c84391f0803375a185831ecec4

There shouldn't be any failure that is ONLY visible to end users in the frontend without raising any error log in the backend.

[alexonlyoffice]

@wkloucek, please specify if you are deploying the solution from master branch. Let me also know what namespace Redis is deployed at, is it the default one? If you used another namespace for Redis deployment, please make sure you indicated it here and changed service name for Redis.

There shouldn't be any failure that is ONLY visible to end users in the frontend without raising any error log in the backend.

we will add logging to the controller.

[wkloucek]

I was working on this deployment example: owncloud/ocis-charts#765

I got it working after looking into the sharded chart. But actually since the lua configmaps have

Kubernetes-Docs-Shards/templates/configmaps/balancer-lua.yaml

Line 18 in fe4a0a1

helm.sh/hook: pre-install

, the config map is only created once. Therefore changes to the redis host, are not being applied. Which is somehow counter-intuitive to be honest...

[alexonlyoffice]

@wkloucek, it is better to use branch 1.2.0 if you want to keep testing at the moment. In the upcoming release 8.2 we are going to discontinue using balancer-lua and the whole controller schema will be changed.

[wkloucek]

Thanks a lot for this info! I'm gonna dig into that branch!

[wkloucek]

From my side we can close this ticket because the lua is configured on the ingress object itself (as of release/1.2.0 branch)