Maintenance - Safe Shutdown / Preserve state after server downtime

[arunseelagan-cisa]

Is your feature request related to a problem? Please describe.
When we need to take down the server hosting LME for maintenance purposes, what is the best way to stop the LME docker stack without completely destroying the existing containers?

We want to be able to re-run docker stack after the downtime and still have access to all the data that was present beforehand.

Describe the solution you'd like
Please add a section under the "Maintenance" section of https://github.com/cisagov/LME, where you provide guidance on how to properly stop LME in a manner that preserves state.

Describe alternatives you've considered
Tried to run "docker stop <lme_container_ids>" but the containers come right back up automatically.

docker remove commands are not a good alternative, because that would theoretically destroy all the data inside the containers.

Additional context
N/A

[aarz-snl]

May want to look into docker stack commands...

Something like

docker stack services lme -q | xargs docker service update --replicas 0

Which would temporarily stop the containers

And then to restart

docker stack services lme -q | xargs docker service update --replicas 1

[aarz-snl]

This worked for me:

docker service scale lme_elasticsearch=0 lme_kibana=0 lme_logstash=0

Then

docker service scale lme_elasticsearch=1 lme_kibana=1 lme_logstash=1
To restart

took about a minute to stop them, and about 3 minutes to start back up