[DPE-3243][DPE-3246] Async backup and new list-backups output

This method implements an async backup method: now, the charm will not
wait anymore for the backup action to be finished. Users can check the status
of backups using the list-backups command.

Restore action closes all the target indices and the action will execute
synchronously with the restore process itself. Indices that are closed will not
be writable during that time. If the restore takes longer and the `juju run`
action times out, then the action continues to run in the background.
It is possible to see it happening with `juju status`. OpenSearch opens the
indices after the restore action has been executed.

This PR also fixes the integration tests in backups: now, we wait for the
backup to asynchronously finish its task before moving on to the next ones.

Closed tickets:
* [DPE-3243](https://warthogs.atlassian.net/browse/DPE-3243): list-backups
now contains only relevant information, in this case the backup-id and
its status; the PR removes the "backup-type" column, as it has no
meaning in OpenSearch
* [DPE-3246](https://warthogs.atlassian.net/browse/DPE-3246): backups will
be async and restores will be synchronous. Users can follow the restore
in the juju status. In any case, the indices that are being restored
will be closed, so the user needs to wait.

.github/workflows/integration.yaml

@@ -160,16 +160,6 @@ jobs:
           sudo rm -rf /usr/local/share/boost
           sudo rm -rf "$AGENT_TOOLSDIRECTORY"
 
-      - name: Setup microceph
-        uses: phvalguima/microceph-action@main
-        with:
-          channel: 'latest/edge'
-          devname: '/dev/sdi'
-          accesskey: 'accesskey'
-          secretkey: 'secretkey'
-          bucket: 'testbucket'
-          osdsize: '5G'
-
       - name: Select tests
         id: select-tests
         run: |
@@ -184,13 +174,6 @@ jobs:
 
       - name: Run backup integration
         run: |
-          # load microceph output
-          # ATM: remove the https:// reference and stick with http only
-          sed -i 's@https://@http://@g' microceph.source
-          for i in $(cat microceph.source); do export $i; done
-          export TEST_NUM_APP_UNITS=2
-
-          # Set kernel params
           sudo sysctl -w vm.max_map_count=262144 vm.swappiness=0 net.ipv4.tcp_retries2=5
           tox run -e ha-backup-integration -- -m '${{ steps.select-tests.outputs.mark_expression }}'
         env:

actions.yaml

@@ -60,6 +60,6 @@ restore:
     backup-id:
       type: integer
       description: |
-        A backup-id to identify the backup to restore. Format: <backup-id, int>
+        A backup-id to identify the backup to restore. Format: backup-id=<integer>.
   required:
-    - backup-id
+    - backup-id

lib/charms/opensearch/v0/constants_charm.py

@@ -70,6 +70,8 @@
 HorizontalScaleUpSuggest = "Horizontal scale up advised: {} shards unassigned."
 WaitingForOtherUnitServiceOps = "Waiting for other units to complete the ops on their service."
 NewIndexRequested = "new index {index} requested"
+RestoreInProgress = "Restore in progress..."
+PluginConfigStart = "Plugin configuration started."
 
 
 # Relation Interfaces

lib/charms/opensearch/v0/helper_cluster.py

@@ -6,6 +6,7 @@
 from random import choice
 from typing import Dict, List, Optional
 
+from charms.opensearch.v0.helper_enums import BaseStrEnum
 from charms.opensearch.v0.models import Node
 from charms.opensearch.v0.opensearch_distro import OpenSearchDistribution
 from tenacity import retry, stop_after_attempt, wait_exponential
@@ -24,6 +25,13 @@
 logger = logging.getLogger(__name__)
 
 
+class IndexStateEnum(BaseStrEnum):
+    """Enum for index states."""
+
+    OPEN = "open"
+    CLOSED = "closed"
+
+
 class ClusterTopology:
     """Class for creating the best possible configuration for a Node."""
 
@@ -267,9 +275,9 @@ def indices(
         alt_hosts: Optional[List[str]] = None,
     ) -> List[Dict[str, str]]:
         """Get all shards of all indexes in the cluster."""
-        idx = opensearch.request("GET", "/_cat/indices", host=host, alt_hosts=alt_hosts)
+        endpoint = "/_cat/indices?expand_wildcards=all"
         idx = {}
-        for index in opensearch.request("GET", "/_cat/indices", host=host, alt_hosts=alt_hosts):
+        for index in opensearch.request("GET", endpoint, host=host, alt_hosts=alt_hosts):
             idx[index["index"]] = {"health": index["health"], "status": index["status"]}
         return idx