Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Restore in different kubernetes cluster fails from s3 backup source #1856

Open
mahdiG opened this issue Oct 30, 2024 · 0 comments
Open

Restore in different kubernetes cluster fails from s3 backup source #1856

mahdiG opened this issue Oct 30, 2024 · 0 comments
Labels
bug

Comments

@mahdiG
Copy link

mahdiG commented Oct 30, 2024 •
edited
Loading

Report

Hi. I've successfully done backup to s3 using the Backup CR. The backup restores fine if I set backupName in Restore CR but if I try to restore it with setting s3 in backupSource field in the restore CR, it creates the restore job but the job fails.

More about the problem

Here's what I did. I installed the operator version 1.15.1 with helm chart.
Helm values:

# Default values for pxc-operator.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

replicaCount: 1

operatorImageRepository: percona/percona-xtradb-cluster-operator
imagePullPolicy: IfNotPresent
image: ""

# set if you want to specify a namespace to watch
# defaults to `.Release.namespace` if left blank
# watchNamespace:

# set if operator should be deployed in cluster wide mode. defaults to false
watchAllNamespaces: true

# rbac: settings for deployer RBAC creation
rbac:
  # rbac.create: if false RBAC resources should be in place
  create: true

# serviceAccount: settings for Service Accounts used by the deployer
serviceAccount:
  # serviceAccount.create: Whether to create the Service Accounts or not
  create: true

# set if you want to use a different operator name
# defaults to `percona-xtradb-cluster-operator`
# operatorName:

imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""

resources:
  # We usually recommend not to specify default resources and to leave this as a conscious
  # choice for the user. This also increases chances charts run on environments with little
  # resources, such as Minikube. If you don't want to specify resources, comment the following
  # lines and add the curly braces after 'resources:'.
  limits:
    cpu: 1
    memory: 2Gi
  requests:
    cpu: 100m
    memory: 20Mi

containerSecurityContext: {}

nodeSelector: {}

tolerations: []

affinity: {}

podAnnotations: {}

logStructured: false
logLevel: "INFO"
disableTelemetry: true

extraEnvVars: []
# - name: http_proxy
#   value: "example-proxy-http"
# - name: https_proxy
#   value: "example-proxy-https"

Here's the operator deployment created:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: percona-operator-pxc-operator
  namespace: percona-operator
  uid: 09470be8-de17-4401-8f08-35e2d92639ab
  resourceVersion: '669410210'
  generation: 1
  creationTimestamp: '2024-10-30T13:54:32Z'
  labels:
    app.kubernetes.io/instance: percona-operator
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: pxc-operator
    app.kubernetes.io/version: 1.15.1
    helm.sh/chart: pxc-operator-1.15.1
  annotations:
    deployment.kubernetes.io/revision: '1'
    meta.helm.sh/release-name: percona-operator
    meta.helm.sh/release-namespace: percona-operator
  managedFields:
    - manager: helm
      operation: Update
      apiVersion: apps/v1
      time: '2024-10-30T13:54:32Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:annotations:
            .: {}
            f:meta.helm.sh/release-name: {}
            f:meta.helm.sh/release-namespace: {}
          f:labels:
            .: {}
            f:app.kubernetes.io/instance: {}
            f:app.kubernetes.io/managed-by: {}
            f:app.kubernetes.io/name: {}
            f:app.kubernetes.io/version: {}
            f:helm.sh/chart: {}
        f:spec:
          f:progressDeadlineSeconds: {}
          f:replicas: {}
          f:revisionHistoryLimit: {}
          f:selector: {}
          f:strategy:
            f:rollingUpdate:
              .: {}
              f:maxSurge: {}
              f:maxUnavailable: {}
            f:type: {}
          f:template:
            f:metadata:
              f:labels:
                .: {}
                f:app.kubernetes.io/component: {}
                f:app.kubernetes.io/instance: {}
                f:app.kubernetes.io/name: {}
                f:app.kubernetes.io/part-of: {}
            f:spec:
              f:containers:
                k:{"name":"percona-xtradb-cluster-operator"}:
                  .: {}
                  f:command: {}
                  f:env:
                    .: {}
                    k:{"name":"DISABLE_TELEMETRY"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"LOG_LEVEL"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"LOG_STRUCTURED"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"OPERATOR_NAME"}:
                      .: {}
                      f:name: {}
                      f:value: {}
                    k:{"name":"POD_NAME"}:
                      .: {}
                      f:name: {}
                      f:valueFrom:
                        .: {}
                        f:fieldRef: {}
                    k:{"name":"WATCH_NAMESPACE"}:
                      .: {}
                      f:name: {}
                  f:image: {}
                  f:imagePullPolicy: {}
                  f:livenessProbe:
                    .: {}
                    f:failureThreshold: {}
                    f:httpGet:
                      .: {}
                      f:path: {}
                      f:port: {}
                      f:scheme: {}
                    f:periodSeconds: {}
                    f:successThreshold: {}
                    f:timeoutSeconds: {}
                  f:name: {}
                  f:ports:
                    .: {}
                    k:{"containerPort":8080,"protocol":"TCP"}:
                      .: {}
                      f:containerPort: {}
                      f:name: {}
                      f:protocol: {}
                  f:resources:
                    .: {}
                    f:limits:
                      .: {}
                      f:cpu: {}
                      f:memory: {}
                    f:requests:
                      .: {}
                      f:cpu: {}
                      f:memory: {}
                  f:terminationMessagePath: {}
                  f:terminationMessagePolicy: {}
              f:dnsPolicy: {}
              f:restartPolicy: {}
              f:schedulerName: {}
              f:securityContext: {}
              f:serviceAccount: {}
              f:serviceAccountName: {}
              f:terminationGracePeriodSeconds: {}
    - manager: kube-controller-manager
      operation: Update
      apiVersion: apps/v1
      time: '2024-10-30T13:54:35Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:annotations:
            f:deployment.kubernetes.io/revision: {}
        f:status:
          f:availableReplicas: {}
          f:conditions:
            .: {}
            k:{"type":"Available"}:
              .: {}
              f:lastTransitionTime: {}
              f:lastUpdateTime: {}
              f:message: {}
              f:reason: {}
              f:status: {}
              f:type: {}
            k:{"type":"Progressing"}:
              .: {}
              f:lastTransitionTime: {}
              f:lastUpdateTime: {}
              f:message: {}
              f:reason: {}
              f:status: {}
              f:type: {}
          f:observedGeneration: {}
          f:readyReplicas: {}
          f:replicas: {}
          f:updatedReplicas: {}
      subresource: status
  selfLink: >-
    /apis/apps/v1/namespaces/percona-operator/deployments/percona-operator-pxc-operator
status:
  observedGeneration: 1
  replicas: 1
  updatedReplicas: 1
  readyReplicas: 1
  availableReplicas: 1
  conditions:
    - type: Available
      status: 'True'
      lastUpdateTime: '2024-10-30T13:54:32Z'
      lastTransitionTime: '2024-10-30T13:54:32Z'
      reason: MinimumReplicasAvailable
      message: Deployment has minimum availability.
    - type: Progressing
      status: 'True'
      lastUpdateTime: '2024-10-30T13:54:35Z'
      lastTransitionTime: '2024-10-30T13:54:32Z'
      reason: NewReplicaSetAvailable
      message: >-
        ReplicaSet "percona-operator-pxc-operator-5d5495b695" has successfully
        progressed.
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/component: operator
      app.kubernetes.io/instance: percona-operator
      app.kubernetes.io/name: pxc-operator
      app.kubernetes.io/part-of: pxc-operator
  template:
    metadata:
      creationTimestamp: null
      labels:
        app.kubernetes.io/component: operator
        app.kubernetes.io/instance: percona-operator
        app.kubernetes.io/name: pxc-operator
        app.kubernetes.io/part-of: pxc-operator
    spec:
      containers:
        - name: percona-xtradb-cluster-operator
          image: percona/percona-xtradb-cluster-operator:1.15.1
          command:
            - percona-xtradb-cluster-operator
          ports:
            - name: metrics
              containerPort: 8080
              protocol: TCP
          env:
            - name: WATCH_NAMESPACE
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.name
            - name: OPERATOR_NAME
              value: percona-operator-pxc-operator
            - name: LOG_STRUCTURED
              value: 'false'
            - name: LOG_LEVEL
              value: INFO
            - name: DISABLE_TELEMETRY
              value: 'true'
          resources:
            limits:
              cpu: '1'
              memory: 2Gi
            requests:
              cpu: 100m
              memory: 20Mi
          livenessProbe:
            httpGet:
              path: /metrics
              port: metrics
              scheme: HTTP
            timeoutSeconds: 1
            periodSeconds: 10
            successThreshold: 1
            failureThreshold: 3
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: IfNotPresent
      restartPolicy: Always
      terminationGracePeriodSeconds: 600
      dnsPolicy: ClusterFirst
      serviceAccountName: percona-operator-pxc-operator
      serviceAccount: percona-operator-pxc-operator
      securityContext: {}
      schedulerName: default-scheduler
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
      maxSurge: 25%
  revisionHistoryLimit: 10
  progressDeadlineSeconds: 600

Then I created a cluster with kubectl:

created pxc secrets:

apiVersion: v1
kind: Secret
metadata:
  name: pxc-secrets
  namespace: testtt
type: Opaque
stringData:
  root: kjhsgaejwoifapweihaowgej
  xtrabackup: kjhsgaejwoifapweihaowgej
  monitor: kjhsgaejwoifapweihaowgej
  proxyadmin: kjhsgaejwoifapweihaowgej
  #  pmmserverkey: my_pmm_server_key
  operator: kjhsgaejwoifapweihaowgej
  replication: kjhsgaejwoifapweihaowgej
  AWS_ACCESS_KEY_ID: AAAAAAA
  AWS_SECRET_ACCESS_KEY: BBBBBBBBBB

created xtradb cluster CR:

apiVersion: pxc.percona.com/v1
kind: PerconaXtraDBCluster
metadata:
  name: cluster1
  namespace: testtt
spec:
  crVersion: 1.15.1
  enableVolumeExpansion: true
  secretsName: pxc-secrets
  tls:
    enabled: true
  unsafeFlags:
    pxcSize: true
    proxySize: true
  pause: false
  updateStrategy: SmartUpdate
  upgradeOptions:
    apply: disabled
  pxc:
    size: 1
    image: percona/percona-xtradb-cluster:8.0.36-28.1
    imagePullPolicy: IfNotPresent
    autoRecovery: true
    configuration: |
      [mysqld]
      binlog_expire_logs_seconds=240000
    labels:
      mahdi: rocks
    annotations:
      "restarted-by-mahdi": "1"
    resources:
      requests:
        memory: 1G
        cpu: 600m
      limits:
        memory: 2G
        cpu: "1"
    affinity:
      advanced:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 1
              podAffinityTerm:
                topologyKey: kubernetes.io/hostname
                labelSelector:
                  matchExpressions:
                    - key: mahdi
                      operator: In
                      values:
                        - rocks
    podDisruptionBudget:
      maxUnavailable: 1
    volumeSpec:
      persistentVolumeClaim:
        storageClassName: openebs-lvm
        resources:
          requests:
            storage: 6G
    gracePeriod: 600
  haproxy:
    enabled: true
    size: 1
    image: percona/haproxy:2.8.5
    imagePullPolicy: IfNotPresent
    resources:
      requests:
        memory: 1G
        cpu: 600m
    affinity:
      antiAffinityTopologyKey: "kubernetes.io/hostname"
    podDisruptionBudget:
      maxUnavailable: 1
    gracePeriod: 30
  proxysql:
    enabled: false
  logcollector:
    enabled: false
  pmm:
    enabled: false
  backup:
    image: percona/percona-xtradb-cluster-operator:1.15.1-pxc8.0-backup-pxb8.0.35
    backoffLimit: 1
    pitr:
      enabled: false
    storages:
      lex-ir-s3:
        type: s3
        verifyTLS: true
        s3:
          bucket: backups-khuhtziv/my-pxc-backup
          credentialsSecret: pxc-secrets
          endpointUrl: https://s3.lexoya.com

cluster-1-pxc-0 and cluster-1-haproxy-0 pods become running after a few minutes

Then I port-forward the haproxy service and connect to the database with DbGate and create a database, a table and insert a record in it. It all works fine and database now has data.

Now I create a backup CR with this yaml:

apiVersion: pxc.percona.com/v1
kind: PerconaXtraDBClusterBackup
metadata:
  #  finalizers:
  #    - percona.com/delete-backup
  name: backup1
  namespace: testtt
spec:
  pxcCluster: cluster1
  storageName: lex-ir-s3

backup job is created and status becomes Succeeded.
backup job logs:

+ LIB_PATH=/usr/lib/pxc
+ . /usr/lib/pxc/backup.sh
++ set -o errexit
++ SST_INFO_NAME=sst_info
++ XBCLOUD_ARGS='--curl-retriable-errors=7 '
++ INSECURE_ARG=
++ '[' -n true ']'
++ [[ true == \f\a\l\s\e ]]
++ S3_BUCKET_PATH=my-pxc-backup/cluster1-2024-10-30-14:11:23-full
+++ date +%F-%H-%M
++ BACKUP_PATH=cluster1-pxc-2024-10-30-14-11-xtrabackup.stream
+ GARBD_OPTS=
+ check_ssl
+ CA=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
+ '[' -f /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt ']'
+ SSL_DIR=/etc/mysql/ssl
+ '[' -f /etc/mysql/ssl/ca.crt ']'
+ CA=/etc/mysql/ssl/ca.crt
+ SSL_INTERNAL_DIR=/etc/mysql/ssl-internal
+ '[' -f /etc/mysql/ssl-internal/ca.crt ']'
+ CA=/etc/mysql/ssl-internal/ca.crt
+ KEY=/etc/mysql/ssl/tls.key
+ CERT=/etc/mysql/ssl/tls.crt
+ '[' -f /etc/mysql/ssl-internal/tls.key -a -f /etc/mysql/ssl-internal/tls.crt ']'
+ KEY=/etc/mysql/ssl-internal/tls.key
+ CERT=/etc/mysql/ssl-internal/tls.crt
+ '[' -f /etc/mysql/ssl-internal/ca.crt -a -f /etc/mysql/ssl-internal/tls.key -a -f /etc/mysql/ssl-internal/tls.crt ']'
+ GARBD_OPTS='socket.ssl_ca=/etc/mysql/ssl-internal/ca.crt;socket.ssl_cert=/etc/mysql/ssl-internal/tls.crt;socket.ssl_key=/etc/mysql/ssl-internal/tls.key;socket.ssl_cipher=;pc.weight=0;'
+ '[' -n backups-khuhtziv ']'
+ clean_backup_s3
+ mc_add_bucket_dest
+ echo '+ mc -C /tmp/mc  config host add dest https://s3.lexoya.com ACCESS_KEY_ID SECRET_ACCESS_KEY '
+ mc -C /tmp/mc  config host add dest https://s3.lexoya.com ACCESS_KEY_ID SECRET_ACCESS_KEY 
Added `dest` successfully.
+ is_object_exist backups-khuhtziv my-pxc-backup/cluster1-2024-10-30-14:11:23-full.sst_info
+ local bucket=backups-khuhtziv
+ local object=my-pxc-backup/cluster1-2024-10-30-14:11:23-full.sst_info
++ mc -C /tmp/mc --json ls dest/backups-khuhtziv/my-pxc-backup/cluster1-2024-10-30-14:11:23-full.sst_info
++ jq .status
+ [[ -n '' ]]
+ is_object_exist backups-khuhtziv my-pxc-backup/cluster1-2024-10-30-14:11:23-full/
+ local bucket=backups-khuhtziv
+ local object=my-pxc-backup/cluster1-2024-10-30-14:11:23-full/
++ mc -C /tmp/mc --json ls dest/backups-khuhtziv/my-pxc-backup/cluster1-2024-10-30-14:11:23-full/
++ jq .status
+ [[ -n '' ]]
+ request_streaming
++ hostname -i
++ sed -E 's/.*\b([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})\b.*/\1/'
+ local LOCAL_IP=10.42.4.30
++ get_backup_source
+++ /opt/percona/peer-list -on-start=/usr/bin/get-pxc-state -service=cluster1-pxc
+++ grep wsrep_cluster_size
+++ tail -1
+++ sort
+++ cut -d : -f 12
++ CLUSTER_SIZE=1
++ '[' -z 1 ']'
+++ /opt/percona/peer-list -on-start=/usr/bin/get-pxc-state -service=cluster1-pxc
+++ grep wsrep_ready:ON:wsrep_connected:ON:wsrep_local_state_comment:Synced:wsrep_cluster_status:Primary
+++ sort -r
+++ tail -1
+++ cut -d : -f 2
+++ cut -d . -f 1
++ FIRST_NODE=cluster1-pxc-0
++ SKIP_FIRST_POD='|'
++ (( 1 > 1 ))
++ /opt/percona/peer-list -on-start=/usr/bin/get-pxc-state -service=cluster1-pxc
++ grep wsrep_ready:ON:wsrep_connected:ON:wsrep_local_state_comment:Synced:wsrep_cluster_status:Primary
++ grep -v '|'
++ sort
++ tail -1
++ cut -d : -f 2
++ cut -d . -f 1
+ local NODE_NAME=cluster1-pxc-0
+ '[' -z cluster1-pxc-0 ']'
+ set +o errexit
+ log INFO 'Garbd was started'
2024-10-30 14:11:33 [INFO] Garbd was started
+ garbd --address 'gcomm://cluster1-pxc-0.cluster1-pxc?gmcast.listen_addr=tcp://0.0.0.0:4567' --donor cluster1-pxc-0 --group cluster1-pxc --options 'socket.ssl_ca=/etc/mysql/ssl-internal/ca.crt;socket.ssl_cert=/etc/mysql/ssl-internal/tls.crt;socket.ssl_key=/etc/mysql/ssl-internal/tls.key;socket.ssl_cipher=;pc.weight=0;' --sst xtrabackup-v2:10.42.4.30:4444/xtrabackup_sst//1 --recv-script=/usr/bin/run_backup.sh
2024-10-30 14:11:33.292  INFO: CRC-32C: using 64-bit x86 acceleration.
2024-10-30 14:11:33.293  INFO: Read config: 
	daemon:      0
	name:        garb
	address:     gcomm://cluster1-pxc-0.cluster1-pxc?gmcast.listen_addr=tcp://0.0.0.0:4567
	group:       cluster1-pxc
	sst:         xtrabackup-v2:10.42.4.30:4444/xtrabackup_sst//1
	donor:       cluster1-pxc-0
	options:     socket.ssl_ca=/etc/mysql/ssl-internal/ca.crt;socket.ssl_cert=/etc/mysql/ssl-internal/tls.crt;socket.ssl_key=/etc/mysql/ssl-internal/tls.key;socket.ssl_cipher=;pc.weight=0;; gcs.fc_limit=9999999; gcs.fc_factor=1.0; gcs.fc_single_primary=yes; socket.ssl=YES
	cfg:         
	log:         
	recv_script: /usr/bin/run_backup.sh
	workdir: 

2024-10-30 14:11:33.293  WARN: SSL compression is not effective. The option socket.ssl_compression is deprecated and will be removed in future releases.
2024-10-30 14:11:33.293  WARN: Parameter 'socket.ssl_compression' is deprecated and will be removed in future versions
2024-10-30 14:11:33.308  INFO: protonet asio version 0
2024-10-30 14:11:33.314  INFO: Using CRC-32C for message checksums.
2024-10-30 14:11:33.314  INFO: backend: asio
2024-10-30 14:11:33.314  INFO: gcomm thread scheduling priority set to other:0 
2024-10-30 14:11:33.314  INFO: Fail to access the file (./gvwstate.dat) error (No such file or directory). It is possible if node is booting for first time or re-booting after a graceful shutdown
2024-10-30 14:11:33.315  INFO: Restoring primary-component from disk failed. Either node is booting for first time or re-booting after a graceful shutdown
2024-10-30 14:11:33.315  INFO: GMCast version 0
2024-10-30 14:11:33.319  INFO: (ddf3edf5-b92c, 'ssl://0.0.0.0:4567') listening at ssl://0.0.0.0:4567
2024-10-30 14:11:33.319  INFO: (ddf3edf5-b92c, 'ssl://0.0.0.0:4567') multicast: , ttl: 1
2024-10-30 14:11:33.320  INFO: EVS version 1
2024-10-30 14:11:33.320  INFO: gcomm: connecting to group 'cluster1-pxc', peer 'cluster1-pxc-0.cluster1-pxc:'
2024-10-30 14:11:33.343  INFO: (ddf3edf5-b92c, 'ssl://0.0.0.0:4567') connection established to d07962ff-b601 ssl://10.42.4.27:4567
2024-10-30 14:11:33.825  INFO: EVS version upgrade 0 -> 1
2024-10-30 14:11:33.825  INFO: declaring d07962ff-b601 at ssl://10.42.4.27:4567 stable
2024-10-30 14:11:33.825  INFO: PC protocol upgrade 0 -> 1
2024-10-30 14:11:33.826  INFO: Node d07962ff-b601 state primary
2024-10-30 14:11:33.832  INFO: Current view of cluster as seen by this node
view (view_id(PRIM,d07962ff-b601,2)
memb {
	d07962ff-b601,0
	ddf3edf5-b92c,0
	}
joined {
	}
left {
	}
partitioned {
	}
)
2024-10-30 14:11:33.832  INFO: Save the discovered primary-component to disk
2024-10-30 14:11:33.832  WARN: open file(./gvwstate.dat.tmp) failed(Permission denied)
2024-10-30 14:11:34.322  INFO: gcomm: connected
2024-10-30 14:11:34.322  INFO: Changing maximum packet size to 64500, resulting msg size: 32636
2024-10-30 14:11:34.323  INFO: Shifting CLOSED -> OPEN (TO: 0)
2024-10-30 14:11:34.323  INFO: Opened channel 'cluster1-pxc'
2024-10-30 14:11:34.323  INFO: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 2
2024-10-30 14:11:34.323  INFO: STATE EXCHANGE: Waiting for state UUID.
2024-10-30 14:11:34.323  INFO: STATE EXCHANGE: sent state msg: de42e06f-96c8-11ef-9d7b-53b819439918
2024-10-30 14:11:34.323  INFO: STATE EXCHANGE: got state msg: de42e06f-96c8-11ef-9d7b-53b819439918 from 0 (cluster1-pxc-0)
2024-10-30 14:11:34.324  INFO: STATE EXCHANGE: got state msg: de42e06f-96c8-11ef-9d7b-53b819439918 from 1 (garb)
2024-10-30 14:11:34.324  INFO: Quorum results:
	version    = 6,
	component  = PRIMARY,
	conf_id    = 1,
	members    = 1/2 (primary/total),
	act_id     = 23,
	last_appl. = 19,
	protocols  = 2/11/4 (gcs/repl/appl),
	vote policy= 0,
	group UUID = bab867c4-96c7-11ef-ae67-4303a26160b2
2024-10-30 14:11:34.324  INFO: Flow-control interval: [9999999, 9999999]
2024-10-30 14:11:34.324  INFO: Shifting OPEN -> PRIMARY (TO: 24)
2024-10-30 14:11:34.325  INFO: Sending state transfer request: 'xtrabackup-v2:10.42.4.30:4444/xtrabackup_sst//1', size: 47
2024-10-30 14:11:34.325  INFO: Member 1.0 (garb) requested state transfer from 'cluster1-pxc-0'. Selected 0.0 (cluster1-pxc-0)(SYNCED) as donor.
2024-10-30 14:11:34.325  INFO: Shifting PRIMARY -> JOINER (TO: 24)
2024-10-30 14:11:34.326  INFO: Starting SST script
2024-10-30 14:11:34.335  INFO: [SST script] + set -m

2024-10-30 14:11:34.335  INFO: [SST script] + LIB_PATH=/usr/lib/pxc

2024-10-30 14:11:34.335  INFO: [SST script] + . /usr/lib/pxc/vault.sh

2024-10-30 14:11:34.335  INFO: [SST script] ++ set -o errexit

2024-10-30 14:11:34.335  INFO: [SST script] ++ keyring_vault=/etc/mysql/vault-keyring-secret/keyring_vault.conf

2024-10-30 14:11:34.336  INFO: [SST script] + . /usr/lib/pxc/backup.sh

2024-10-30 14:11:34.336  INFO: [SST script] ++ set -o errexit

2024-10-30 14:11:34.336  INFO: [SST script] ++ SST_INFO_NAME=sst_info

2024-10-30 14:11:34.336  INFO: [SST script] ++ XBCLOUD_ARGS='--curl-retriable-errors=7 '

2024-10-30 14:11:34.336  INFO: [SST script] ++ INSECURE_ARG=

2024-10-30 14:11:34.336  INFO: [SST script] ++ '[' -n true ']'

2024-10-30 14:11:34.336  INFO: [SST script] ++ [[ true == \f\a\l\s\e ]]

2024-10-30 14:11:34.336  INFO: [SST script] ++ S3_BUCKET_PATH=my-pxc-backup/cluster1-2024-10-30-14:11:23-full

2024-10-30 14:11:34.338  INFO: [SST script] +++ date +%F-%H-%M

2024-10-30 14:11:34.343  INFO: [SST script] ++ BACKUP_PATH=cluster1-pxc-2024-10-30-14-11-xtrabackup.stream

2024-10-30 14:11:34.344  INFO: [SST script] + SOCAT_OPTS=TCP-LISTEN:4444,reuseaddr,retry=30

2024-10-30 14:11:34.344  INFO: [SST script] + FIRST_RECEIVED=0

2024-10-30 14:11:34.344  INFO: [SST script] + SST_FAILED=0

2024-10-30 14:11:34.346  INFO: [SST script] + check_ssl

2024-10-30 14:11:34.346  INFO: [SST script] + CA=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt

2024-10-30 14:11:34.346  INFO: [SST script] + '[' -f /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt ']'

2024-10-30 14:11:34.346  INFO: [SST script] + SSL_DIR=/etc/mysql/ssl

2024-10-30 14:11:34.346  INFO: [SST script] + '[' -f /etc/mysql/ssl/ca.crt ']'

2024-10-30 14:11:34.346  INFO: [SST script] + CA=/etc/mysql/ssl/ca.crt

2024-10-30 14:11:34.346  INFO: [SST script] + SSL_INTERNAL_DIR=/etc/mysql/ssl-internal

2024-10-30 14:11:34.346  INFO: [SST script] + '[' -f /etc/mysql/ssl-internal/ca.crt ']'

2024-10-30 14:11:34.346  INFO: [SST script] + CA=/etc/mysql/ssl-internal/ca.crt

2024-10-30 14:11:34.346  INFO: [SST script] + KEY=/etc/mysql/ssl/tls.key

2024-10-30 14:11:34.346  INFO: [SST script] + CERT=/etc/mysql/ssl/tls.crt

2024-10-30 14:11:34.346  INFO: [SST script] + '[' -f /etc/mysql/ssl-internal/tls.key -a -f /etc/mysql/ssl-internal/tls.crt ']'

2024-10-30 14:11:34.346  INFO: [SST script] + KEY=/etc/mysql/ssl-internal/tls.key

2024-10-30 14:11:34.346  INFO: [SST script] + CERT=/etc/mysql/ssl-internal/tls.crt

2024-10-30 14:11:34.346  INFO: [SST script] + '[' -f /etc/mysql/ssl-internal/ca.crt -a -f /etc/mysql/ssl-internal/tls.key -a -f /etc/mysql/ssl-internal/tls.crt ']'

2024-10-30 14:11:34.346  INFO: [SST script] + SOCAT_OPTS=openssl-listen:4444,reuseaddr,cert=/etc/mysql/ssl-internal/tls.crt,key=/etc/mysql/ssl-internal/tls.key,cafile=/etc/mysql/ssl-internal/ca.crt,verify=1,retry=30

2024-10-30 14:11:34.346  INFO: [SST script] + trap handle_sigterm 15

2024-10-30 14:11:34.346  INFO: [SST script] + '[' -n backups-khuhtziv ']'

2024-10-30 14:11:34.347  INFO: [SST script] + backup_s3

2024-10-30 14:11:34.347  INFO: [SST script] + mc_add_bucket_dest

2024-10-30 14:11:34.347  INFO: [SST script] + echo '+ mc -C /tmp/mc  config host add dest https://s3.lexoya.com ACCESS_KEY_ID SECRET_ACCESS_KEY '

2024-10-30 14:11:34.347  INFO: [SST script] + mc -C /tmp/mc  config host add dest https://s3.lexoya.com ACCESS_KEY_ID SECRET_ACCESS_KEY 

2024-10-30 14:11:34.595  INFO: [SST script] Added `dest` successfully.

2024-10-30 14:11:34.603  INFO: [SST script] + wait 217

2024-10-30 14:11:34.603  INFO: [SST script] + xbstream -x -C /tmp

2024-10-30 14:11:34.603  INFO: [SST script] + socat -u openssl-listen:4444,reuseaddr,cert=/etc/mysql/ssl-internal/tls.crt,key=/etc/mysql/ssl-internal/tls.key,cafile=/etc/mysql/ssl-internal/ca.crt,verify=1,retry=30 stdio

2024-10-30 14:11:36.823  INFO: (ddf3edf5-b92c, 'ssl://0.0.0.0:4567') turning message relay requesting off
2024-10-30 14:11:37.443  INFO: [SST script] + log INFO 'Socat was started'

2024-10-30 14:11:37.451  INFO: [SST script] 2024-10-30 14:11:37 [INFO] Socat was started

2024-10-30 14:11:37.451  INFO: [SST script] + FIRST_RECEIVED=1

2024-10-30 14:11:37.451  INFO: [SST script] + [[ 0 -ne 0 ]]

2024-10-30 14:11:37.451  INFO: [SST script] + vault_store /tmp/sst_info

2024-10-30 14:11:37.451  INFO: [SST script] + local sst_info=/tmp/sst_info

2024-10-30 14:11:37.451  INFO: [SST script] + '[' '!' -f /etc/mysql/vault-keyring-secret/keyring_vault.conf ']'

2024-10-30 14:11:37.452  INFO: [SST script] + echo 'vault configuration not found'

2024-10-30 14:11:37.452  INFO: [SST script] vault configuration not found

2024-10-30 14:11:37.452  INFO: [SST script] + return 0

2024-10-30 14:11:37.455  INFO: [SST script] + xbstream -C /tmp -c sst_info

2024-10-30 14:11:37.456  INFO: [SST script] + grep -v 'error: http request failed: Couldn'\''t resolve host name'

2024-10-30 14:11:37.456  INFO: [SST script] ++ grep -c processor /proc/cpuinfo

2024-10-30 14:11:37.461  INFO: [SST script] + xbcloud put --parallel=30 --storage=s3 --md5 --curl-retriable-errors=7 --s3-bucket=backups-khuhtziv my-pxc-backup/cluster1-2024-10-30-14:11:23-full.sst_info

2024-10-30 14:11:38.912  INFO: [SST script] 241030 14:11:38 xbcloud: Successfully connected.

2024-10-30 14:11:38.912  INFO: [SST script] 241030 14:11:38 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full.sst_info/sst_info.00000000000000000000, size: 145

2024-10-30 14:11:38.912  INFO: [SST script] 241030 14:11:38 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full.sst_info/sst_info.00000000000000000001, size: 22

2024-10-30 14:11:38.912  INFO: [SST script] 241030 14:11:38 xbcloud: Uploading md5

2024-10-30 14:11:38.912  INFO: [SST script] 241030 14:11:38 xbcloud: Upload failed: backup is incomplete.

2024-10-30 14:11:38.912  INFO: [SST script] Backup doesn't contain last file with prefix xtrabackup_tablespaces in the cloud storage

2024-10-30 14:11:38.914  INFO: [SST script] + (( 0 == 0 ))

2024-10-30 14:11:38.914  INFO: [SST script] + FIRST_RECEIVED=0

2024-10-30 14:11:38.916  INFO: [SST script] + socat -u openssl-listen:4444,reuseaddr,cert=/etc/mysql/ssl-internal/tls.crt,key=/etc/mysql/ssl-internal/tls.key,cafile=/etc/mysql/ssl-internal/ca.crt,verify=1,retry=30 stdio

2024-10-30 14:11:38.916  INFO: [SST script] + grep -v 'error: http request failed: Couldn'\''t resolve host name'

2024-10-30 14:11:38.917  INFO: [SST script] ++ grep -c processor /proc/cpuinfo

2024-10-30 14:11:38.922  INFO: [SST script] + xbcloud put --storage=s3 --parallel=30 --md5 --curl-retriable-errors=7 --s3-bucket=backups-khuhtziv my-pxc-backup/cluster1-2024-10-30-14:11:23-full

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:39 xbcloud: Successfully connected.

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:49 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/sys/sys_config.ibd.00000000000000000001, size: 32

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:49 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/wsrep_cluster_members.ibd.00000000000000000001, size: 45

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:49 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/wsrep_cluster.ibd.00000000000000000001, size: 37

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:49 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/wsrep_cluster_members.ibd.00000000000000000000, size: 114753

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:49 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/wsrep_streaming_log.ibd.00000000000000000000, size: 114751

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:49 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/wsrep_cluster.ibd.00000000000000000000, size: 114745

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:49 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/wsrep_streaming_log.ibd.00000000000000000001, size: 43

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:49 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/newdb/newTable.ibd.00000000000000000000, size: 114740

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:49 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/newdb/newTable.ibd.00000000000000000001, size: 32

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:49 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/sys/sys_config.ibd.00000000000000000000, size: 114740

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:49 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/ibdata1.00000000000000000000, size: 10485801

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:50 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/ibdata1.00000000000000000002, size: 21

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:50 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql.ibd.00000000000000000000, size: 10485803

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:50 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/ibdata1.00000000000000000001, size: 2097193

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:50 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/undo_002.00000000000000000002, size: 22

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:50 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/undo_002.00000000000000000000, size: 10485802

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:50 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/undo_001.00000000000000000002, size: 22

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:50 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/undo_001.00000000000000000000, size: 10485802

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:50 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/undo_002.00000000000000000001, size: 6291498

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:50 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql.ibd.00000000000000000004, size: 23

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:50 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql.ibd.00000000000000000003, size: 1048619

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:50 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql.ibd.00000000000000000001, size: 10485803

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:50 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/undo_001.00000000000000000001, size: 6291498

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql.ibd.00000000000000000002, size: 10485803

2024-10-30 14:11:51.119  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/metadata_locks_168.sdi.00000000000000000001, size: 55

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_grou_172.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_grou_172.sdi.00000000000000000000, size: 9587

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/data_lock_waits_170.sdi.00000000000000000001, size: 56

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/processlist_109.sdi.00000000000000000001, size: 52

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_129.sdi.00000000000000000000, size: 39008

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_145.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_129.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/variables_info_195.sdi.00000000000000000001, size: 55

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/metadata_locks_168.sdi.00000000000000000000, size: 11994

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/file_instances_102.sdi.00000000000000000001, size: 55

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_144.sdi.00000000000000000000, size: 16712

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_appl_180.sdi.00000000000000000000, size: 4700

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_appl_180.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/data_lock_waits_170.sdi.00000000000000000000, size: 14214

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_sum_101.sdi.00000000000000000000, size: 6918

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_147.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/file_instances_102.sdi.00000000000000000000, size: 7640

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/processlist_109.sdi.00000000000000000000, size: 10878

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/keyring_keys_161.sdi.00000000000000000000, size: 4758

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/threads_119.sdi.00000000000000000000, size: 23733

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_sum_99.sdi.00000000000000000000, size: 7763

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/log_status_183.sdi.00000000000000000000, size: 4376

2024-10-30 14:11:51.233  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_appl_176.sdi.00000000000000000000, size: 19978

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_appl_175.sdi.00000000000000000000, size: 4985

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/keyring_keys_161.sdi.00000000000000000001, size: 53

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_sum_99.sdi.00000000000000000001, size: 56

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_147.sdi.00000000000000000000, size: 15009

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_appl_176.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_sum_101.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/variables_info_195.sdi.00000000000000000000, size: 8057

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_his_94.sdi.00000000000000000000, size: 17482

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_his_94.sdi.00000000000000000001, size: 56

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/threads_119.sdi.00000000000000000001, size: 48

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_appl_175.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_grou_178.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_144.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/setup_threads_115.sdi.00000000000000000001, size: 54

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/table_io_waits_s_116.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_139.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/table_io_waits_s_116.sdi.00000000000000000000, size: 32923

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/log_status_183.sdi.00000000000000000001, size: 51

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_grou_178.sdi.00000000000000000000, size: 12455

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/memory_summary_b_165.sdi.00000000000000000000, size: 11815

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_140.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_145.sdi.00000000000000000000, size: 15856

2024-10-30 14:11:51.308  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/setup_threads_115.sdi.00000000000000000000, size: 6986

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/memory_summary_b_165.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_127.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_123.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_139.sdi.00000000000000000000, size: 7946

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_127.sdi.00000000000000000000, size: 6919

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/memory_summary_b_166.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_appl_179.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/malloc_stats_tot_200.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_133.sdi.00000000000000000000, size: 26331

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/memory_summary_b_166.sdi.00000000000000000000, size: 11786

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_126.sdi.00000000000000000000, size: 7766

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_136.sdi.00000000000000000000, size: 32818

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/user_defined_fun_197.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/file_summary_by__104.sdi.00000000000000000000, size: 25721

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_124.sdi.00000000000000000000, size: 8620

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_124.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_123.sdi.00000000000000000000, size: 7794

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_140.sdi.00000000000000000000, size: 21546

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_137.sdi.00000000000000000000, size: 31445

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_137.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/memory_summary_b_163.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.452  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_130.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_130.sdi.00000000000000000000, size: 38567

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/hosts_155.sdi.00000000000000000000, size: 5368

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/hosts_155.sdi.00000000000000000001, size: 46

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_appl_174.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/user_defined_fun_197.sdi.00000000000000000000, size: 5890

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/file_summary_by__104.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_errors_su_151.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/memory_summary_b_163.sdi.00000000000000000000, size: 12642

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_appl_179.sdi.00000000000000000000, size: 6829

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_appl_174.sdi.00000000000000000000, size: 8324

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/status_by_host_187.sdi.00000000000000000001, size: 55

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_132.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/socket_instances_156.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_hi_122.sdi.00000000000000000000, size: 11369

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/malloc_stats_tot_200.sdi.00000000000000000000, size: 5799

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_errors_su_151.sdi.00000000000000000000, size: 7793

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/mutex_instances_106.sdi.00000000000000000001, size: 56

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_133.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_132.sdi.00000000000000000000, size: 27187

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_sum_96.sdi.00000000000000000001, size: 56

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_126.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_136.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.510  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_sum_96.sdi.00000000000000000000, size: 7764

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/status_by_host_187.sdi.00000000000000000000, size: 4162

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/socket_instances_156.sdi.00000000000000000000, size: 9209

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_hi_122.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/mutex_instances_106.sdi.00000000000000000000, size: 5234

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/keyring_componen_202.sdi.00000000000000000000, size: 9608

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/setup_objects_114.sdi.00000000000000000001, size: 54

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/setup_objects_114.sdi.00000000000000000000, size: 6502

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/host_cache_105.sdi.00000000000000000000, size: 25403

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_errors_su_149.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_asyn_181.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_errors_su_148.sdi.00000000000000000000, size: 7762

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/keyring_componen_202.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_sum_98.sdi.00000000000000000000, size: 7792

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/pxc_cluster_view_203.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_appl_177.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_asyn_181.sdi.00000000000000000000, size: 7342

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_errors_su_150.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_143.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_appl_177.sdi.00000000000000000000, size: 27459

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/table_handles_167.sdi.00000000000000000001, size: 54

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/global_variables_193.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.625  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/global_variables_193.sdi.00000000000000000000, size: 3321

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_errors_su_148.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_143.sdi.00000000000000000000, size: 15885

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_errors_su_150.sdi.00000000000000000000, size: 8617

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_sum_97.sdi.00000000000000000001, size: 56

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_errors_su_149.sdi.00000000000000000000, size: 7763

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_sum_98.sdi.00000000000000000001, size: 56

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/file_summary_by__103.sdi.00000000000000000000, size: 20566

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/file_summary_by__103.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/host_cache_105.sdi.00000000000000000001, size: 51

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/cond_instances_91.sdi.00000000000000000000, size: 4051

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_sum_97.sdi.00000000000000000000, size: 8094

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_142.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/objects_summary__107.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/user_variables_b_185.sdi.00000000000000000000, size: 4199

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/objects_summary__107.sdi.00000000000000000000, size: 7921

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/session_account__160.sdi.00000000000000000000, size: 4802

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/user_variables_b_185.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/binary_log_trans_198.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/table_handles_167.sdi.00000000000000000000, size: 9745

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/malloc_stats_201.sdi.00000000000000000001, size: 53

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/pxc_cluster_view_203.sdi.00000000000000000000, size: 5831

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/binary_log_trans_198.sdi.00000000000000000000, size: 13298

2024-10-30 14:11:51.735  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_131.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/malloc_stats_201.sdi.00000000000000000000, size: 5014

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/prepared_stateme_184.sdi.00000000000000000000, size: 35609

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/session_connect__159.sdi.00000000000000000000, size: 4794

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/session_connect__159.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/setup_instrument_113.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/cond_instances_91.sdi.00000000000000000001, size: 54

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/session_account__160.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/prepared_stateme_184.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_142.sdi.00000000000000000000, size: 21105

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_128.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/performance_time_108.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/performance_time_108.sdi.00000000000000000000, size: 4392

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/session_variable_194.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_131.sdi.00000000000000000000, size: 26360

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/session_variable_194.sdi.00000000000000000000, size: 3322

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_128.sdi.00000000000000000000, size: 39008

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/setup_instrument_113.sdi.00000000000000000000, size: 8036

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/socket_summary_b_158.sdi.00000000000000000000, size: 20584

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/error_log_92.sdi.00000000000000000001, size: 49

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/error_log_92.sdi.00000000000000000000, size: 7730

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/status_by_user_189.sdi.00000000000000000000, size: 4161

2024-10-30 14:11:51.850  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/socket_summary_b_158.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_conn_171.sdi.00000000000000000000, size: 46320

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_conn_171.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/status_by_user_189.sdi.00000000000000000001, size: 55

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/users_153.sdi.00000000000000000000, size: 5367

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_cur_93.sdi.00000000000000000001, size: 56

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_138.sdi.00000000000000000000, size: 6224

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_138.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/accounts_154.sdi.00000000000000000000, size: 6225

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/session_status_191.sdi.00000000000000000001, size: 55

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/users_153.sdi.00000000000000000001, size: 46

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/tls_channel_stat_199.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_141.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/session_status_191.sdi.00000000000000000000, size: 3316

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/global_status_190.sdi.00000000000000000000, size: 3314

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/tls_channel_stat_199.sdi.00000000000000000000, size: 15992

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_135.sdi.00000000000000000000, size: 25484

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/global_status_190.sdi.00000000000000000001, size: 54

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/persisted_variab_196.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_cur_93.sdi.00000000000000000000, size: 17482

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/data_locks_169.sdi.00000000000000000001, size: 51

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_141.sdi.00000000000000000000, size: 21546

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/data_locks_169.sdi.00000000000000000000, size: 16203

2024-10-30 14:11:51.896  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/accounts_154.sdi.00000000000000000001, size: 49

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/persisted_variab_196.sdi.00000000000000000000, size: 3324

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_cu_120.sdi.00000000000000000000, size: 11810

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_135.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_cu_120.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/status_by_thread_188.sdi.00000000000000000000, size: 4192

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_hi_121.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_hi_121.sdi.00000000000000000000, size: 11810

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_his_95.sdi.00000000000000000000, size: 17041

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/memory_summary_b_164.sdi.00000000000000000000, size: 11787

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_125.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/setup_consumers_112.sdi.00000000000000000001, size: 56

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/socket_summary_b_157.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/status_by_accoun_186.sdi.00000000000000000000, size: 5018

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/table_lock_waits_118.sdi.00000000000000000000, size: 56730

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/variables_by_thr_192.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/setup_consumers_112.sdi.00000000000000000000, size: 3359

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/memory_summary_b_164.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/socket_summary_b_157.sdi.00000000000000000000, size: 21769

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/status_by_thread_188.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/variables_by_thr_192.sdi.00000000000000000000, size: 4195

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_his_95.sdi.00000000000000000001, size: 56

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_134.sdi.00000000000000000001, size: 57

2024-10-30 14:11:51.966  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_conn_173.sdi.00000000000000000001, size: 57

2024-10-30 14:11:52.008  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/status_by_accoun_186.sdi.00000000000000000001, size: 57

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_125.sdi.00000000000000000000, size: 7765

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_conn_173.sdi.00000000000000000000, size: 24447

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_asyn_182.sdi.00000000000000000000, size: 5416

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/memory_summary_g_162.sdi.00000000000000000001, size: 57

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/table_io_waits_s_117.sdi.00000000000000000000, size: 32057

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_asyn_182.sdi.00000000000000000001, size: 57

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_134.sdi.00000000000000000000, size: 26332

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_errors_su_152.sdi.00000000000000000000, size: 6921

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_errors_su_152.sdi.00000000000000000001, size: 57

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/table_lock_waits_118.sdi.00000000000000000001, size: 57

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/memory_summary_g_162.sdi.00000000000000000000, size: 10939

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/setup_actors_111.sdi.00000000000000000001, size: 53

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_146.sdi.00000000000000000001, size: 57

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/slow_log_226.sdi.00000000000000000000, size: 12583

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_sum_100.sdi.00000000000000000000, size: 8619

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/setup_actors_111.sdi.00000000000000000000, size: 6254

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/rwlock_instances_110.sdi.00000000000000000000, size: 6050

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:51 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/table_io_waits_s_117.sdi.00000000000000000001, size: 57

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/rwlock_instances_110.sdi.00000000000000000001, size: 57

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/general_log.CSV.00000000000000000000, size: 35

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_146.sdi.00000000000000000000, size: 15857

2024-10-30 14:11:52.009  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/general_log.CSM.00000000000000000000, size: 90

2024-10-30 14:11:52.732  INFO: 0.0 (cluster1-pxc-0): State transfer to 1.0 (garb) complete.
2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_sum_100.sdi.00000000000000000001, size: 57

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/slow_log.CSM.00000000000000000000, size: 87

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/slow_log.CSV.00000000000000000000, size: 32

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/general_log.CSM.00000000000000000001, size: 35

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/binlog.000003.00000000000000000000, size: 244

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/general_log_225.sdi.00000000000000000001, size: 39

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/general_log_225.sdi.00000000000000000000, size: 6053

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/slow_log_226.sdi.00000000000000000001, size: 36

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/binlog.index.00000000000000000001, size: 26

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/backup-my.cnf.00000000000000000000, size: 501

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_binlog_info.00000000000000000001, size: 36

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/binlog.index.00000000000000000000, size: 62

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/ib_buffer_pool.00000000000000000000, size: 7376

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_info.00000000000000000000, size: 851

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_info.00000000000000000001, size: 29

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_logfile.00000000000000000000, size: 2612

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_binlog_info.00000000000000000000, size: 115

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_tablespaces.00000000000000000000, size: 95

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_checkpoints.00000000000000000000, size: 190

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/slow_log.CSM.00000000000000000001, size: 32

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/ib_buffer_pool.00000000000000000001, size: 28

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/binlog.000003.00000000000000000001, size: 27

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_checkpoints.00000000000000000001, size: 36

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_tablespaces.00000000000000000001, size: 36

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_logfile.00000000000000000001, size: 32

2024-10-30 14:11:52.740  INFO: [SST script] 241030 14:11:52 xbcloud: [0] successfully uploaded chunk: my-pxc-backup/cluster1-2024-10-30-14:11:23-full/backup-my.cnf.00000000000000000001, size: 27

2024-10-30 14:11:52.790  INFO: Member 0.0 (cluster1-pxc-0) synced with group.
2024-10-30 14:11:52.797  INFO: [SST script] 241030 14:11:52 xbcloud: Uploading md5

2024-10-30 14:11:52.797  INFO: [SST script] 241030 14:11:52 xbcloud: Upload completed.

2024-10-30 14:11:52.798  INFO: [SST script] + FIRST_RECEIVED=1

2024-10-30 14:11:52.799  INFO: [SST script] + mc -C /tmp/mc stat dest/backups-khuhtziv/my-pxc-backup/cluster1-2024-10-30-14:11:23-full.md5

2024-10-30 14:11:52.995  INFO: [SST script] Name      : cluster1-2024-10-30-14:11:23-full.md5

2024-10-30 14:11:52.996  INFO: [SST script] Date      : 2024-10-30 14:11:52 UTC 

2024-10-30 14:11:52.996  INFO: [SST script] Size      : 25 KiB 

2024-10-30 14:11:52.996  INFO: [SST script] ETag      : 6470a491c4dfdd64436940963b15d8a5 

2024-10-30 14:11:52.996  INFO: [SST script] Type      : file 

2024-10-30 14:11:52.996  INFO: [SST script] Expires   : 2024-11-29 14:11:52 UTC 

2024-10-30 14:11:52.996  INFO: [SST script] Metadata  :

2024-10-30 14:11:52.996  INFO: [SST script]   Cache-Control: max-age=2592000 

2024-10-30 14:11:52.996  INFO: [SST script]   Content-Type : application/octet-stream 

2024-10-30 14:11:52.996  INFO: [SST script] 

2024-10-30 14:11:53.006  INFO: [SST script] ++ mc -C /tmp/mc stat --json dest/backups-khuhtziv/my-pxc-backup/cluster1-2024-10-30-14:11:23-full.md5

2024-10-30 14:11:53.006  INFO: [SST script] ++ sed -e 's/.*"size":\([0-9]*\).*/\1/'

2024-10-30 14:11:53.208  INFO: [SST script] + md5_size=25913

2024-10-30 14:11:53.208  INFO: [SST script] + [[ 25913 =~ Object does not exist ]]

2024-10-30 14:11:53.208  INFO: [SST script] + (( md5_size < 23000 ))

2024-10-30 14:11:53.209  INFO: [SST script] + (( 0 == 0 ))

2024-10-30 14:11:53.209  INFO: [SST script] + touch /tmp/backup-is-completed

2024-10-30 14:11:53.214  INFO: [SST script] + exit 0

2024-10-30 14:11:53.215  INFO: SST script ended
2024-10-30 14:11:53.215  INFO: Closing send monitor...
2024-10-30 14:11:53.215  INFO: Closed send monitor.
2024-10-30 14:11:53.215  INFO: gcomm: terminating thread
2024-10-30 14:11:53.215  INFO: gcomm: joining thread
2024-10-30 14:11:53.215  INFO: gcomm: closing backend
2024-10-30 14:11:53.332  INFO: Donor no longer in donor state, interrupting script
2024-10-30 14:11:54.220  INFO: Current view of cluster as seen by this node
view (view_id(NON_PRIM,d07962ff-b601,2)
memb {
	ddf3edf5-b92c,0
	}
joined {
	}
left {
	}
partitioned {
	d07962ff-b601,0
	}
)
2024-10-30 14:11:54.220  INFO: PC protocol downgrade 1 -> 0
2024-10-30 14:11:54.220  INFO: Current view of cluster as seen by this node
view ((empty))
2024-10-30 14:11:54.220  INFO: gcomm: closed
2024-10-30 14:11:54.220  INFO: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
2024-10-30 14:11:54.220  INFO: Flow-control interval: [9999999, 9999999]
2024-10-30 14:11:54.220  INFO: Received NON-PRIMARY.
2024-10-30 14:11:54.220  INFO: Shifting JOINER -> OPEN (TO: 24)
2024-10-30 14:11:54.220  INFO: New SELF-LEAVE.
2024-10-30 14:11:54.220  INFO: Flow-control interval: [9999999, 9999999]
2024-10-30 14:11:54.220  INFO: Received SELF-LEAVE. Closing connection.
2024-10-30 14:11:54.220  INFO: Shifting OPEN -> CLOSED (TO: 24)
2024-10-30 14:11:54.220  INFO: RECV thread exiting 0: Success
2024-10-30 14:11:54.220  INFO: SST script already terminated
2024-10-30 14:11:54.220  INFO: recv_thread() joined.
2024-10-30 14:11:54.220  INFO: Closing replication queue.
2024-10-30 14:11:54.220  INFO: Closing slave action queue.
2024-10-30 14:11:54.221  INFO: Exiting main loop
+ EXID_CODE=0
+ '[' -f /tmp/backup-is-completed ']'
+ log INFO 'Backup was finished successfully'
Terminating process2024-10-30 14:11:54 [INFO] Backup was finished successfully
+ exit 0

I can view the backup files in s3.

Now I want to create a new cluster (cluster2) and restore the backup to it. I use the same CR yaml as cluster1 but only change metadata.name to cluster2 and apply.
Cluster2 pods become Running.

Now I want to restore data to it and here's when the problems happen.
I followed this official doc: https://docs.percona.com/percona-operator-for-mysql/pxc/backups-restore-to-new-cluster.html

According to the doc, I needed to find the 'name' of the pxc backup. I ran this command to show my pxc-backup:

mahdi@sajjad-debian12:~/code/xtradb-test$ lexkube -n testtt get pxc-backups
NAME      CLUSTER    STORAGE     DESTINATION                                                             STATUS      COMPLETED   AGE
backup1   cluster1   lex-ir-s3   s3://backups-khuhtziv/my-pxc-backup/cluster1-2024-10-30-14:11:23-full   Succeeded   8m48s       9m22s

The NAME is backup1.
I created a Restore CR:

apiVersion: pxc.percona.com/v1
kind: PerconaXtraDBClusterRestore
metadata:
  name: restore-to-cluster2
  namespace: testtt
spec:
  pxcCluster: cluster2
  backupSource:
    destination: s3://backups-khuhtziv/backup1
    storageName: lex-ir-s3
    verifyTLS: true
    s3:
      bucket: backups-khuhtziv
      credentialsSecret: pxc-secrets
      endpointUrl: https://s3.lexoya.com
      region: default

The operator showed error and did not create the restore job

2024-10-30T14:23:56.687Z	ERROR	Reconciler error	{"controller": "pxcrestore-controller", "namespace": "testtt", "name": "restore-to-cluster2", "reconcileID": "c049b4bf-4b13-40a1-a201-8a467c47aa5a", "error": "failed to validate restore job: failed to validate backup existence: backup not found", "errorVerbose": "backup not found\ngithub.com/percona/percona-xtradb-cluster-operator/pkg/controller/pxcrestore.(*s3).Validate\n\t/go/src/github.com/percona/percona-xtradb-cluster-operator/pkg/controller/pxcrestore/restorer.go:73

It didn't make sense to me to put the backup "name" in destination either. I tried putting the destination from kubectl get pxc-backups into the destination field but got the same error.

Then I put what I thought I should put there and this CR finally created a backup job:
restore CR:

apiVersion: pxc.percona.com/v1
kind: PerconaXtraDBClusterRestore
metadata:
  name: restore-to-cluster2
  namespace: testtt
spec:
  pxcCluster: cluster2
  backupSource:
    destination: s3://backups-khuhtziv/my-pxc-backup
    storageName: lex-ir-s3
    verifyTLS: true
    s3:
      bucket: backups-khuhtziv
      credentialsSecret: pxc-secrets
      endpointUrl: https://s3.lexoya.com
      region: default

This creates the job but it fails after a few seconds. restore job logs:

+ LIB_PATH=/usr/lib/pxc
+ . /usr/lib/pxc/check-version.sh
+ . /usr/lib/pxc/vault.sh
++ set -o errexit
++ keyring_vault=/etc/mysql/vault-keyring-secret/keyring_vault.conf
+ XBCLOUD_ARGS='--curl-retriable-errors=7 '
+ MC_ARGS='-C /tmp/mc'
+ '[' -n true ']'
+ [[ true == \f\a\l\s\e ]]
+ '[' -n backups-khuhtziv/my-pxc-backup ']'
+ mc -C /tmp/mc config host add dest https://s3.lexoya.com ACCESS_KEY_ID SECRET_ACCESS_KEY
Added `dest` successfully.
+ mc -C /tmp/mc ls dest/backups-khuhtziv/my-pxc-backup
[2024-10-30 14:11:52 UTC]  25KiB STANDARD cluster1-2024-10-30-14:11:23-full.md5
[2024-10-30 14:11:38 UTC]   128B STANDARD cluster1-2024-10-30-14:11:23-full.sst_info.md5
[2024-10-30 14:29:36 UTC]     0B cluster1-2024-10-30-14:11:23-full.sst_info/
[2024-10-30 14:29:36 UTC]     0B cluster1-2024-10-30-14:11:23-full/
+ '[' -n '' ']'
+ rm -rf /datadir/#ib_16384_0.dblwr /datadir/#ib_16384_1.dblwr /datadir/#ib_16384_10.dblwr /datadir/#ib_16384_11.dblwr /datadir/#ib_16384_12.dblwr /datadir/#ib_16384_13.dblwr /datadir/#ib_16384_14.dblwr /datadir/#ib_16384_15.dblwr /datadir/#ib_16384_2.dblwr /datadir/#ib_16384_3.dblwr /datadir/#ib_16384_4.dblwr /datadir/#ib_16384_5.dblwr /datadir/#ib_16384_6.dblwr /datadir/#ib_16384_7.dblwr /datadir/#ib_16384_8.dblwr /datadir/#ib_16384_9.dblwr /datadir/#innodb_redo /datadir/#innodb_temp /datadir/auth_plugin /datadir/auto.cnf /datadir/binlog.000001 /datadir/binlog.000002 /datadir/binlog.000003 /datadir/binlog.index /datadir/galera.cache /datadir/get-pxc-state /datadir/grastate.dat /datadir/ib_buffer_pool /datadir/ibdata1 /datadir/liveness-check.sh /datadir/mysql /datadir/mysql.ibd /datadir/peer-list /datadir/performance_schema /datadir/pmm-prerun.sh /datadir/private_key.pem /datadir/public_key.pem /datadir/pxc-configure-pxc.sh /datadir/pxc-entrypoint.sh /datadir/readiness-check.sh /datadir/sys /datadir/undo_001 /datadir/undo_002 /datadir/unsafe-bootstrap.sh /datadir/version_info
++ mktemp --directory /datadir/pxc_sst_XXXX
+ tmp=/datadir/pxc_sst_guHg
++ grep -c processor /proc/cpuinfo
++ grep -c processor /proc/cpuinfo
+ xbstream -x -C /datadir/pxc_sst_guHg --parallel=30
++ destination
++ '[' -n backups-khuhtziv/my-pxc-backup ']'
++ echo -n s3://backups-khuhtziv/my-pxc-backup
+ xbcloud get --parallel=30 --curl-retriable-errors=7 s3://backups-khuhtziv/my-pxc-backup.sst_info
error: http request failed: Couldn't resolve host name
error: http request failed: Couldn't resolve host name
241030 14:29:37 xbcloud: Successfully connected.
241030 14:29:37 xbcloud: Download failed. Cannot list my-pxc-backup.sst_info.
++ grep -c processor /proc/cpuinfo
++ grep -c processor /proc/cpuinfo
+ xbstream --decompress -x -C /datadir/pxc_sst_guHg --parallel=30
++ destination
++ '[' -n backups-khuhtziv/my-pxc-backup ']'
++ echo -n s3://backups-khuhtziv/my-pxc-backup
+ xbcloud get --parallel=30 --curl-retriable-errors=7 s3://backups-khuhtziv/my-pxc-backup
error: http request failed: Couldn't resolve host name
error: http request failed: Couldn't resolve host name
241030 14:29:37 xbcloud: Successfully connected.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_tablespaces.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_logfile.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/undo_002.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/undo_001.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/sys/sys_config.ibd.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/variables_info_195.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/variables_by_thr_192.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/users_153.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/user_variables_b_185.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/user_defined_fun_197.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_145.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_147.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_132.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_142.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_140.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_137.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_135.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/general_log.CSV.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_144.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_cu_120.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_errors_su_150.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_126.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/hosts_155.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_asyn_181.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_127.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_hi_122.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_123.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_139.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/slow_log_226.sdi.00000000000000000000.
241030 14:29:37 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/newdb/newTable.ibd.00000000000000000000.
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/user_variables_b_185.sdi.00000000000000000000, size 4199
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/user_variables_b_185.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_127.sdi.00000000000000000000, size 6919
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_asyn_181.sdi.00000000000000000000, size 7342
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_tablespaces.00000000000000000000, size 95
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/sys/sys_config.ibd.00000000000000000000, size 114740
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/sys/sys_config.ibd.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_tablespaces.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_127.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_asyn_181.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_logfile.00000000000000000000, size 2612
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_logfile.00000000000000000001.
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_errors_su_150.sdi.00000000000000000000, size 8617
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_cu_120.sdi.00000000000000000000, size 11810
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_126.sdi.00000000000000000000, size 7766
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_144.sdi.00000000000000000000, size 16712
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_135.sdi.00000000000000000000, size 25484
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_135.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_144.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_126.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_errors_su_150.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_cu_120.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/general_log.CSV.00000000000000000000, size 35
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/user_variables_b_185.sdi.00000000000000000001, size 57
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_137.sdi.00000000000000000000, size 31445
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_hi_122.sdi.00000000000000000000, size 11369
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_123.sdi.00000000000000000000, size 7794
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_139.sdi.00000000000000000000, size 7946
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_137.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_123.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_139.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_hi_122.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_146.sdi.00000000000000000000.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/error_log_92.sdi.00000000000000000000.
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/hosts_155.sdi.00000000000000000000, size 5368
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/newdb/newTable.ibd.00000000000000000000, size 114740
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_142.sdi.00000000000000000000, size 21105
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_140.sdi.00000000000000000000, size 21546
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_132.sdi.00000000000000000000, size 27187
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_147.sdi.00000000000000000000, size 15009
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_145.sdi.00000000000000000000, size 15856
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/slow_log_226.sdi.00000000000000000000, size 12583
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/newdb/newTable.ibd.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_147.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_145.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_142.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_140.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_132.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/hosts_155.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/slow_log_226.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/variables_info_195.sdi.00000000000000000000, size 8057
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/user_defined_fun_197.sdi.00000000000000000000, size 5890
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/variables_by_thr_192.sdi.00000000000000000000, size 4195
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/users_153.sdi.00000000000000000000, size 5367
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/sys/sys_config.ibd.00000000000000000001, size 32
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_tablespaces.00000000000000000001, size 36
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_127.sdi.00000000000000000001, size 57
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_asyn_181.sdi.00000000000000000001, size 57
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/variables_info_195.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/variables_by_thr_192.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/user_defined_fun_197.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/users_153.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_errors_su_152.sdi.00000000000000000000.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/data_lock_waits_170.sdi.00000000000000000000.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_124.sdi.00000000000000000000.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/binlog.index.00000000000000000000.
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_logfile.00000000000000000001, size 32
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_138.sdi.00000000000000000000.
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_135.sdi.00000000000000000001, size 57
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_126.sdi.00000000000000000001, size 57
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_errors_su_150.sdi.00000000000000000001, size 57
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_144.sdi.00000000000000000001, size 57
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_146.sdi.00000000000000000000, size 15857
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_cu_120.sdi.00000000000000000001, size 57
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_137.sdi.00000000000000000001, size 57
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_123.sdi.00000000000000000001, size 57
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/error_log_92.sdi.00000000000000000000, size 7730
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_hi_122.sdi.00000000000000000001, size 57
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_139.sdi.00000000000000000001, size 57
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/newdb/newTable.ibd.00000000000000000001, size 32
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_145.sdi.00000000000000000001, size 57
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_142.sdi.00000000000000000001, size 57
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/error_log_92.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_140.sdi.00000000000000000001, size 57
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_146.sdi.00000000000000000001.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/binlog.000003.00000000000000000000.
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_147.sdi.00000000000000000001, size 57
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_130.sdi.00000000000000000000.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/setup_instrument_113.sdi.00000000000000000000.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql.ibd.00000000000000000000.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_grou_178.sdi.00000000000000000000.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/wsrep_streaming_log.ibd.00000000000000000000.
241030 14:29:39 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_132.sdi.00000000000000000001, size 57
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/table_handles_167.sdi.00000000000000000000.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/host_cache_105.sdi.00000000000000000000.
241030 14:29:39 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/setup_threads_115.sdi.00000000000000000000.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_143.sdi.00000000000000000000.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/backup-my.cnf.00000000000000000000.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/slow_log.CSM.00000000000000000000.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_appl_179.sdi.00000000000000000000.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/binary_log_trans_198.sdi.00000000000000000000.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_sum_98.sdi.00000000000000000000.
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/hosts_155.sdi.00000000000000000001, size 46
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/slow_log_226.sdi.00000000000000000001, size 36
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/variables_by_thr_192.sdi.00000000000000000001, size 57
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/data_lock_waits_170.sdi.00000000000000000000, size 14214
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/data_lock_waits_170.sdi.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/ibdata1.00000000000000000000.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/global_variables_193.sdi.00000000000000000000.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_checkpoints.00000000000000000000.
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/undo_001.00000000000000000000, size 10485802
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/binlog.index.00000000000000000000, size 62
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_124.sdi.00000000000000000000, size 8620
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_errors_su_152.sdi.00000000000000000000, size 6921
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/users_153.sdi.00000000000000000001, size 46
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/user_defined_fun_197.sdi.00000000000000000001, size 57
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_138.sdi.00000000000000000000, size 6224
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/error_log_92.sdi.00000000000000000001, size 49
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/variables_info_195.sdi.00000000000000000001, size 55
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/binlog.000003.00000000000000000000, size 244
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_146.sdi.00000000000000000001, size 57
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/binlog.000003.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_138.sdi.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/binlog.index.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_124.sdi.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/undo_001.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_errors_su_152.sdi.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/keyring_componen_202.sdi.00000000000000000000.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_errors_su_149.sdi.00000000000000000000.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/malloc_stats_tot_200.sdi.00000000000000000000.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11.00000000000000000000.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10.00000000000000000000.
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/undo_002.00000000000000000000, size 10485802
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/undo_002.00000000000000000001.
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_grou_178.sdi.00000000000000000000, size 12455
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/setup_instrument_113.sdi.00000000000000000000, size 8036
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_sum_98.sdi.00000000000000000000, size 7792
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/table_handles_167.sdi.00000000000000000000, size 9745
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/slow_log.CSM.00000000000000000000, size 87
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_appl_179.sdi.00000000000000000000, size 6829
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_130.sdi.00000000000000000000, size 38567
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/binary_log_trans_198.sdi.00000000000000000000, size 13298
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_143.sdi.00000000000000000000, size 15885
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/backup-my.cnf.00000000000000000000, size 501
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/wsrep_streaming_log.ibd.00000000000000000000, size 114751
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/setup_threads_115.sdi.00000000000000000000, size 6986
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/host_cache_105.sdi.00000000000000000000, size 25403
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_checkpoints.00000000000000000000, size 190
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/data_lock_waits_170.sdi.00000000000000000001, size 56
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/global_variables_193.sdi.00000000000000000000, size 3321
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_checkpoints.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_sum_98.sdi.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/binary_log_trans_198.sdi.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_appl_179.sdi.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/slow_log.CSM.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/backup-my.cnf.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_143.sdi.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/setup_threads_115.sdi.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/host_cache_105.sdi.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_grou_178.sdi.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_130.sdi.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/table_handles_167.sdi.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/wsrep_streaming_log.ibd.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/global_variables_193.sdi.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/setup_instrument_113.sdi.00000000000000000001.
241030 14:29:40 xbcloud: [0] Downloading my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_appl_176.sdi.00000000000000000000.
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/binlog.000003.00000000000000000001, size 27
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/binlog.index.00000000000000000001, size 26
241030 14:29:40 xbcloud: S3 error message: 
241030 14:29:40 xbcloud: [0] Download failed. Cannot download my-pxc-backup/cluster1-2024-10-30-14:11.00000000000000000000.
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_errors_su_152.sdi.00000000000000000001, size 57
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_138.sdi.00000000000000000001, size 57
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_errors_su_149.sdi.00000000000000000000, size 7763
241030 14:29:40 xbcloud: S3 error message: 
241030 14:29:40 xbcloud: [0] Download failed. Cannot download my-pxc-backup/cluster1-2024-10.00000000000000000000.
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/keyring_componen_202.sdi.00000000000000000000, size 9608
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_stages_su_124.sdi.00000000000000000001, size 57
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/malloc_stats_tot_200.sdi.00000000000000000000, size 5799
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql.ibd.00000000000000000000, size 10485803
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/xtrabackup_checkpoints.00000000000000000001, size 36
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/binary_log_trans_198.sdi.00000000000000000001, size 57
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/global_variables_193.sdi.00000000000000000001, size 57
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_waits_sum_98.sdi.00000000000000000001, size 56
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_appl_179.sdi.00000000000000000001, size 57
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/slow_log.CSM.00000000000000000001, size 32
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_transacti_143.sdi.00000000000000000001, size 57
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/events_statement_130.sdi.00000000000000000001, size 57
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_appl_176.sdi.00000000000000000000, size 19978
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/setup_threads_115.sdi.00000000000000000001, size 54
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/host_cache_105.sdi.00000000000000000001, size 51
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/mysql/wsrep_streaming_log.ibd.00000000000000000001, size 43
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/table_handles_167.sdi.00000000000000000001, size 54
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/backup-my.cnf.00000000000000000001, size 27
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/setup_instrument_113.sdi.00000000000000000001, size 57
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/performance_schema/replication_grou_178.sdi.00000000000000000001, size 57
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/ibdata1.00000000000000000000, size 10485801
241030 14:29:40 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/undo_001.00000000000000000001, size 6291498
241030 14:29:41 xbcloud: [0] Download successfull my-pxc-backup/cluster1-2024-10-30-14:11:23-full/undo_002.00000000000000000001, size 6291498
241030 14:29:41 xbcloud: Download failed.
+ set +o xtrace
vault configuration not found
+ xtrabackup --use-memory=100MB --prepare  --rollback-prepared-trx     --xtrabackup-plugin-dir=/usr/lib64/xtrabackup/plugin --target-dir=/datadir/pxc_sst_guHg
2024-10-30T14:29:41.383213-00:00 0 [Note] [MY-011825] [Xtrabackup] recognized server arguments: --innodb_checksum_algorithm=crc32 --innodb_log_checksums=1 --innodb_data_file_path=ibdata1:12M:autoextend --innodb_log_file_size=50331648 --innodb_page_size=16384 --innodb_undo_directory=./ --innodb_undo_tablespaces=2 --server-id=32135110 --innodb_log_checksums=ON --innodb_redo_log_encrypt=0 --innodb_undo_log_encrypt=0 
2024-10-30T14:29:41.383708-00:00 0 [Note] [MY-011825] [Xtrabackup] recognized client arguments: --use-memory=100MB --prepare=1 --rollback-prepared-trx=1 --xtrabackup-plugin-dir=/usr/lib64/xtrabackup/plugin --target-dir=/datadir/pxc_sst_guHg 
xtrabackup version 8.0.35-30 based on MySQL server 8.0.35 Linux (x86_64) (revision id: 6beb4b49)
2024-10-30T14:29:41.383878-00:00 0 [Note] [MY-011825] [Xtrabackup] cd to /datadir/pxc_sst_guHg/
2024-10-30T14:29:41.384052-00:00 0 [ERROR] [MY-011825] [Xtrabackup] cannot open ./xtrabackup_info
2024-10-30T14:29:41.384131-00:00 0 [ERROR] [MY-011825] [Xtrabackup] Failed to parse xtrabackup_info from './xtrabackup_info'

But if I create a new cluster (cluster3) and restore using the backupName field, the job succeeds and data is restored meaning the backup is actually healthy.

apiVersion: pxc.percona.com/v1
kind: PerconaXtraDBClusterRestore
metadata:
  name: restore-that-works
  namespace: testtt
spec:
  pxcCluster: cluster3
  backupName: backup1

So without having access to backup CR (in a separate k8s cluster), it's not possible to restore data with specifying s3 config in restore CR.

Steps to reproduce

  1. Create new cluster and insert data
  2. Take backup to s3 using backup CR
  3. Try to restore from s3 using restore CR with backupSource field
  4. It fails

Versions

  1. Kubernetes
    Client Version: version.Info{Major:"1", Minor:"24+", GitVersion:"v1.24.13-eks-0a21954", GitCommit:"6305d65c340554ad8b4d7a5f21391c9fa34932cb", GitTreeState:"clean", BuildDate:"2023-04-15T00:37:31Z", GoVersion:"go1.19.8", Compiler:"gc", Platform:"linux/amd64"}
    Kustomize Version: v4.5.4
    Server Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.6", GitCommit:"741c8db18a52787d734cbe4795f0b4ad860906d6", GitTreeState:"clean", BuildDate:"2023-09-13T09:14:09Z", GoVersion:"go1.20.8", Compiler:"gc", Platform:"linux/amd64"}

  2. Operator
    1.15.1

  3. Database
    percona-xtradb-cluster:8.0.36-28.1

Anything else?

The "How to restore backup to a new Kubernetes-based environment" doc (I posted the link) was confusing and even at some point it linked to mongodb operator restore CR instead of mysql.

No response
@mahdiG mahdiG added the bug label Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@mahdiG