Skip to content

Commit

Permalink
cleanup some of the storage docs after refactor
Browse files Browse the repository at this point in the history 
Signed-off-by: Kevin Carter <[email protected]>
  • Loading branch information
@cloudnull
cloudnull committed Mar 7, 2024
1 parent 02d1b8c commit 821fa4c
Showing 1 changed file with 44 additions and 30 deletions.
74 changes: 44 additions & 30 deletions docs/storage-ceph-rook-external.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,9 @@ rbd pool init general
ceph fs new general-multi-attach general-multi-attach-metadata general-multi-attach-data
```

## You must have a MDS service running, in this example I am tagging my 3 ceph nodes with MDS labels and creating a MDS service for the general-multi-attach Cephfs Pool
!!! info

You must have a MDS service running, in this example I am tagging my 3 ceph nodes with MDS labels and creating a MDS service for the general-multi-attach Cephfs Pool

``` shell
ceph orch host label add genestack-ceph1 mds
Expand All @@ -21,41 +23,49 @@ ceph orch host label add genestack-ceph3 mds
ceph orch apply mds myfs label:mds
```

## We will now download create-external-cluster-resources.py and create exports to run on your controller node. Using cephadm in this example:
!!! note

We will now download create-external-cluster-resources.py and create exports to run on your controller node. Using cephadm in this example:

``` shell
./cephadm shell
yum install wget -y ; wget https://raw.githubusercontent.com/rook/rook/release-1.12/deploy/examples/create-external-cluster-resources.py
python3 create-external-cluster-resources.py --rbd-data-pool-name general --cephfs-filesystem-name general-multi-attach --namespace rook-ceph-external --format bash
```

## Copy and paste the output, here is an example:
``` shell
root@genestack-ceph1:/# python3 create-external-cluster-resources.py --rbd-data-pool-name general --cephfs-filesystem-name general-multi-attach --namespace rook-ceph-external --format bash
export NAMESPACE=rook-ceph-external
export ROOK_EXTERNAL_FSID=d45869e0-ccdf-11ee-8177-1d25f5ec2433
export ROOK_EXTERNAL_USERNAME=client.healthchecker
export ROOK_EXTERNAL_CEPH_MON_DATA=genestack-ceph1=10.1.1.209:6789
export ROOK_EXTERNAL_USER_SECRET=AQATh89lf5KiBBAATgaOGAMELzPOIpiCg6ANfA==
export ROOK_EXTERNAL_DASHBOARD_LINK=https://10.1.1.209:8443/
export CSI_RBD_NODE_SECRET=AQATh89l3AJjBRAAYD+/cuf3XPdMBmdmz4iWIA==
export CSI_RBD_NODE_SECRET_NAME=csi-rbd-node
export CSI_RBD_PROVISIONER_SECRET=AQATh89l9dH4BRAApBKzqwtaUqw9bNcBI/iGGw==
export CSI_RBD_PROVISIONER_SECRET_NAME=csi-rbd-provisioner
export CEPHFS_POOL_NAME=general-multi-attach-data
export CEPHFS_METADATA_POOL_NAME=general-multi-attach-metadata
export CEPHFS_FS_NAME=general-multi-attach
export CSI_CEPHFS_NODE_SECRET=AQATh89lFeqMBhAAJpHAE5vtukXYuRj2+WTh2g==
export CSI_CEPHFS_PROVISIONER_SECRET=AQATh89lHB0dBxAA7CHM/9rTSs79SLJSKVBYeg==
export CSI_CEPHFS_NODE_SECRET_NAME=csi-cephfs-node
export CSI_CEPHFS_PROVISIONER_SECRET_NAME=csi-cephfs-provisioner
export MONITORING_ENDPOINT=10.1.1.209
export MONITORING_ENDPOINT_PORT=9283
export RBD_POOL_NAME=general
export RGW_POOL_PREFIX=default
```
!!! example "Example create-external-cluster-resources.py output"

The script generates a lot of output, you will need to capture all of the exports. These exports will be used in the next command.

``` shell
root@genestack-ceph1:/# python3 create-external-cluster-resources.py --rbd-data-pool-name general --cephfs-filesystem-name general-multi-attach --namespace rook-ceph-external --format bash
export NAMESPACE=rook-ceph-external
export ROOK_EXTERNAL_FSID=d45869e0-ccdf-11ee-8177-1d25f5ec2433
export ROOK_EXTERNAL_USERNAME=client.healthchecker
export ROOK_EXTERNAL_CEPH_MON_DATA=genestack-ceph1=10.1.1.209:6789
export ROOK_EXTERNAL_USER_SECRET=AQATh89lf5KiBBAATgaOGAMELzPOIpiCg6ANfA==
export ROOK_EXTERNAL_DASHBOARD_LINK=https://10.1.1.209:8443/
export CSI_RBD_NODE_SECRET=AQATh89l3AJjBRAAYD+/cuf3XPdMBmdmz4iWIA==
export CSI_RBD_NODE_SECRET_NAME=csi-rbd-node
export CSI_RBD_PROVISIONER_SECRET=AQATh89l9dH4BRAApBKzqwtaUqw9bNcBI/iGGw==
export CSI_RBD_PROVISIONER_SECRET_NAME=csi-rbd-provisioner
export CEPHFS_POOL_NAME=general-multi-attach-data
export CEPHFS_METADATA_POOL_NAME=general-multi-attach-metadata
export CEPHFS_FS_NAME=general-multi-attach
export CSI_CEPHFS_NODE_SECRET=AQATh89lFeqMBhAAJpHAE5vtukXYuRj2+WTh2g==
export CSI_CEPHFS_PROVISIONER_SECRET=AQATh89lHB0dBxAA7CHM/9rTSs79SLJSKVBYeg==
export CSI_CEPHFS_NODE_SECRET_NAME=csi-cephfs-node
export CSI_CEPHFS_PROVISIONER_SECRET_NAME=csi-cephfs-provisioner
export MONITORING_ENDPOINT=10.1.1.209
export MONITORING_ENDPOINT_PORT=9283
export RBD_POOL_NAME=general
export RGW_POOL_PREFIX=default
```

!!! info

Run the following commands to import the cluster after pasting in exports from external cluster

## Run the following commands to import the cluster after pasting in exports from external cluster
``` shell
kubectl apply -k /opt/genestack/kustomize/rook-operator/
/opt/genestack/scripts/import-external-cluster.sh
Expand All @@ -64,12 +74,16 @@ helm install --create-namespace --namespace rook-ceph-external rook-ceph-cluster
kubectl patch storageclass general -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
```

## Monitor progress:
## Monitor progress

``` shell
kubectl --namespace rook-ceph-external get cephcluster -w
```

## Should return when finished:
## Validate the Connection

When the monitor shows the message "Cluster connected successfully" the cluster is connected and ready for use.

``` shell
NAME DATADIRHOSTPATH MONCOUNT AGE PHASE MESSAGE HEALTH EXTERNAL FSID
rook-ceph-external /var/lib/rook 3 3m24s Connected Cluster connected successfully HEALTH_OK true d45869e0-ccdf-11ee-8177-1d25f5ec2433
Expand Down

0 comments on commit 821fa4c

Please sign in to comment.