Skip to content

Commit

Permalink
Adding QoS examples for Cinder Volumes and in particular NetApp ONTAP (
Browse files Browse the repository at this point in the history
…#622)

* Adding QoS examples for Cinder Volumes and in particular NetApp ONTAP

The following documentation had warning notes updated:
openstack-cinder-lvmisci.md
openstack-cinder-netapp-container.md
openstack-cinder-netapp-worker.md
openstack-cinder-volume-provisioning-specs.md
openstack-cinder-volume-qos-policies.md
openstack-cinder-volume-type-specs.md

* newline fix

* extra line fix

* volume type specs edit

* Add link to current NetApp ONTAP doc with QoS explanations
  • Loading branch information
dwith authored Dec 17, 2024
1 parent 5e61f83 commit 9b0aa83
Show file tree
Hide file tree
Showing 8 changed files with 157 additions and 50 deletions.
10 changes: 1 addition & 9 deletions ansible/playbooks/deploy-cinder-netapp-volumes-reference.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
cinder_release: "2024.1"
cinder_storage_network_interface: ansible_br_storage
cinder_storage_network_interface_secondary: ansible_br_storage_secondary
cinder_backend_name: "block-ha-performance-at-rest-encrypted,block-ha-standard-at-rest-encrypted"
cinder_backend_name: "block-ha-performance-at-rest-encrypted,block-ha-standard-at-rest-encrypted,block-ha-performance-end-to-end-encrypted,block-ha-standard-end-to-end-encrypted"
custom_multipath: false
handlers:
- name: Restart cinder-volume-netapp systemd services
Expand Down Expand Up @@ -198,14 +198,6 @@
group: root
mode: "0644"

- name: Replace yaml safe qos policy key in netapp-backends configuration
ansible.builtin.replace:
path: /etc/cinder/netapp-backends.conf
regexp: 'netapp_qos_policy_group'
replace: 'netapp:qos_policy_group'
notify:
- Restart cinder-volume-netapp systemd services

- name: Replace exec path in rootwrap
community.general.ini_file:
path: /etc/cinder/rootwrap.conf
Expand Down
29 changes: 1 addition & 28 deletions docs/openstack-cinder-lvmisci.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,35 +119,8 @@ root@openstack-node-0:~# kubectl --namespace openstack exec -ti openstack-admin-

!!! warning

You must define and apply the volume type QoS policies **before** creating a volume within your new volume type (`lvmdriver-1`). Also, any additional volume provisioning specifications like the minimum and maximum size for the volume type must be set **before** creating a volume within your new volume type (`lvmdriver-1`).
**Before** creating a volume, ensure that your volume type has been set up correctly and that you have applied QoS policies, provisioning specifications (min and max volume size), and any extra specs. See [Cinder Volume QoS Policies](openstack-cinder-volume-qos-policies.md), [Cinder Volume Provisioning Specs](openstack-cinder-volume-provisioning-specs.md), and [Cinder Volume Type Specs](openstack-cinder-volume-type-specs.md).

In order to apply a QoS policy to the `lvmdriver-1` volume type, you must first create the QoS policy.

``` shell
root@openstack-node-0:~# kubectl --namespace openstack exec -ti openstack-admin-client -- openstack volume qos create --consumer "both" --property "read_iops_sec_per_gb=1" --property "write_iops_sec_per_gb=1" lvmdriver-1-iops
+------------+-----------------------------------------------------+
| Field | Value |
+------------+-----------------------------------------------------+
| consumer | both |
| id | b35fdf9c-d5bd-40f9-ae3a-8605c246ef2e |
| name | lvmdriver-1-iops |
| properties | read_iops_sec_per_gb='1', write_iops_sec_per_gb='1' |
+------------+-----------------------------------------------------+
```

Once you have created the QoS policy, apply it to the `lvmdriver-1` volume type.
The command will utilize the `QOS_ID` and `VOLUME_TYPE_ID`.

``` shell
root@openstack-node-0:~# kubectl --namespace openstack exec -ti openstack-admin-client -- openstack volume qos associate b35fdf9c-d5bd-40f9-ae3a-8605c246ef2e 6af6ade2-53ca-4260-8b79-1ba2f208c91d
```

Set any additional volume provisioning specifications like the minimum and maximum volume size. These specifications are set in the volume type. The following commands constrain the `lvmdriver-1` volume type to a size between 10 GB and 2 TB.

``` shell
root@openstack-node-0:~# kubectl --namespace openstack exec -ti openstack-admin-client -- openstack volume type set --property provisioning:min_vol_size=10 6af6ade2-53ca-4260-8b79-1ba2f208c91d
root@openstack-node-0:~# kubectl --namespace openstack exec -ti openstack-admin-client -- openstack volume type set --property provisioning:max_vol_size=2048 6af6ade2-53ca-4260-8b79-1ba2f208c91d
```

## Validate functionality

Expand Down
17 changes: 10 additions & 7 deletions docs/openstack-cinder-netapp-container.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@
This document provides information on configuring NetApp backends for the isolated Cinder volume worker. Each backend is defined by a set of
11 comma-separated options, and multiple backends can be specified by separating them with semicolons.

!!! warning "The NetApp container is incompatible with iSCSI workloads"

!!! warning "The NetApp container is incompatible with iSCSI workloads"

The NetApp container is incompatible with iSCSI workloads. If the environment requires iSCSI support, review the [Cinder NetApp Worker ](openstack-cinder-netapp-worker.md) documentation instead.

Expand All @@ -19,11 +20,10 @@ Below is a table detailing each option, its position in the backend configuratio
| 3 | `netapp_server_hostname` | Hostname or IP address of the NetApp storage system. | String |
| 4 | `netapp_server_port` | Port number to communicate with the NetApp storage system. | Integer |
| 5 | `netapp_vserver` | The name of the Vserver on the NetApp storage system. | String |
| 6 | `netapp:qos_policy_group` | The name of the QoS policy group. | String |
| 7 | `netapp_dedup` | Enable (`True`) or disable (`False`) deduplication. | Boolean |
| 8 | `netapp_compression` | Enable (`True`) or disable (`False`) compression. | Boolean |
| 9 | `netapp_thick_provisioned` | Use thick (`True`) or thin (`False`) provisioning. | Boolean |
| 10 | `netapp_lun_space_reservation`| Enable (`enabled`) or disable (`disabled`) LUN space reservation. | String |
| 6 | `netapp_dedup` | Enable (`True`) or disable (`False`) deduplication. | Boolean |
| 7 | `netapp_compression` | Enable (`True`) or disable (`False`) compression. | Boolean |
| 8 | `netapp_thick_provisioned` | Use thick (`True`) or thin (`False`) provisioning. | Boolean |
| 9 | `netapp_lun_space_reservation`| Enable (`enabled`) or disable (`disabled`) LUN space reservation. | String |

### Detailed Option Descriptions

Expand All @@ -33,7 +33,6 @@ Below is a table detailing each option, its position in the backend configuratio
- **`netapp_server_hostname`**: The address of the NetApp storage system. This can be either an IP address or a fully qualified domain name (FQDN).
- **`netapp_server_port`**: The port number used for communication with the NetApp storage system. Common ports are `80` for HTTP and `443` for HTTPS.
- **`netapp_vserver`**: Specifies the virtual storage server (Vserver) on the NetApp storage system that will serve the volumes.
- **`netapp:qos_policy_group`**: The Quality of Service (QoS) policy group name that will be applied to volumes for this backend.
- **`netapp_dedup`**: A boolean value to enable or disable deduplication on the storage volumes. Acceptable values are `True` or `False`.
- **`netapp_compression`**: A boolean value to enable or disable compression on the storage volumes. Acceptable values are `True` or `False`.
- **`netapp_thick_provisioned`**: Determines whether volumes are thick (`True`) or thin (`False`) provisioned.
Expand Down Expand Up @@ -62,6 +61,10 @@ in a specific order.

## Run the deployment

!!! warning

**Before** deploying a new backend, ensure that your volume type has been set up correctly and that you have applied QoS policies, provisioning specifications (min and max volume size), and any extra specs. See [Cinder Volume QoS Policies](openstack-cinder-volume-qos-policies.md), [Cinder Volume Provisioning Specs](openstack-cinder-volume-provisioning-specs.md), and [Cinder Volume Type Specs](openstack-cinder-volume-type-specs.md).

With your configuration defined, run the deployment with a standard `kubectl apply` command.

``` shell
Expand Down
10 changes: 4 additions & 6 deletions docs/openstack-cinder-netapp-worker.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ The NetApp Volume Worker is a cinder-volume service that is configured to use th

The `deploy-cinder-volumes-netapp-reference.yaml` playbook is used to deploy the NetApp Volume Worker. The playbook will deploy the cinder-volume service on the specified nodes and configure the service to use the NetApp ONTAP driver. This playbook will read from the Kubernetes environment to determine the necessary configuration parameters for the NetApp ONTAP driver. As an operator, you will need to ensure that the necessary configuration parameters are set in the Kubernetes environment before running the playbook.

!!! warning

**Before** deploying a new backend, ensure that your volume type has been set up correctly and that you have applied QoS policies, provisioning specifications (min and max volume size), and any extra specs. See [Cinder Volume QoS Policies](openstack-cinder-volume-qos-policies.md), [Cinder Volume Provisioning Specs](openstack-cinder-volume-provisioning-specs.md), and [Cinder Volume Type Specs](openstack-cinder-volume-type-specs.md).

### Backends Configuration

The NetApp ONTAP driver requires a backend configuration to be set in the Kubernetes environment. The backend configuration specifies the storage system that the NetApp Volume Worker will use to create and manage volumes. The backend configuration is a Kubernetes secret that contains the necessary configuration parameters for the NetApp ONTAP driver. To define the backends, update the helm overrides file with the necessary configuration parameters.
Expand All @@ -24,7 +28,6 @@ The NetApp ONTAP driver requires a backend configuration to be set in the Kubern
netapp_storage_protocol: iscsi
netapp_transport_type: http
netapp_vserver: <VSERVER>
netapp_qos_policy_group: <QOS_POLICY_GROUP>
netapp_dedup: True
netapp_compression: True
netapp_thick_provisioned: True
Expand All @@ -40,7 +43,6 @@ The NetApp ONTAP driver requires a backend configuration to be set in the Kubern
netapp_storage_protocol: iscsi
netapp_transport_type: http
netapp_vserver: <VSERVER>
netapp_qos_policy_group: <QOS_POLICY_GROUP>
netapp_dedup: True
netapp_compression: True
netapp_thick_provisioned: True
Expand All @@ -56,10 +58,6 @@ The NetApp ONTAP driver requires a backend configuration to be set in the Kubern
- **`SERVER_NAME_OR_ADDRESS`**: The address of the NetApp storage system. This can be either an IP address or a fully qualified domain name (FQDN).
- **`SERVER_PORT`**: The port number used for communication with the NetApp storage system. Common ports are `80` for HTTP and `443` for HTTPS.
- **`VSERVER`**: Specifies the virtual storage server (Vserver) on the NetApp storage system that will serve the volumes.
- **`QOS_POLICY_GROUP`**: The Quality of Service (QoS) policy group name that will be applied to volumes for this backend.

!!! note
The Cinder backends configuration file utilizes a colon (:) in the variable name `netapp:qos_policy_group`, but it has been replaced in the Helm overrides file with an underscore. Helm recognizes a colon (:) as a special character and renders it as an equal (=) sign. This is undesired behavior for the final rendering of the Cinder backends configuration file, and the Ansible playbook replaces the underscore with a colon (:). The extra spec requires the colon (:) in its name because it is used by the NetAppdriver to assign the QoS policy group to the OpenStack Block Storage volume after it has been provisioned. See [NetApp unified driver](https://docs.openstack.org/cinder/latest/configuration/block-storage/drivers/netapp-volume-driver.html) for more detailed information on the NetApp driver extra specs.

## Host Setup

Expand Down
12 changes: 12 additions & 0 deletions docs/openstack-cinder-volume-provisioning-specs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Set Volume Type Provisioning Specifications

*Before* creating a volume within any volume type, the provisioning specifications must be set.

## Minimum and Maximum volume size

These specifications are set in the volume type. The following commands constrain the `lvmdriver-1` volume type to a size between 10 GB and 2 TB.

``` shell
root@openstack-node-0:~# kubectl --namespace openstack exec -ti openstack-admin-client -- openstack volume type set --property provisioning:min_vol_size=10 6af6ade2-53ca-4260-8b79-1ba2f208c91d
root@openstack-node-0:~# kubectl --namespace openstack exec -ti openstack-admin-client -- openstack volume type set --property provisioning:max_vol_size=2048 6af6ade2-53ca-4260-8b79-1ba2f208c91d
```
109 changes: 109 additions & 0 deletions docs/openstack-cinder-volume-qos-policies.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Volume QoS Policies

## LVM


### Example QoS policy for LVM driver volume type

In order to apply a QoS policy to the `lvmdriver-1` volume type, you must first create the QoS policy.

``` shell
root@openstack-node-0:~# kubectl --namespace openstack exec -ti openstack-admin-client -- openstack volume qos create --consumer "both" --property "read_iops_sec_per_gb=1" --property "write_iops_sec_per_gb=1" lvmdriver-1-iops
+------------+-----------------------------------------------------+
| Field | Value |
+------------+-----------------------------------------------------+
| consumer | both |
| id | b35fdf9c-d5bd-40f9-ae3a-8605c246ef2e |
| name | lvmdriver-1-iops |
| properties | read_iops_sec_per_gb='1', write_iops_sec_per_gb='1' |
+------------+-----------------------------------------------------+
```

Once you have created the QoS policy, apply it to the `lvmdriver-1` volume type.
The command will utilize the `QOS_ID` and `VOLUME_TYPE_ID`.

``` shell
root@openstack-node-0:~# kubectl --namespace openstack exec -ti openstack-admin-client -- openstack volume qos associate b35fdf9c-d5bd-40f9-ae3a-8605c246ef2e 6af6ade2-53ca-4260-8b79-1ba2f208c91d
```

## NetAPP ONTAP

The most recent releases of the ONTAP driver (OpenStack Train and higher) allow QoS policies to be set per volume at the Cinder volume type rather than trying to utilize a QoS policy created on a target NetApp device. For a more detailed explanation, consult [NetApp Cinder QoS Concepts](https://netapp-openstack-dev.github.io/openstack-docs/train/cinder/key_concepts/section_cinder-key-concepts.html#qos-spec)

### Example QoS policy for NetApp ONTAP volume type

In order to apply a QoS policy to the `netapp-1` volume type, you must first create the QoS policy.

``` shell
root@openstack-node-0:~# kubectl --namespace openstack exec -ti openstack-admin-client -- openstack volume qos create --consumer "back-end" --property "peakIOPSperGiB=8" --property "expectedIOPSperGiB=7" netapp-qos
+------------+--------------------------------------------+
| Field | Value |
+------------+--------------------------------------------+
| consumer | back-end |
| id | 9435160f-0e4a-4486-88b0-d6beb022732a |
| name | netapp-qos |
| properties | expectedIOPSperGiB='7', peakIOPSperGiB='8' |
+------------+--------------------------------------------+
```

`expectedIOPSperGiB=7` was chosen because the target IOPSperGiB or expectedIOPSperGiB will be observed to be `5 IOPSperGiB` or `5,000 IOPS` when running FIO tests. Likewise, the `peakIOPSperGiB=8` was chosen because it is a value of `1` over the `expectedIOPSperGiB=7` and will effectively cap a burst in IOPs to an actual observerd value of `6 IOPSperGiB` or `6,000 IOPS`.

``` shell
root@openstack-node-0:~# kubectl --namespace openstack exec -ti openstack-admin-client -- openstack volume type show 1bdb5364-ed04-4bbe-8e41-9c5fae148c3d
+--------------------+----------------------------------------+
| Field | Value |
+--------------------+----------------------------------------+
| access_project_ids | |
| description | None |
| id | 1bdb5364-ed04-4bbe-8e41-9c5fae148c3d |
| is_public | False |
| name | netapp-1 |
| properties | volume_backend_name='netapp-1-backend' |
| qos_specs_id | None |
+--------------------+----------------------------------------+
```

Once you have created the QoS policy, apply it to the `netapp-1` volume type.
The command will utilize the `QOS_ID` and `VOLUME_TYPE_ID`.

``` shell
root@openstack-node-0:~# kubectl --namespace openstack exec -ti openstack-admin-client -- openstack volume qos associate 9435160f-0e4a-4486-88b0-d6beb022732a 1bdb5364-ed04-4bbe-8e41-9c5fae148c3d
root@openstack-node-0:~# kubectl --namespace openstack exec -ti openstack-admin-client -- openstack volume type show 1bdb5364-ed04-4bbe-8e41-9c5fae148c3d
+--------------------+----------------------------------------+
| Field | Value |
+--------------------+----------------------------------------+
| access_project_ids | |
| description | None |
| id | 1bdb5364-ed04-4bbe-8e41-9c5fae148c3d |
| is_public | False |
| name | netapp-1 |
| properties | volume_backend_name='netapp-1-backend' |
| qos_specs_id | 9435160f-0e4a-4486-88b0-d6beb022732a |
+--------------------+----------------------------------------+
```

## Disassociate a QoS policy from a volume type

In order to delete a QoS policy, you must first disassociate it from any volume types that it has been associated with.

``` shell
root@openstack-node-0:~# kubectl --namespace openstack exec -ti openstack-admin-client -- openstack volume qos disassociate --volume-type 1bdb5364-ed04-4bbe-8e41-9c5fae148c3d 9435160f-0e4a-4486-88b0-d6beb022732a
root@openstack-node-0:~# kubectl --namespace openstack exec -ti openstack-admin-client -- openstack volume type show 1bdb5364-ed04-4bbe-8e41-9c5fae148c3d
+--------------------+----------------------------------------+
| Field | Value |
+--------------------+----------------------------------------+
| access_project_ids | |
| description | None |
| id | 1bdb5364-ed04-4bbe-8e41-9c5fae148c3d |
| is_public | False |
| name | netapp-1 |
| properties | volume_backend_name='netapp-1-backend' |
| qos_specs_id | None |
+--------------------+----------------------------------------+
```

## Delete a QoS policy

``` shell
root@openstack-node-0:~# kubectl --namespace openstack exec -ti openstack-admin-client -- openstack volume qos delete 9435160f-0e4a-4486-88b0-d6beb022732a
```
16 changes: 16 additions & 0 deletions docs/openstack-cinder-volume-type-specs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Set Volume Type Specifications

## Example: NetApp ONTAP driver volume specifications (deduplication and compression)

To set and unset additional properties on a NetApp volume type, use the syntax examples.

``` shell

root@openstack-node-0:~# kubectl --namespace openstack exec -ti openstack-admin-client -- openstack volume type set --property netapp_dedup='true' <VOLUME_TYPE_ID>

root@openstack-node-0:~# kubectl --namespace openstack exec -ti openstack-admin-client -- openstack volume type set --property netapp_compression='true' <VOLUME_TYPE_ID>

root@openstack-node-0:~# kubectl --namespace openstack exec -ti openstack-admin-client -- openstack volume type unset --property netapp_dedup <VOLUME_TYPE_ID>

root@openstack-node-0:~# kubectl --namespace openstack exec -ti openstack-admin-client -- openstack volume type unset --property netapp_compression <VOLUME_TYPE_ID>
```
Loading

0 comments on commit 9b0aa83

Please sign in to comment.