Introduction¶
The HPE Alletra 9000 and Primera and 3PAR Storage Container Storage Provider (CSP) for Kubernetes is part of the HPE CSI Driver for Kubernetes. The CSP abstract the data management capabilities of the array for use by Kubernetes.
- Introduction
- Platform Requirements
- StorageClass Parameters
- Common Provisioning Parameters
- Ephemeral Inline Volumes
- VLUN Templates
- Importing Volumes
- Cloning Persistent Volumes
- Creating Snapshots of Persistent Volumes
- Remote Copy with Peer Persistence Synchronous Replication Parameters
- Add Non-Replicated Volume to Remote Copy group
- Specifying iSCSI Target Portal IPs
- Creating a VolumeSnapshotClass
- Import a Snapshot
- Creating a QoS StorageClass
- Creating a SnapshotGroupClass
- Creating a VolumeGroupClass with QoS Settings
- Specifying Fibre Channel Ports
- Support
Note
For help getting started with deploying the HPE CSI Driver using HPE Alletra 9000, Primera or 3PAR storage, check out the tutorial over at HPE Developer.
Platform Requirements¶
Check the corresponding CSI driver version in the compatibility and support table for the latest updates on supported Kubernetes version, orchestrators and host OS.
Refer to the HPE Single Point of Connectivity Knowledge (SPOCK) for specific platform details (requires an HPE Passport account) of the CSP. The documentation reflected here always corresponds to the latest supported version and may contain references to future features and capabilities.
Network Port Requirements¶
The HPE Alletra 9000, Primera and 3PAR Container Storage Provider requires the following TCP ports to be open inbound to the array from the Kubernetes cluster worker nodes running the HPE CSI Driver for Kubernetes.
Port | Protocol | Description |
---|---|---|
443 | HTTPS | WSAPI (HPE Alletra 9000/Primera) |
8080 | HTTPS | WSAPI (HPE 3PAR) |
22 | SSH | Array communication |
User Role Requirements¶
The CSP requires access to a local user with either edit
or the super
role. It's recommended to use the edit
role for security best practices.
Note
LDAP users are not supported by the CSP.
StorageClass Parameters¶
All parameters enumerated reflects the current version and may contain unannounced features and capabilities.
Common Provisioning Parameters¶
Parameter | Option | Description |
---|---|---|
accessProtocol (Required) |
fc | The access protocol to use when accessing the persistent volume. |
iscsi | The access protocol to use when accessing the persistent volume. | |
cpg 1 | Text | The name of existing CPG to be used for volume provisioning. If the cpg parameter is not specified, the CSP will automatically set cpg parameter based upon a CPG available to the array. |
snapCpg 1 | Text | The name of the snapshot CPG to be used for volume provisioning. Defaults to value of cpg if not specified. |
compression 1 | Boolean | Indicates that the volume should be compressed. (3PAR only) |
provisioningType 1 (Default: tpvv) |
tpvv | Indicates Thin provisioned volume type. |
full 3 | Indicates Full provisioned volume type. | |
dedup 3 | Indicates Thin Deduplication volume type. | |
reduce 4 | Indicates Data Reduction volume type. | |
hostSeesVLUN | Boolean | Enable "host sees" VLUN template. |
importVol | Text | Name of the volume to import. |
importVolAsClone | Text | Name of the volume to clone and import. |
cloneOf 2 | Text | Name of the PersistentVolumeClaim to clone. |
virtualCopyOf 2 | Text | Name of the PersistentVolumeClaim to snapshot. |
qosName | Text | Name of the volume set which has QoS rules applied. |
remoteCopyGroup 1 | Text | Name of a new or existing Remote Copy group on the array. |
replicationDevices | Text | Indicates name of custom resource of type hpereplicationdeviceinfos . |
allowBatchReplicatedVolumeCreation | Boolean | Enable the batch processing of persistent volumes in 10 second intervals and add them to a single Remote Copy group. During this process, the Remote Copy group is stopped and started once. |
oneRcgPerPvc | Boolean | Creates a dedicated Remote Copy group per persistent volume. |
iscsiPortalIps | Text | Comma separated list of the array iSCSI port IPs. |
Restrictions applicable when using the CSI volume mutator:
1 = Parameters that are editable after provisioning.
2 = Volumes with snapshots/clones can't be modified.
3 = HPE 3PAR only parameter
4 = HPE Primera/Alletra 9000 only parameter
Please see using the HPE CSI Driver for additional StorageClass
examples like CSI snapshots and clones.
Important
The HPE CSI Driver allows the PersistentVolumeClaim
to override the StorageClass
parameters by annotating the PersistentVolumeClaim
. Please see Using PVC Overrides for more details.
Ephemeral Inline Volumes¶
These parameters are applicable only for Pod inline volumes and to be specified within Pod spec.
Parameter | String | Description |
---|---|---|
csi.storage.k8s.io/ephemeral | Boolean | Indicates that the request is for ephemeral inline volume. This is a mandatory parameter and must be set to "true". |
inline-volume-secret-name | Text | A reference to the secret object containing sensitive information to pass to the CSI driver to complete the CSI NodePublishVolume call. |
inline-volume-secret-namespace | Text | The namespace of inline-volume-secret-name for ephemeral inline volume. |
size | Text | The size of ephemeral volume specified in MiB or GiB. If unspecified, a default value will be used. |
accessProtocol | Text | Storage access protocol to use, "iscsi" or "fc". |
Important
All parameters are required for inline ephemeral volumes.
VLUN Templates¶
A VLUN template enables the export of a virtual volume as a VLUN to hosts. For more information, see the HPE Primera OS Commmand Line Interface - Installation and Reference Guide.
The HPE CSI Driver supports the following types of VLUN templates:
- Matched set - (Default) The VLUN is visible to initiators with the host's WWNs only on the specified port(s).
- Host sees - The VLUN is visible to the initiators with any of the host's WWNs.
Parameter | String | Description |
---|---|---|
hostSeesVLUN | Boolean | Enable "host sees" VLUN template. |
Note
To modify an existing PVC, hostSeesVLUN needs to be specified with the allowMutations parameter along with adding the PVC annotation csi.hpe.com/hostSeesVLUN with the string values of either "true" or "false". The HPE CSI Driver creates the vlun template based upon the hostSeesVLUN
parameter during the volume publish operation. For the change to take effect, the Pod
will need to be scheduled on another node by either deleting the Pod
or draining the node.
Importing Volumes¶
During the import volume process, any legacy (non-container volumes) or existing docker/k8s volume defined in the ImportVol parameter, within a StorageClass
, will be renamed to match the PersistentVolumeClaim
that leverages the StorageClass
. The new volumes will be exposed through the HPE CSI Driver and made available to the Kubernetes cluster. Note: All previous Access Control Records and Initiator Groups will be removed from the volume when it is imported. The hostSeesVLUN
parameter improves the performance in clusters where many pods are being migrated or created.
Parameter | Option | Description |
---|---|---|
accessProtocol | fc or iscsi | The access protocol to use when accessing the persistent volume. |
importVol | Text | The name of the array volume to import. |
Important
• No other parameters are required in the StorageClass
when importing a volume outside of those parameters listed in the table above.
• Support for importVol
is available from HPE CSI Driver 1.2.0+.
Cloning Persistent Volumes¶
Cloning supports two modes of cloning. Either use cloneOf
and reference a PersistentVolumeClaim
in the current namespace to clone or use importVolAsClone
and reference an array volume name to clone and import into the Kubernetes cluster. Volumes with clones are immutable once created.
Parameter | Option | Description |
---|---|---|
cloneOf | Text | The name of the PersistentVolumeClaim to be cloned. cloneOf and importVolAsClone are mutually exclusive. |
importVolAsClone | Text | The name of the array volume to clone and import. importVolAsClone and cloneOf are mutually exclusive. |
accessProtocol | fc or iscsi | The access protocol to use when accessing the persistent volume. |
Important
• No other parameters are required in the StorageClass
while cloning outside of those parameters listed in the table above.
• Cloning using above parameters is independent of snapshot CRD
availability on Kubernetes and it can be performed on any supported Kubernetes version.
• Support for importVolAsClone
and cloneOf
is available from HPE CSI Driver 1.3.0+.
Creating Snapshots of Persistent Volumes¶
During the snapshotting process, any existing PersistentVolumeClaim
defined in the virtualCopyOf
parameter within a StorageClass
, will be snapped as PersistentVolumeClaim
and exposed through the HPE CSI Driver and made available to the Kubernetes cluster. Volumes with snapshots are immutable once created.
Parameter | Option | Description |
---|---|---|
accessProtocol | fc or iscsi | The access protocol to use when accessing the persistent volume. |
virtualCopyOf | Text | The name of existing PersistentVolumeClaim to be snapped |
Important
• No other parameters are required in the StorageClass
when snapshotting a volume outside of those parameters listed in the table above.
• Snapshotting using virtualCopyOf
is independent of snapshot CRD
availability on Kubernetes and it can be performed on any supported Kubernetes version.
• Support for virtualCopyOf
is available from HPE CSI Driver 1.3.0+.
Remote Copy with Peer Persistence Synchronous Replication Parameters¶
To enable replication within the HPE CSI Driver, the following steps must be completed:
- Create
Secrets
for both primary and target arrays. Refer to Configuring Additional Storage Backends. - Create replication custom resource.
- Create replication enabled
StorageClass
.
For a tutorial on how to enable replication, check out the blog Enabling Remote Copy using the HPE CSI Driver for Kubernetes on HPE Primera
A Custom Resource Definition (CRD) of type hpereplicationdeviceinfos.storage.hpe.com
must be created to define the target array information. The CRD object name will be used to define the StorageClass
parameter replicationDevices. CRD mandatory parameters: targetCpg
, targetName
, targetSecret
and targetSecretNamespace
.
apiVersion: storage.hpe.com/v2
kind: HPEReplicationDeviceInfo
metadata:
name: r1
spec:
target_array_details:
- targetCpg: <cpg_name>
targetSnapCpg: <snapcpg_name> #optional.
targetName: <target_array_name>
targetSecret: <target_secret_name>
targetSecretNamespace: hpe-storage
apiVersion: storage.hpe.com/v1
kind: HPEReplicationDeviceInfo
metadata:
name: r1
spec:
target_array_details:
- targetCpg: <cpg_name>
targetSnapCpg: <snapcpg_name> #optional.
targetName: <target_array_name>
targetSecret: <target_secret_name>
targetSecretNamespace: hpe-storage
Important
The HPE CSI Driver only supports Remote Copy Peer Persistence mode.
These parameters are applicable only for replication. Both parameters are mandatory. If the Remote Copy volume group (RCG) name, as defined within the StorageClass
, does not exist on the array, then a new RCG will be created.
Parameter | Option | Description |
---|---|---|
remoteCopyGroup | Text | Name of new or existing Remote Copy group 1 on the array. |
replicationDevices | Text | Indicates name of hpereplicationdeviceinfos Custom Resource Definition (CRD). |
allowBatchReplicatedVolumeCreation | Boolean | Enable the batch processing of persistent volumes in 10 second intervals and add them to a single Remote Copy group. (Optional) During this process, the Remote Copy group is stopped and started once. |
oneRcgPerPvc | Boolean | Creates a dedicated Remote Copy group per persistent volume. (Optional) |
Remote Copy additional details:
1 = Existing RCG must have CPG and Copy CPG configured.
Link to HPE Primera OS: Configuring data replication using Remote Copy
Important
Remote Copy groups (RCG) created by the HPE CSI driver 2.1 and later have the Auto synchronize and Auto recover policies applied.
To add or remove these policies from RCGs, modify the existing RCG using the SSMC or CLI with the following command:
Add
setrcopygroup pol auto_recover,auto_synchronize <group_name>
Remove
setrcopygroup pol no_auto_recover,no_auto_synchronize <group_name>
Add Non-Replicated Volume to Remote Copy group¶
To add a non-replicated volume to an existing Remote Copy group, allowMutations: description
at minimum must be defined within the StorageClass
. Refer to Remote Copy with Peer Persistence Replication for more details.
Edit the non-replicated PVC and annotate the following parameters:
Parameter | Option | Description |
---|---|---|
remoteCopyGroup | Text | Name of existing Remote Copy group. |
oneRcgPerPvc | Boolean | Creates a dedicated Remote Copy group per persistent volume. (Optional) |
replicationDevices | Text | Indicates name of hpereplicationdeviceinfos Custom Resource Definition (CRD). |
Note
remoteCopyGroup
and oneRcgPerPvc
parameters are mutually exclusive and cannot be added together when editing a PVC
.
Specifying iSCSI Target Portal IPs¶
This parameter allows the ability to specify a subset of the array iSCSI ports for iSCSI sessions. By default, the HPE CSI Driver uses all available iSCSI ports.
Parameter | Option | Description |
---|---|---|
iscsiPortalIps | Text | Comma separated list of target portal IPs. |
Creating a VolumeSnapshotClass¶
These parameters are for VolumeSnapshotClass
objects when using CSI snapshots. The external snapshotter needs to be deployed on the Kubernetes cluster and is usually performed by the Kubernetes vendor. Check enabling CSI snapshots for more information. Volumes with snapshots are immutable.
How to use VolumeSnapshotClass
and VolumeSnapshot
objects is elaborated on in using CSI snapshots.
Parameter | String | Description |
---|---|---|
read_only | Boolean | Indicates if the snapshot is writable on the array. |
Import a Snapshot¶
During the import snapshot process, any legacy (non-container snapshot) or an existing docker/k8s snapshot defined in the ImportVol parameter, within a VolumeSnapshotClass
, will be renamed with the prefix "snapshot-". The new snapshot will be exposed through the HPE CSI Driver and made available to the Kubernetes cluster. Note: All previous Access Control Records and Initiator Groups will be removed from the snapshot when it is imported.
Parameter | Option | Description |
---|---|---|
importVol | Text | The name of the array snapshot to import. |
Creating a QoS StorageClass¶
In the array, QoS rules are applied to a volume set. To use an existing volume set with QoS rules on a PersistentVolumeClaim
, set the qosName
parameter within a StorageClass
to the name of the existing array volume set.
Parameter | Option | Description |
---|---|---|
qosName | Text | Name of the HPE Primera or 3PAR volume set which has QoS rules. This parameter is optional. If specified, the PersistentVolumeClaim will be associated with the array volume set, for purposes of applying the QoS rules. |
Creating a SnapshotGroupClass¶
These parameters are for SnapshotGroupClass
objects when using CSI snapshots. The external snapshotter needs to be deployed on the Kubernetes cluster and is usually performed by the Kubernetes vendor. Check enabling CSI snapshots for more information. Volumes with snapshots are immutable.
How to use VolumeSnapshotClass
and VolumeSnapshot
objects is elaborated on in using CSI snapshots.
Parameter | String | Description |
---|---|---|
read_only | Boolean | Indicates if the snapshot is writable on the array. |
Creating a VolumeGroupClass with QoS Settings¶
In the HPE CSI Driver version 1.4.0+, a volume set with QoS settings can be created dynamically using the QoS parameters for the VolumeGroupClass
. The following parameters are available for a VolumeGroup
on the array. Learn more about VolumeGroups
in the provisioning concepts documentation.
Parameter | String | Description |
---|---|---|
priority | Text | The priority level for the target volume set. Example: "low", "normal", "high" |
ioMinGoal | Text | IOPS minimum goal for the target volume set. Example: "300" |
ioMaxLimit | Text | IOPS maximum limit for the target volume set. Example: "10000" |
bwMinGoalKb | Text | Bandwidth minimum goal in kilobytes per second for the target volume set. Example: "300" |
bwMaxLimitKb | Text | Bandwidth maximum limit in kilobytes per second for the target volume set. Example: "30000" |
latencyGoal | Text | Latency goal in milliseconds (ms) or microseconds(us) for the target volume set. Example: "300ms" or "500us" |
domain | Text | The array Virtual Domain, with which the volume group and related objects are associated with. Example: "sample_domain" |
Note
QoS settings are mandatory when creating a VolumeGroupClass on the array.
Specifying Fibre Channel Ports¶
Use this parameter to specify a subset of Fibre Channel (FC) ports on the array to create vluns. By default, the HPE CSI Driver uses all available FC ports.
Parameter | Option | Description |
---|---|---|
fcPortsList | Text | Comma separated list of available FC ports. |
Support¶
Please refer to the HPE Alletra 9000 and Primera and 3PAR Storage CSP support statement.