Introduction¶

The HPE Alletra Storage MP B10000, Alletra 9000 and Primera and 3PAR Storage Container Storage Provider (CSP) for Kubernetes is part of the HPE CSI Driver for Kubernetes. The CSP abstract the data management capabilities of the array for use by Kubernetes.

Platform Requirements¶

Check the corresponding CSI driver version in the compatibility and support table for the latest updates on supported Kubernetes version, orchestrators and host OS.

Network Port Requirements¶

The HPE Alletra Storage MP B10000, Alletra 9000, Primera and 3PAR Container Storage Provider requires the following TCP ports to be open inbound to the array from the Kubernetes cluster worker nodes running the HPE CSI Driver for Kubernetes.

User Role Requirements¶

The CSP requires access to a local user with either edit or the super role. It's recommended to use the edit role for security best practices.

Virtual Domains¶

Virtual Domains are not yet fully supported by the CSP. From HPE CSI Driver v2.5.0, it's possible to manually create the Kubernetes hosts connecting to storage within the Virtual Domain. Once the hosts have been created, deploy the CSI driver with the Helm chart using the "disableHostDeletion" parameter set to "true". The Virtual Domain user may create the hosts through the Virtual Domain if the "AllowDomainUsersAffectNoDomain" parameter is set to either "hostonly" or "yes" on the array.

Detailed steps to use Virtual Domains¶

These steps assumes access to the storage platform with privileges to create domains and change settings.

Login to the storage platform with SSH. Create an new domain:

cli% createdomain -comment "This is a test domain." my-kubernetes-domain-0

Then, create a new user and assign to the domain. These credentials will be used by the CSI driver.

cli% createuser -c my-password-0 domain-user-0 my-kubernetes-domain-0 edit

Next, make sure domain users are allowed to create hosts outside the domain.

cli% setsys AllowDomainUsersAffectNoDomain hostonly

The next steps involve installing the HPE CSI Driver for Kubernetes with disableHostDeletion set to true. The steps to supply the parameter depends on if the Helm chart or Operator is being used.

• Helm chart install from ArtifactHub.io.
• Operator install for OpenShift.

Once the CSI driver is installed and running, add an HPE storage backend with the credentials provided in the steps above.

VLUN Templates¶

A VLUN template enables the export of a virtual volume as a VLUN to hosts. For more information, see the HPE Primera OS Commmand Line Interface - Installation and Reference Guide.

The CSP supports the following types of VLUN templates:

The boolean string "hostSeesVLUN" StorageClass parameter controls which VLUN template to use.

Change VLUN Template for existing PVCs¶

To modify an existing PVC, "hostSeesVLUN" needs to be specified with the "allowMutations" parameter along with adding the PVC annotation "csi.hpe.com/hostSeesVLUN" with the string values of either "true" or "false". The HPE CSI Driver creates the VLUN template based upon the hostSeesVLUN parameter during the volume publish operation. For the change to take effect, the Pod will need to be scheduled on another node by either deleting the Pod or draining the node.

StorageClass Parameters¶

All parameters enumerated reflects the current version and may contain unannounced features and capabilities.

Common Provisioning Parameters¶

Restrictions applicable when using the CSI volume mutator:
¹ = Parameters that are editable after provisioning.
² = Volumes with snapshots/clones can't be modified.
³ = HPE 3PAR only parameter
⁴ = HPE Primera/Alletra 9000 only parameter

Please see using the HPE CSI Driver for additional StorageClass examples like CSI snapshots and clones.

Cloning Parameters¶

Cloning supports two modes of cloning. Either use cloneOf and reference a PersistentVolumeClaim in the current namespace to clone or use importVolAsClone and reference an array volume name to clone and import into the Kubernetes cluster. Volumes with clones are immutable once created.

Array Snapshot Parameters¶

During the snapshotting process, any existing PersistentVolumeClaim defined in the virtualCopyOf parameter within a StorageClass, will be snapped as PersistentVolumeClaim and exposed through the HPE CSI Driver and made available to the Kubernetes cluster. Volumes with snapshots are immutable once created.

Import Parameters¶

During the import volume process, any legacy (non-container volumes) defined in the ImportVol parameter, within a StorageClass, will be renamed to match the PersistentVolumeClaim that leverages the StorageClass. The new volumes will be exposed through the HPE CSI Driver and made available to the Kubernetes cluster. Note: All previous Access Control Records and Initiator Groups will be removed from the volume when it is imported.

Remote Copy with Peer Persistence Synchronous Replication Parameters¶

To enable replication within the HPE CSI Driver, the following steps must be completed:

• Create Secrets for both primary and target arrays. Refer to Configuring Additional Storage Backends.
• Create replication custom resource.
• Create replication enabled StorageClass.

For a tutorial on how to enable replication, check out the blog Enabling Remote Copy using the HPE CSI Driver for Kubernetes on HPE Primera

A Custom Resource Definition (CRD) of type hpereplicationdeviceinfos.storage.hpe.com must be created to define the target array information. The CRD object name will be used to define the StorageClass parameter replicationDevices. CRD mandatory parameters: targetCpg, targetName, targetSecret and targetSecretNamespace.

apiVersion: storage.hpe.com/v2
kind: HPEReplicationDeviceInfo
metadata:
  name: r1
spec:
  target_array_details:
  - targetCpg: <cpg_name>
    targetSnapCpg: <snapcpg_name> #optional.
    targetName: <target_array_name>
    targetSecret: <target_secret_name>
    targetSecretNamespace: hpe-storage

apiVersion: storage.hpe.com/v1
kind: HPEReplicationDeviceInfo
metadata:
  name: r1
spec:
  target_array_details:
  - targetCpg: <cpg_name>
    targetSnapCpg: <snapcpg_name> #optional.
    targetName: <target_array_name>
    targetSecret: <target_secret_name>
    targetSecretNamespace: hpe-storage

These parameters are applicable only for replication. Both parameters are mandatory. If the Remote Copy volume group (RCG) name, as defined within the StorageClass, does not exist on the array, then a new RCG will be created.

Remote Copy additional details:
¹ = Existing RCG must have CPG and Copy CPG configured.
Link to HPE Primera OS: Configuring data replication using Remote Copy

Add Non-Replicated Volume to Remote Copy group¶

To add a non-replicated volume to an existing Remote Copy group, allowMutations: description at minimum must be defined within the StorageClass. Refer to Remote Copy with Peer Persistence Replication for more details.

Edit the non-replicated PVC and annotate the following parameters:

VolumeSnapshotClass Parameters¶

These parameters are for VolumeSnapshotClass objects when using CSI snapshots. The external snapshotter needs to be deployed on the Kubernetes cluster and is usually performed by the Kubernetes vendor. Check enabling CSI snapshots for more information. Volumes with snapshots are immutable.

How to use VolumeSnapshotClass and VolumeSnapshot objects is elaborated on in using CSI snapshots.

VolumeGroupClass Parameters¶

In the HPE CSI Driver version 1.4.0+, a volume set with QoS settings can be created dynamically using the QoS parameters for the VolumeGroupClass. The following parameters are available for a VolumeGroup on the array. Learn more about VolumeGroups in the provisioning concepts documentation.

Example:

apiVersion: storage.hpe.com/v1
kind: VolumeGroupClass
metadata:
  name: my-volume-group-class
provisioner: csi.hpe.com
deletionPolicy: Delete
parameters:
  description: "HPE CSI Driver for Kubernetes Volume Group"
  csi.hpe.com/volume-group-provisioner-secret-name: hpe-backend
  csi.hpe.com/volume-group-provisioner-secret-namespace: hpe-storage
  priority: normal
  ioMinGoal: "300"
  ioMaxLimit: "10000"
  bwMinGoalKb: "3000"
  bwMaxLimitKb: "30000"
  latencyGoal: "300ms"

SnapshotGroupClass Parameters¶

These parameters are for SnapshotGroupClass objects when using CSI snapshots. The external snapshotter needs to be deployed on the Kubernetes cluster and is usually performed by the Kubernetes vendor. Check enabling CSI snapshots for more information. Volumes with snapshots are immutable.

How to use VolumeSnapshotClass and VolumeSnapshot objects is elaborated on in using CSI snapshots.

Static Provisioning¶

Static provisioning of PVs and PVCs may be used when absolute control over physical volumes are required by the storage administrator. This CSP also supports importing volumes and clones of volumes using the import parameters in a StorageClass.

Prerequisites¶

The CSP expects a certain naming convention for PersistentVolumes and Virtual Volumes on the array.

• Persistent Volume: pvc-00000000-0000-0000-0000-000000000000
• Virtual Volume: pvc-00000000-0000-0000-0000-000

The following example uses the above scheme as a naming convention. Have a storage administrator rename the existing Virtual Volume on the array:

setvv -name pvc-00000000-0000-0000-0000-000 my-existing-virtual-volume

HPEVolumeInfo¶

Create a new HPEVolumeInfo resource.

apiVersion: storage.hpe.com/v2
kind: HPEVolumeInfo
metadata:
  name: pvc-00000000-0000-0000-0000-000000000000
spec:
  record:
    Id: pvc-00000000-0000-0000-0000-000000000000
    Name: pvc-00000000-0000-0000-0000-000
  uuid: pvc-00000000-0000-0000-0000-000000000000

Persistent Volume¶

Create a PV referencing the HPEVolumeInfo resource.

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pvc-00000000-0000-0000-0000-000000000000
spec:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 16Gi
  csi:
    volumeHandle: pvc-00000000-0000-0000-0000-000000000000
    driver: csi.hpe.com
    fsType: xfs
    volumeAttributes:
      volumeAccessMode: mount
      fsType: xfs
    controllerPublishSecretRef:
      name: hpe-backend
      namespace: hpe-storage
    nodePublishSecretRef:
      name: hpe-backend
      namespace: hpe-storage
    controllerExpandSecretRef:
      name: hpe-backend
      namespace: hpe-storage
  persistentVolumeReclaimPolicy: Retain
  volumeMode: Filesystem

Persistent Volume Claim¶

Now, a user may claim the static PV by creating a PVC referencing the PV name in .spec.volumeName.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 16Gi
  volumeName: my-static-pv-1
  storageClassName: ""

Remote Copy Limitations¶

These are the current limitations of the Remote Copy Peer Persistence integration with the HPE CSI Driver.

• Only what is considered Classic Peer Persistence is supported. Active/Active hostset proximity is not supported.
• Peer Persistence does not provide disaster recovery for workloads running on Kubernetes. Peer Persistence provide disaster recovery for the storage system.
• Peer Persistence only provide data path resilience. If the primary array is unreachable for the CSP or the role of the remote copy group has changed due to disaster recovery operations (manual or automatic switchover/failover), all CSI operations will cease to function until the primary array comes back up and the role of the remote copy groups returned to original state.
• When the primary array is unavailable for the Kubernetes cluster and remote copy group has failed over to the secondary array successfully, running workloads will continue to run if the host the workload was running on has redundant data paths to the secondary array (current primary array).
• It's possible to access volumes from the secondary array by statically provisioning PersistentVolumes without renaming the volume on the array. This is only safe if it has been determined that the primary array does not have active hosts accessing the volume against the primary array.

Support¶

Please refer to the HPE Alletra Storage MP B10000, Alletra 9000 and Primera and 3PAR Storage CSP support statement.