Deploy or download the SCC:

oc apply -f https://scod.hpedev.io/partners/redhat_openshift/examples/scc/hpe-csi-scc.yaml
securitycontextconstraints.security.openshift.io/hpe-csi-controller-scc created
securitycontextconstraints.security.openshift.io/hpe-csi-node-scc created
securitycontextconstraints.security.openshift.io/hpe-csi-csp-scc created
securitycontextconstraints.security.openshift.io/hpe-csi-nfs-scc created

OpenShift web console¶

Once the SCC has been applied to the project, login to the OpenShift web console as kube:admin and navigate to Operators -> OperatorHub.

Search for 'HPE CSI' in the search field and select the non-marketplace version.

Click 'Install'.

Select the Namespace where the SCC was applied, select 'Manual' Update Approval, click 'Install'.

Click 'Approve' to finalize installation of the Operator

The HPE CSI Operator is now installed, select 'View Operator'.

Click 'Create Instance'.

Normally, no customizations are needed, scroll all the way down and click 'Create'.

By navigating to the Developer view, it should now be possible to inspect the CSI driver and Operator topology.

The CSI driver is now ready for use. Next, an HPE storage backend needs to be added along with a StorageClass.

See Caveats below for information on creating StorageClasses in Red Hat OpenShift.

OpenShift CLI¶

This provides an example Operator deployment using oc. If you want to use the web console, proceed to the previous section.

It's assumed the SCC has been applied to the project and have kube:admin privileges. As an example, we'll deploy to the hpe-storage project as described in previous steps.

First, an OperatorGroup needs to be created.

apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: hpe-csi-driver-for-kubernetes
  namespace: hpe-storage
spec:
  targetNamespaces:
  - hpe-storage

Next, create a Subscription to the Operator.

apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: hpe-csi-operator
  namespace: hpe-storage
spec:
  channel: stable
  installPlanApproval: Manual
  name: hpe-csi-operator
  source: certified-operators
  sourceNamespace: openshift-marketplace

Next, approve the installation.

oc -n hpe-storage patch $(oc get installplans -n hpe-storage -o name) -p '{"spec":{"approved":true}}' --type merge

The Operator will now be installed on the OpenShift cluster. Before instantiating a CSI driver, watch the roll-out of the Operator.

oc rollout status deploy/hpe-csi-driver-operator -n hpe-storage
Waiting for deployment "hpe-csi-driver-operator" rollout to finish: 0 of 1 updated replicas are available...
deployment "hpe-csi-driver-operator" successfully rolled out

The next step is to create a HPECSIDriver object.

# oc apply -n hpe-storage -f https://scod.hpedev.io/csi_driver/examples/deployment/hpecsidriver-v2.5.1-sample.yaml
apiVersion: storage.hpe.com/v1
kind: HPECSIDriver
metadata:
  name: hpecsidriver-sample
spec:
  # Default values copied from <project_dir>/helm-charts/hpe-csi-driver/values.yaml
  controller:
    affinity: {}
    labels: {}
    nodeSelector: {}
    resources:
      limits:
        cpu: 2000m
        memory: 1Gi
      requests:
        cpu: 100m
        memory: 128Mi
    tolerations: []
  csp:
    affinity: {}
    labels: {}
    nodeSelector: {}
    resources:
      limits:
        cpu: 2000m
        memory: 1Gi
      requests:
        cpu: 100m
        memory: 128Mi
    tolerations: []
  disable:
    alletra6000: false
    alletra9000: false
    alletraStorageMP: false
    nimble: false
    primera: false
  disableHostDeletion: false
  disableNodeConfiguration: false
  disableNodeConformance: false
  disableNodeGetVolumeStats: false
  disableNodeMonitor: false
  imagePullPolicy: IfNotPresent
  images:
    csiAttacher: registry.k8s.io/sig-storage/csi-attacher:v4.6.1
    csiControllerDriver: quay.io/hpestorage/csi-driver:v2.5.0
    csiExtensions: quay.io/hpestorage/csi-extensions:v1.2.7
    csiNodeDriver: quay.io/hpestorage/csi-driver:v2.5.0
    csiNodeDriverRegistrar: registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.10.1
    csiProvisioner: registry.k8s.io/sig-storage/csi-provisioner:v5.0.1
    csiResizer: registry.k8s.io/sig-storage/csi-resizer:v1.11.1
    csiSnapshotter: registry.k8s.io/sig-storage/csi-snapshotter:v8.0.1
    csiVolumeGroupProvisioner: quay.io/hpestorage/volume-group-provisioner:v1.0.6
    csiVolumeGroupSnapshotter: quay.io/hpestorage/volume-group-snapshotter:v1.0.6
    csiVolumeMutator: quay.io/hpestorage/volume-mutator:v1.3.6
    nfsProvisioner: quay.io/hpestorage/nfs-provisioner:v3.0.5
    nimbleCSP: quay.io/hpestorage/alletra-6000-and-nimble-csp:v2.5.0
    primera3parCSP: quay.io/hpestorage/alletra-9000-primera-and-3par-csp:v2.5.0
  iscsi:
    chapSecretName: ""
  kubeletRootDir: /var/lib/kubelet
  logLevel: info
  node:
    affinity: {}
    labels: {}
    nodeSelector: {}
    resources:
      limits:
        cpu: 2000m
        memory: 1Gi
      requests:
        cpu: 100m
        memory: 128Mi
    tolerations: []

# oc apply -n hpe-storage -f https://scod.hpedev.io/csi_driver/examples/deployment/hpecsidriver-v2.4.2-sample.yaml
apiVersion: storage.hpe.com/v1
kind: HPECSIDriver
metadata:
  name: hpecsidriver-sample
spec:
  # Default values copied from <project_dir>/helm-charts/hpe-csi-driver/values.yaml
  controller:
    affinity: {}
    labels: {}
    nodeSelector: {}
    tolerations: []
  csp:
    affinity: {}
    labels: {}
    nodeSelector: {}
    tolerations: []
  disable:
    alletra6000: false
    alletra9000: false
    alletraStorageMP: false
    nimble: false
    primera: false
  disableNodeConfiguration: false
  disableNodeConformance: false
  disableNodeGetVolumeStats: false
  imagePullPolicy: IfNotPresent
  iscsi:
    chapPassword: ""
    chapUser: ""
  kubeletRootDir: /var/lib/kubelet/
  logLevel: info
  node:
    affinity: {}
    labels: {}
    nodeSelector: {}
    tolerations: []
  registry: quay.io

# oc apply -n hpe-storage -f https://scod.hpedev.io/csi_driver/examples/deployment/hpecsidriver-v2.4.1-sample.yaml
apiVersion: storage.hpe.com/v1
kind: HPECSIDriver
metadata:
  name: hpecsidriver-sample
spec:
  # Default values copied from <project_dir>/helm-charts/hpe-csi-driver/values.yaml
  controller:
    affinity: {}
    labels: {}
    nodeSelector: {}
    tolerations: []
  csp:
    affinity: {}
    labels: {}
    nodeSelector: {}
    tolerations: []
  disable:
    alletra6000: false
    alletra9000: false
    alletraStorageMP: false
    nimble: false
    primera: false
  disableNodeConfiguration: false
  disableNodeConformance: false
  disableNodeGetVolumeStats: false
  imagePullPolicy: IfNotPresent
  iscsi:
    chapPassword: ""
    chapUser: ""
  kubeletRootDir: /var/lib/kubelet/
  logLevel: info
  node:
    affinity: {}
    labels: {}
    nodeSelector: {}
    tolerations: []
  registry: quay.io

The CSI driver is now ready for use. Next, an HPE storage backend needs to be added along with a StorageClass.

Additional information¶

At this point the CSI driver is managed like any other Operator on Kubernetes and the life-cycle management capabilities may be explored further in the official Red Hat OpenShift documentation.

Uninstall the HPE CSI Operator¶

When uninstalling an operator managed by OLM, a Cluster Admin must decide whether or not to remove the CustomResourceDefinitions (CRD), APIServices, and resources related to these types owned by the operator. By design, when OLM uninstalls an operator it does not remove any of the operator’s owned CRDs, APIServices, or CRs in order to prevent data loss.

The following are CRDs installed by the HPE CSI driver.

hpecsidrivers.storage.hpe.com
hpenodeinfos.storage.hpe.com
hpereplicationdeviceinfos.storage.hpe.com
hpesnapshotgroupinfos.storage.hpe.com
hpevolumegroupinfos.storage.hpe.com
hpevolumeinfos.storage.hpe.com
snapshotgroupclasses.storage.hpe.com
snapshotgroupcontents.storage.hpe.com
snapshotgroups.storage.hpe.com
volumegroupclasses.storage.hpe.com
volumegroupcontents.storage.hpe.com
volumegroups.storage.hpe.com

The following are APIServices installed by the HPE CSI driver.

v1.storage.hpe.com
v2.storage.hpe.com

Please refer to the OLM Lifecycle Manager documentation on how to safely Uninstall your operator.

NFS Server Provisioner Considerations¶

When deploying NFS servers on OpenShift there's currently two things to keep in mind for a successful deployment. Also, be understood with the Limitations and Considerations for the NFS Server Provisioner in general.

Non-standard hpe-nfs Namespace¶

If NFS servers are deployed in a different Namespace than the default "hpe-nfs" by using the "nfsNamespace" StorageClass parameter, the "hpe-csi-nfs-scc" SCC needs to be updated to include the Namespace ServiceAccount.

This example adds "my-namespace" NFS server ServiceAccount to the SCC:

oc patch scc hpe-csi-nfs-scc --type=json -p='[{"op": "add", "path": "/users/-", "value": "system:serviceaccount:my-namespace:hpe-csi-nfs-sa" }]'

Operators Requesting NFS Persistent Volume Claims¶

Object references in OpenShift are not compatible with the NFS Server Provisioner. If a user deploys an Operator of any kind that creates a NFS server backed PVC, the operation will fail. Instead, pre-provision the PVC manually for the Operator instance to use.

Use the ext4 filesystem for NFS servers¶

On certain versions of OpenShift the NFS clients may experience stale NFS file handles like the one below when the NFS server is being restarted.

Error: failed to resolve symlink "/var/lib/kubelet/pods/290ff9e1-cc1e-4d05-b884-0ddcc05a9631/volumes/kubernetes.io~csi/pvc-321cf523-c063-4ce4-97e8-bc1365b8a05b/mount": lstat /var/lib/kubelet/pods/290ff9e1-cc1e-4d05-b884-0ddcc05a9631/volumes/kubernetes.io~csi/pvc-321cf523-c063-4ce4-97e8-bc1365b8a05b/mount: stale NFS file handle

If this problem occurs, use the ext4 filesystem on the backing volumes. The fsType is set in the StorageClass. Example:

...
parameters:
  csi.storage.k8s.io/fstype: ext4
...

StorageProfile for OpenShift Virtualization Source PVCs¶

If OpenShift Virtualization is being used and Live Migration is desired for virtual machines PVCs cloned from the "openshift-virtualization-os-images" Namespace, the StorageProfile needs to be updated to "ReadWriteMany".

If the default StorageClass is named "hpe-standard", issue the following command:

oc edit -n openshift-cnv storageprofile hpe-standard

Replace the spec: {} with the following:

spec:
  claimPropertySets:
  - accessModes:
    - ReadWriteMany
    volumeMode: Block

Ensure there are no errors. Recreate the OS images:

oc delete pvc -n openshift-virtualization-os-images --all

Inspect the PVCs and ensure they are re-created with "RWX":

oc get pvc -n openshift-virtualization-os-images -w

Live VM migrations for Alletra Storage MP B10000¶

With HPE CSI Operator for Kubernetes v2.4.2 and older there's an issue that prevents live migrations of VMs that has PVCs attached that has been clones from an OS image residing on Alletra Storage MP B10000 backends including 3PAR, Primera and Alletra 9000.

Identify the PVC that that has been cloned from an OS image. The VM name is "centos7-silver-bedbug-14" in this case.

oc get vm/centos7-silver-bedbug-14 -o jsonpath='{.spec.template.spec.volumes}' | jq

In this instance, the dataVolume is the same name as the VM. Grab the PV name from the PVC name.

MY_PV_NAME=$(oc get pvc/centos7-silver-bedbug-14 -o jsonpath='{.spec.volumeName}')

Next, patch the hpevolumeinfo CRD.

oc patch hpevolumeinfo/${MY_PV_NAME} --type=merge --patch '{"spec": {"record": {"MultiInitiator": "true"}}}'

The VM is now ready to be migrated.

Unsupported Version of the Operator Install¶

In the event on older version of the Operator needs to be installed, the bundle can be installed directly by installing the Operator SDK. Make sure a recent version of the operator-sdk binary is available and that no HPE CSI Driver is currently installed on the cluster.

Install a specific version prior and including v2.4.2:

operator-sdk run bundle --timeout 5m -n hpe-storage quay.io/hpestorage/csi-driver-operator-bundle:v2.4.2

Install a specific version after and including v2.5.0:

operator-sdk run bundle --timeout 5m -n hpe-storage quay.io/hpestorage/csi-driver-operator-bundle-ocp:v2.5.0

When the unsupported install isn't needed any longer, run:

operator-sdk cleanup -n hpe-storage hpe-csi-operator

Unsupported Helm Chart Install¶

In the event Red Hat releases a new version of OpenShift between HPE CSI Driver releases or if interest arises to run the HPE CSI Driver on an uncertified version of OpenShift, it's possible to install the CSI driver using the Helm chart instead.

It's not recommended to install the Helm chart unless it's listed as "Field Tested" in the support matrix above.

Steps to install.¶

• Follow the steps in the prerequisites to apply the SCC in the Namespace (Project) you wish to install the driver.
• Install the Helm chart with the steps provided on ArtifactHub. Pay attention to which version combination has been field tested.