Overview

The HPE Volume Driver for Kubernetes FlexVolume Plugin leverages HPE Nimble Storage or HPE Cloud Volumes to provide scalable and persistent storage for stateful applications.

Important

Using HPE Nimble Storage with Kubernetes 1.13 and newer, please use the HPE CSI Driver for Kubernetes.

Source code and developer documentation is available in the hpe-storage/flexvolume-driver GitHub repo.

Platform requirements

The FlexVolume driver supports multiple backends that are based on a "container provider" architecture. Currently, Nimble and Cloud Volumes are supported.

HPE Nimble Storage Platform Requirements

Driver HPE Nimble Storage Version Release Notes Blog
v3.0.0 5.0.8.x and 5.1.3.x onwards v3.0.0 HPE Storage Tech Insiders
v3.1.0 5.0.8.x and 5.1.3.x onwards v3.1.0
  • OpenShift Container Platform 3.9, 3.10 and 3.11.
  • Kubernetes 1.10 and above.
  • Redhat/CentOS 7.5+
  • Ubuntu 16.04/18.04 LTS

Note: Synchronous replication (Peer Persistence) is not supported by the HPE Volume Driver for Kubernetes FlexVolume Plugin.

HPE Cloud Volumes Platform Requirements

Driver Release Notes Blog
v3.1.0 v3.1.0 Using HPE Cloud Volumes with Amazon EKS
  • Amazon EKS 1.12/1.13
  • Microsoft Azure AKS 1.12/1.13
  • US regions only

Deploying to Kubernetes

The recommended way to deploy and manage the HPE Volume Driver for Kubernetes FlexVolume Plugin is to use Helm. Please see the co-deployments repository for further information.

Use the following steps for a manual installation.

Step 1: Create a secret

HPE Nimble Storage

Replace the password string (YWRtaW4=) below with a base64 encoded version of your password and replace the backend with your array IP address and save it as hpe-secret.yaml.

apiVersion: v1
kind: Secret
metadata:
  name: hpe-secret
  namespace: kube-system
stringData:
  backend: 192.168.1.1
  username: admin
  protocol: "iscsi"
data:
  # echo -n "admin" | base64
  password: YWRtaW4=

HPE Cloud Volumes

Replace the username and password strings (YWRtaW4=) with a base64 encoded version of your HPE Cloud Volumes "access_key" and "access_secret". Also, replace the backend with HPE Cloud Volumes portal fully qualified domain name (FQDN) and save it as hpe-secret.yaml.

apiVersion: v1
kind: Secret
metadata:
  name: hpe-secret
  namespace: kube-system
stringData:
  backend: cloudvolumes.hpe.com
  protocol: "iscsi"
  serviceName: cv-cp-svc
  servicePort: "8080"
data:
  # echo -n "<my very confidential access key>" | base64
  username: YWRtaW4=
  # echo -n "<my very confidential secret key>" | base64
  password: YWRtaW4=

Create the secret:

kubectl create -f hpe-secret.yaml
secret "hpe-secret" created

You should now see the HPE secret in the kube-system namespace.

kubectl get secret/hpe-secret -n kube-system
NAME                  TYPE                                  DATA      AGE
hpe-secret            Opaque                                5         3s

Step 2. Create a ConfigMap

The ConfigMap is used to set and tweak defaults for both the FlexVolume driver and Dynamic Provisioner.

HPE Nimble Storage

Edit the below default parameters as required for FlexVolume driver and save it as hpe-config.yaml.

kind: ConfigMap
apiVersion: v1
metadata:
  name: hpe-config
  namespace: kube-system
data:
  volume-driver.json: |-
    {
      "global":   {},
      "defaults": {
                 "limitIOPS":"-1",
                 "limitMBPS":"-1",
                 "perfPolicy": "Other"
                },
      "overrides":{}
    }

Tip

Please see Advanced for more volume-driver.json configuration options.

HPE Cloud Volumes

Edit the below parameters as required with your public cloud info and save it as hpe-config.yaml.

kind: ConfigMap
apiVersion: v1
metadata:
  name: hpe-config
  namespace: kube-system
data:
  volume-driver.json: |-
    {
      "global": {
                "snapPrefix": "BaseFor",
                "initiators": ["eth0"],
                "automatedConnection": true,
                "existingCloudSubnet": "10.1.0.0/24",
                "region": "us-east-1",
                "privateCloud": "vpc-data",
                "cloudComputeProvider": "Amazon AWS"
      },
      "defaults": {
                "limitIOPS": 1000,
                "fsOwner": "0:0",
                "fsMode": "600",
                "description": "Volume provisioned by the HPE Volume Driver for Kubernetes FlexVolume Plugin",
                "perfPolicy": "Other",
                "protectionTemplate": "twicedaily:4",
                "encryption": true,
                "volumeType": "PF",
                "destroyOnRm": true
      },
      "overrides": {
      }
    }

Create the ConfigMap:

kubectl create -f hpe-config.yaml
configmap/hpe-config created

Step 3. Deploy the FlexVolume driver and dynamic provisioner

Deploy the driver as a DaemonSet and the dynamic provisioner as a Deployment.

HPE Nimble Storage

Version 3.0.0:

kubectl create -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/flexvolume-driver/hpe-nimble-storage/hpe-flexvolume-driver-v3.0.0.yaml

Version 3.1.0:

kubectl create -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/flexvolume-driver/hpe-nimble-storage/hpe-flexvolume-driver-v3.1.0.yaml

HPE Cloud Volumes

Container-Provider Service:

kubectl create -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/flexvolume-driver/hpe-cloud-volumes/hpecv-cp-v3.1.0.yaml

The FlexVolume driver have different declarations depending on the Kubernetes distribution.

Amazon EKS:

kubectl create -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/flexvolume-driver/hpe-cloud-volumes/hpecv-aws-flexvolume-driver-v3.1.0.yaml

Microsoft Azure AKS:

kubectl create -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/flexvolume-driver/hpe-cloud-volumes/hpecv-azure-flexvolume-driver-v3.1.0.yaml

Generic:

kubectl create -f https://raw.githubusercontent.com/hpe-storage/co-deployments/master/yaml/flexvolume-driver/hpe-cloud-volumes/hpecv-flexvolume-driver-v3.1.0.yaml

Note

The declarations for HPE Volume Driver for Kubernetes FlexVolume Plugin can be found in the co-deployments repository.

Check to see all hpe-flexvolume-driver Pods (one per compute node) and the hpe-dynamic-provisioner Pod are running.

kubectl get pods -n kube-system
NAME                                            READY   STATUS    RESTARTS   AGE
hpe-flexvolume-driver-2rdt4                     1/1     Running   0          45s
hpe-flexvolume-driver-md562                     1/1     Running   0          44s
hpe-flexvolume-driver-x4k96                     1/1     Running   0          44s
hpe-dynamic-provisioner-59f9d495d4-hxh29        1/1     Running   0          24s

For HPE Cloud Volumes, check that hpe-cv-cp pod is running as well.

kubectl get pods -n kube-system -l=app=cv-cp
NAME                                READY   STATUS    RESTARTS   AGE
hpe-cv-cp-2rdt4                     1/1     Running   0          45s

Using

Get started using the FlexVolume driver by setting up StorageClass, PVC API objects. See Using for examples.

These instructions are provided as an example on how to use the HPE Volume Driver for Kubernetes FlexVolume Plugin with a HPE Nimble Storage Array.

The below YAML declarations are meant to be created with kubectl create. Either copy the content to a file on the host where kubectl is being executed, or copy & paste into the terminal, like this:

kubectl create -f-
< paste the YAML >
^D (CTRL + D)

Tip

Some of the examples supported by the HPE Volume Driver for Kubernetes FlexVolume Plugin are available for HPE Nimble Storage or HPE Cloud Volumes in the GitHub repo.

To get started, create a StorageClass API object referencing the hpe-secret and defining additional (optional) StorageClass parameters:

Sample StorageClass

Sample storage classes can be found for HPE Nimble Storage and HPE Cloud Volumes.

Hint

See StorageClass parameters for HPE Nimble Storage and HPE Clound Volumes for a comprehensive overview.

Test and verify volume provisioning

Create a StorageClass with volume parameters as required.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: sc-nimble
provisioner: hpe.com/nimble
parameters:
  description: "Volume from HPE FlexVolume driver"
  perfPolicy: "Other Workloads"
  limitIOPS: "76800"

Create a PersistentVolumeClaim. This makes sure a volume is created and provisioned on your behalf:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-nimble
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: sc-nimble

Check that a new PersistentVolume is created based on your claim:

kubectl get pv
NAME                                            CAPACITY     ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM               STORAGECLASS   REASON   AGE
sc-nimble-13336da3-7ca3-11e9-826c-00505693581f  10Gi         RWO            Delete           Bound    default/pvc-nimble  sc-nimble               3s

The above output means that the FlexVolume driver successfully provisioned a new volume and bound to the requesting PVC to a new PV. The volume is not attached to any node yet. It will only be attached to a node if a workload is scheduled to a specific node. Now let us create a Pod that refers to the above volume. When the Pod is created, the volume will be attached, formatted and mounted to the specified container:

kind: Pod
apiVersion: v1
metadata:
  name: pod-nimble
spec:
  containers:
    - name: pod-nimble-con-1
      image: nginx
      command: ["bin/sh"]
      args: ["-c", "while true; do date >> /data/mydata.txt; sleep 1; done"]
      volumeMounts:
        - name: export1
          mountPath: /data
    - name: pod-nimble-cont-2
      image: debian
      command: ["bin/sh"]
      args: ["-c", "while true; do date >> /data/mydata.txt; sleep 1; done"]
      volumeMounts:
        - name: export1
          mountPath: /data
  volumes:
    - name: export1
      persistentVolumeClaim:
        claimName: pvc-nimble

Check if the pod is running successfully:

kubectl get pod pod-nimble
NAME         READY   STATUS    RESTARTS   AGE
pod-nimble   2/2     Running   0          2m29s

Use case specific examples

This StorageClass examples help guide combinations of options when provisioning volumes.

Data protection

This StorageClass creates thinly provisioned volumes with deduplication turned on. It will also apply the Performance Policy "SQL Server" along with a Protection Template. The Protection Template needs to be defined on the array.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
    name: oltp-prod
provisioner: hpe.com/nimble
parameters:
    thick: "false"
    dedupe: "true"
    perfPolicy: "SQL Server"
    protectionTemplate: "Retain-48Hourly-30Daily-52Weekly"

Clone and throttle for devs

This StorageClass will create clones of a "production" volume and throttle the performance of each clone to 1000 IOPS. When the PVC is deleted, it will be permanently deleted from the backend array.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: oltp-dev-clone-of-prod
provisioner: hpe.com/nimble
parameters:
  limitIOPS: "1000"
  cloneOf: "oltp-prod-1adee106-110b-11e8-ac84-00505696c45f"
  destroyOnRm: "true"

Clone a non-containerized volume

This StorageClass will clone a standard backend volume (without container metadata on it) from a particular pool on the backend.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: import-clone-legacy-prod
rovisioner: hpe.com/nimble
parameters:
  pool: "flash"
  importVolAsClone: "production-db-vol"
  destroyOnRm: "true"

Import (cutover) a volume

This StorageClass will import an existing Nimble volume to Kubernetes. The source volume needs to be offline for the import to succeed.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: import-clone-legacy-prod
provisioner: hpe.com/nimble
parameters:
  pool: "flash"
  importVol: "production-db-vol"

Using overrides

The HPE Dynamic Provisioner for Kubernetes understands a set of annotation keys a user can set on a PVC. If the corresponding keys exists in the list of the allowOverrides key in the StorageClass, the end-user can tweak certain aspects of the provisioning workflow. This opens up for very advanced data services.

StorageClass object:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: my-sc
provisioner: hpe.com/nimble
parameters:
  description: "Volume provisioned by StorageClass my-sc"
  dedupe: "false"
  destroyOnRm: "true"
  perfPolicy: "Windows File Server"
  folder: "myfolder"
  allowOverrides: snapshot,limitIOPS,perfPolicy

PersistentVolumeClaim object:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
 name: my-pvc
 annotations:
    hpe.com/description: "This is my custom description"
    hpe.com/limitIOPS: "8000"
    hpe.com/perfPolicy: "SQL Server"
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: my-sc

This will create a PV of 8000 IOPS with the Performance Policy of "SQL Server" and a custom volume description.

Creating clones of PVCs

Using a StorageClass to clone a PV is practical when there's needs to clone across namespaces (for example from prod to test or stage). If a user wants to clone any arbitrary volume, it becomes a bit tedious to create a StorageClass for each clone. The annotation hpe.com/CloneOfPVC allows a user to clone any PVC within a namespace.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
 name: my-pvc-clone
 annotations:
    hpe.com/cloneOfPVC: my-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: my-sc

StorageClass parameters

This section highlights all the available StorageClass parameters that are supported.

HPE Nimble Storage StorageClass parameters

A StorageClass is used to provision or clone an HPE Nimble Storage-backed persistent volume. It can also be used to import an existing HPE Nimble Storage volume or clone of a snapshot into the Kubernetes cluster. The parameters are grouped below by those same workflows.

A sample StorageClass is provided.

Note

These are optional parameters.

Common parameters for Provisioning and Cloning

These parameters are mutable betweeen a parent volume and creating a clone from a snapshot.

Parameter String Description
nameSuffix Text Suffix to append to Nimble volumes. Defaults to .docker
destroyOnRm Boolean Indicates the backing Nimble volume (including snapshots) should be destroyed when the PVC is deleted.
limitIOPS Integer The IOPS limit of the volume. The IOPS limit should be in the range 256 to 4294967294, or -1 for unlimited (default).
limitMBPS Integer The MB/s throughput limit for the volume.
description Text Text to be added to the volume's description on the Nimble array.
perfPolicy Text The name of the performance policy to assign to the volume. Default example performance policies include "Backup Repository", "Exchange 2003 data store", "Exchange 2007 data store", "Exchange 2010 data store", "Exchange log", "Oracle OLTP", "Other Workloads", "SharePoint", "SQL Server", "SQL Server 2012", "SQL Server Logs".
protectionTemplate Text The name of the protection template to assign to the volume. Default examples of protection templates include "Retain-30Daily", "Retain-48Hourly-30aily-52Weekly", and "Retain-90Daily".
folder Text The name of the Nimble folder in which to place the volume.
thick Boolean Indicates that the volume should be thick provisioned.
dedupeEnabled Boolean Indicates that the volume should enable deduplication.
syncOnUnmount Boolean Indicates that a snapshot of the volume should be synced to the replication partner each time it is detached from a node.

Note

Performance Policies, Folders and Protection Templates are Nimble specific constructs that can be created on the Nimble array itself to address particular requirements or workloads. Please consult with the storage admin or read the admin guide found on HPE InfoSight.

Provisioning parameters

These parameters are immutable for clones once a volume has been created.

Parameter String Description
fsOwner userId:groupId The user id and group id that should own the root directory of the filesystem.
fsMode Octal digits 1 to 4 octal digits that represent the file mode to be applied to the root directory of the filesystem.
encryption Boolean Indicates that the volume should be encrypted.
pool Text The name of the pool in which to place the volume.

Cloning parameters

Cloning supports two modes of cloning. Either use cloneOf and reference a PVC in the current namespace or use importVolAsClone and reference a Nimble volume name to clone and import to Kubernetes.

Parameter String Description
cloneOf Text The name of the PV to be cloned. cloneOf and importVolAsClone are mutually exclusive.
importVolAsClone Text The name of the Nimble volume to clone and import. importVolAsClone and cloneOf are mutually exclusive.
snapshot Text The name of the snapshot to base the clone on. This is optional. If not specified, a new snapshot is created.
createSnapshot Boolean Indicates that a new snapshot of the volume should be taken matching the name provided in the snapshot parameter. If the snapshot parameter is not specified, a default name will be created.
snapshotPrefix Text A prefix to add to the beginning of the snapshot name.

Import parameters

Importing volumes to Kubernetes requires the source Nimble volume to be offline. All previous Access Control Records and Initiator Groups will be stripped from the volume when put under control of the HPE Volume Driver for Kubernetes FlexVolume Plugin.

Parameter String Description
importVol Text The name of the Nimble volume to import.
snapshot Text The name of the Nimble snapshot to restore the imported volume to after takeover. If not specified, the volume will not be restored.
restore Boolean Restores the volume to the last snapshot taken on the volume.
takeover Boolean Indicates the current group will takeover ownership of the Nimble volume and volume collection. This should be performed against a downstream replica.
reverseRepl Boolean Reverses the replication direction so that writes to the Nimble volume are replicated back to the group where it was replicated from.
forceImport Boolean Forces the import of a volume that is not owned by the group and is not part of a volume collection. If the volume is part of a volume collection, use takeover instead.

Note

HPE Nimble Docker Volume workflows works with a 1-1 mapping between volume and volume collection.

HPE Cloud Volumes StorageClass parameters

A StorageClass is used to provision or clone an HPE Cloud Volumes-backed persistent volume. It can also be used to import an existing HPE Cloud Volumes volume or clone of a snapshot into the Kubernetes cluster. The parameters are grouped below by those same workflows.

A sample StorageClass is provided.

Note

These are optional parameters.

Common parameters for Provisioning and Cloning

These parameters are mutable betweeen a parent volume and creating a clone from a snapshot.

Parameter String Description
nameSuffix Text Suffix to append to Cloud Volumes.
destroyOnRm Boolean Indicates the backing Cloud volume (including snapshots) should be destroyed when the PVC is deleted.
limitIOPS Integer The IOPS limit of the volume. The IOPS limit should be in the range 300 to 50000.
perfPolicy Text The name of the performance policy to assign to the volume. Default example performance policies include "Other, Exchange, Oracle, SharePoint, SQL, Windows File Server".
protectionTemplate Text The name of the protection template to assign to the volume. Default examples of protection templates include "daily:3, daily:7, daily:14, hourly:6, hourly:12, hourly:24, twicedaily:4, twicedaily:8, twicedaily:14, weekly:2, weekly:4, weekly:8, monthly:3, monthly:6, monthly:12 or none".
volumeType Text Cloud Volume type. Supported types are PF and GPF.

Provisioning parameters

These parameters are immutable for clones once a volume has been created.

Parameter String Description
fsOwner userId:groupId The user id and group id that should own the root directory of the filesystem.
fsMode Octal digits 1 to 4 octal digits that represent the file mode to be applied to the root directory of the filesystem.
encryption Boolean Indicates that the volume should be encrypted.

Cloning parameters

Cloning supports two modes of cloning. Either use cloneOf and reference a PVC in the current namespace or use importVolAsClone and reference a Cloud volume name to clone and import to Kubernetes.

Parameter String Description
cloneOf Text The name of the PV to be cloned. cloneOf and importVolAsClone are mutually exclusive.
importVolAsClone Text The name of the Cloud Volume volume to clone and import. importVolAsClone and cloneOf are mutually exclusive.
snapshot Text The name of the snapshot to base the clone on. This is optional. If not specified, a new snapshot is created.
createSnapshot Boolean Indicates that a new snapshot of the volume should be taken matching the name provided in the snapshot parameter. If the snapshot parameter is not specified, a default name will be created.
snapshotPrefix Text A prefix to add to the beginning of the snapshot name.
replStore Text Replication store name. Should be used with importVolAsClone parameter to clone a replica volume

Import parameters

Importing volumes to Kubernetes requires the source Cloud volume to be not attached to any nodes. All previous Access Control Records will be stripped from the volume when put under control of the HPE Volume Driver for Kubernetes FlexVolume Plugin.

Parameter String Description
importVol Text The name of the Cloud volume to import.
forceImport Boolean Forces the import of a volume that is provisioned by another K8s cluster but not attached to any nodes.

Diagnostics

This section outlines a few troubleshooting steps for the HPE Volume Driver for Kubernetes Plugin. This product is supported by HPE, please consult with your support organization (Nimble, Cloud Volumes etc) prior attempting any configuration changes.

Troubleshooting FlexVolume driver

The FlexVolume driver is a binary executed by the kubelet to perform mount/unmount/attach/detach operations as workloads request storage resources. The binary relies on communicating with a socket on the host where the volume plugin responsible for the MUAD operations perform control-plane or data-plane operations against the backend system hosting the actual volumes.

Locations

The driver has a configuration file where certain defaults can be tweaked to accommodate a certain behavior. Under normal circumstances, this file does not need any tweaking.

The name and the location of the binary varies based on Kubernetes distribution (the default 'exec' path) and what backend driver is being used. In a typical scenario, using Nimble, this is expected:

  • Binary: /usr/libexec/kubernetes/kubelet-plugins/volume/exec/hpe.com~nimble/nimble
  • Config file: /usr/libexec/kubernetes/kubelet-plugins/volume/exec/hpe.com~nimble/nimble.json

Override defaults

By default, it contains only the path to the socket file for the volume plugin:

{
    "dockerVolumePluginSocketPath": "/etc/hpe-storage/nimble.sock"
}

Valid options for the FlexVolume driver can be inspected by executing the binary on the host with the config argument:

/usr/libexec/kubernetes/kubelet-plugins/volume/exec/hpe.com~nimble/nimble config
Error processing option 'logFilePath' - key:logFilePath not found
Error processing option 'logDebug' - key:logDebug not found
Error processing option 'supportsCapabilities' - key:supportsCapabilities not found
Error processing option 'stripK8sFromOptions' - key:stripK8sFromOptions not found
Error processing option 'createVolumes' - key:createVolumes not found
Error processing option 'listOfStorageResourceOptions' - key:listOfStorageResourceOptions not found
Error processing option 'factorForConversion' - key:factorForConversion not found
Error processing option 'enable1.6' - key:enable1.6 not found

Driver=nimble Version=v2.5.1-50fbff2aa14a693a9a18adafb834da33b9e7cc89
Current Config:
  dockerVolumePluginSocketPath = /etc/hpe-storage/nimble.sock
           stripK8sFromOptions = true
                   logFilePath = /var/log/dory.log
                      logDebug = false
                 createVolumes = false
                     enable1.6 = false
           factorForConversion = 1073741824
  listOfStorageResourceOptions = [size sizeInGiB]
          supportsCapabilities = true

An example tweak could be to enable debug logging and enable support for Kubernetes 1.6 (which we don't officially support). The config file would then end up like this:

{
    "dockerVolumePluginSocketPath": "/etc/hpe-storage/nimble.sock",
    "logDebug": true,
    "enable1.6": true
}

Execute the binary again (nimble config) to ensure the parameters and config file gets parsed correctly. Since the config file is read on each FlexVolume operation, no restart of anything is needed.

See Advanced for more parameters for the driver.json file.

Connectivity

To verify the FlexVolume binary can actually communicate with the backend volume plugin, issue a faux mount request:

/usr/libexec/kubernetes/kubelet-plugins/volume/exec/hpe.com~nimble/nimble mount no/op '{"name":"myvol1"}'

If the FlexVolume driver can successfully communicate with the volume plugin socket:

{"status":"Failure","message":"configured to NOT create volumes"}

In the case of any other output, check if the backend volume plugin is alive with curl:

curl --unix-socket /etc/hpe-storage/nimble.sock -d '{}' http://localhost/VolumeDriver.Capabilities

It should output:

{"capabilities":{"scope":"global"},"Err":""}

FlexVolume and dynamic provisioner driver logs

Log files associated with the HPE Volume Driver for Kubernetes FlexVolume Plugin logs data to the standard output stream. If the logs need to be retained for long term, use a standard logging solution. Some of the logs on the host are persisted which follow standard logrotate policies.

FlexVolume driver logs:

kubectl logs -f daemonset.apps/hpe-flexvolume-driver -n kube-system

The logs are persisted at /var/log/hpe-docker-plugin.log and /var/log/dory.log

Dynamic Provisioner logs:

kubectl logs -f  deployment.apps/hpe-dynamic-provisioner -n kube-system

The logs are persisted at /var/log/hpe-dynamic-provisioner.log

Log Collector

Log collector script hpe-logcollector.sh can be used to collect diagnostic logs using kubectl

Download the script as follows:

curl -O https://raw.githubusercontent.com/hpe-storage/flexvolume-driver/master/hpe-logcollector.sh
chmod 555 hpe-logcollector.sh

Usage:

./hpe-logcollector.sh -h
Diagnostic Script to collect HPE Storage logs using kubectl

Usage:
     hpe-logcollector.sh [-h|--help][--node-name NODE_NAME][-n|--namespace NAMESPACE][-a|--all]
Where
-h|--help                  Print the Usage text
--node-name NODE_NAME      where NODE_NAME is kubernetes Node Name needed to collect the
                           hpe diagnostic logs of the Node
-n|--namespace NAMESPACE   where NAMESPACE is namespace of the pod deployment. default is kube-system
-a|--all                   collect diagnostic logs of all the nodes.If
                           nothing is specified logs would be collected
                           from all the nodes

Advanced Configuration

This section describes some of the advanced configuration steps available to tweak behavior of the HPE Volume Driver for Kubernetes FlexVolume Plugin.

Set defaults at the compute node level

During normal operations, defaults are set in either the ConfigMap or in a StorageClass itself. The picking order is:

  • StorageClass
  • ConfigMap
  • driver.json

Please see Diagnostics to locate the driver for your particular environment. Add this object to the configuration file, nimble.json, for example:

{
    "defaultOptions": [{"option1": "value1"}, {"option2": "value2"}]
}

Where option1 and option2 are valid backend volume plugin create options.

Note

It's highly recommended to control defaults with StorageClass API objects or the ConfigMap.

Global options

Each driver supports setting certain "global" options in the ConfigMap. Some options are common, some are driver specific.

Common

Parameter String Description
volumeDir Text Root directory on the host to mount the volumes. This parameter needs correlation with the podsmountdir path in the volumeMounts stanzas of the deployment.
logDebug Boolean Turn on debug logging, set to false by default.