In order to use persistent storage external to the pods, you need to set up storage classes. Storage classes are used to dynamically provision the persistent storage required by users in the Aerospike CR configuration.
The specific storage configuration depends on the environment in which you deploy your Kubernetes cluster. Each cloud provider has their own way to set up storage provisioners which dynamically create and attach storage devices to the containers.
To deploy an Aerospike cluster with persistent storage, create a storage-class.yaml file. Add the storage class configurations to this file, then use kubectl to apply these changes to the cluster.
The following storage-class.yaml file uses the GCE provisioner to create a storage class called ssd.
This example uses a local SSD (identified as
/dev/nvme0n1). Attach this SSD to each Kubernetes worker node which will be used for getting the primary storage device for the Aerospike cluster deployment.
Create a Discovery Directory
Before deploying local volume provisioner, create a discovery directory on each worker node and link the block devices to be used in the discovery directory. The provisioner will discover local block volumes from this directory.
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
nvme0n1 8:16 0 375G 0 disk
nvme0n2 8:32 0 375G 0 disk
Create the directory:
Link to the first block device:
sudo ln -s /dev/nvme0n1 /mnt/disks/
Link0 . to the second block device:
sudo ln -s /dev/nvme0n2 /mnt/disks/
You can also use your own discovery directory, but make sure that the provisioner is configured to point to the same directory.
Deploy the Local Volume Provisioner
To automate the local volume provisioning, we will create and run a provisioner based on kubernetes-sigs/sig-storage-local-static-provisioner.
The provisioner runs as a DaemonSet which manages the local SSDs on each node based on a discovery directory, creates and deletes the PersistentVolumes, and cleans up the storage when it is released.
The local volume static provisioner for this example is defined in aerospike_local_volume_provisioner.yaml.
If the Provisioner configuration is set to
shredfor block cleanup, then PersistentVolume release and reclaim takes time proportionate to the size of the block device.
Refer to the
blockCleanerCommandsection in the following configuration:
blkdiscardyou need to be sure that the drive deterministically returns zeros after TRIM (RZAT). You should check with your hardware provider or cloud provider. If you are unsure, it is safer to use
For Openshift platforms, Local Storage Operator can be used for local provisioning.
Create a local storage class:
kubectl create -f local_storage_class.yaml
Then deploy the provisioner:
kubectl create -f aerospike_local_volume_provisioner.yaml
Verify the persistent volumes were created.
$ kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
local-pv-342b45ed 375Gi RWO Delete Available "local-ssd" 3s
local-pv-3587dbec 375Gi RWO Delete Available "local-ssd" 3s
storageclass configured here is
"local-ssd". We will provide this in the Aerospike cluster CR config. This storageclass will be used to talk to the provisioner and request PV resources for the cluster.