Skip to main content
Loading

Configuring Encryption at Rest

Encryption at Rest​

Aerospike’s encryption at rest feature encrypts database record data on storage devices using symmetric AES-128 or AES-256 encryption. By encrypting and decrypting each record in the database, even if storage devices are stolen, the data will be unreadable.

The encryption key is derived using a user-supplied encryption key file. That key file contains a key, which is the secret for encrypting data to and from the drive. If that key is lost, the data on the device becomes inaccessible.

Encryption at Rest is an Enterprise Edition feature requiring a feature-key-file.

  • For Aerospike Server version 4.1 & 4.2, a feature-key-file with encryption-at-rest set to true is required.
  • For Aerospike Server version 4.3 and newer, a feature-key-file with asdb-encryption-at-rest set to true is required.

The default location of the feature-key-file is /etc/aerospike/features.conf.
Contact Aerospike sales/support if you do not have a feature-key.

Configuration​

Encryption is configured per namespace, using three configuration parameters, encryption-key-file, encryption-old-key-file, and encryption.

  • For the namespace's storage devices, encryption-key-file specifies either the path to an encryption key file or the name of a secret in Hashicorp Vault that stores the encryption key.

    If the parameter is specified, encryption is enabled. If it is not specified, encryption is disabled.

  • For the namespace's storage devices, encryption-old-key-file specifies either the path to the previous version of the encryption key file or the name of a secret in Hashicorp Vault that stores the previous encryption key.

    The parameter is optional and was added in Aerospike Server version 5.7.

  • encryption specifies the encryption algorithm to be used. Valid values are aes-128 and aes-256. If this parameter is not specified, it defaults to aes-128.

For information about using Hashicorp Vault, see Optional security with Vault integration.

The configuration parameters belong to a namespace's storage-engine stanza:

namespace test {
    ...
    storage-engine device {
        device /dev/sda1
        ...
        encryption-key-file key.dat
        encryption-old-key-file key.dat
        encryption aes-128
    }
    ...
}

Unpredictability of encryption key file​

The content of the encryption key file needs to be unpredictable. Aerospike suggests you generate the content of the file from a reliable source of randomness. For example:

head --bytes 256 /dev/urandom >key.dat

The keys can be different for different namespaces, or for different servers in the cluster. In all cases, the file must be kept private. If the key file is stolen, then the device's data can be accessed. For this reason, when running in this mode, the key file should only be kept on encrypted and secure operating system environments.

Storage devices must be empty for enabling or disabling encryption at rest​

Enabling or disabling encryption for a namespace or changing the encryption algorithm requires that the namespace’s storage devices or disk files are empty.

General process for adding, removing, or changing a key for all nodes​

Adding or removing the key file for a cluster must be done individually node-by-node. In a rolling process, on each node, one at a time, you must:

  1. Stop the Aerospike daemon.
  2. Zero the node's storage devices.
  3. Restart the Aerospike daemon.

This process must be completed on the node before you proceed to the next node.

Prior to Aerospike Server version 5.7, the above process must also be followed for changing the key file for all nodes. However, starting with Aerospike Server version 5.7, there is no need to zero the node's storage devices (see below).

Variations on this process for different situations are discussed below.

Enabling encryption at rest on a previously unenabled cluster​

You can enable this feature on a cluster that has been running without encryption. It can also be added to a cluster while the cluster is in production, with no outage or downtime, because the conversion process can be done node-by-node.

This leads to the following conversion process, which enables or disables encryption across an Aerospike cluster for a single node in the cluster:

  • Stop the Aerospike server process on the cluster node to be converted.
  • If you need to maintain your configured replication count of data, wait for Aerospike replication to complete. If your deployment durability allows a short period with fewer copies of the data, you may skip this step.
  • Remove the unencrypted data on the device, using a program such as dd for raw devices, or simply rm for file based storage.
  • Either enable or disable encryption by adding or removing the appropriate files and parameter(s) to aerospike.conf.
  • Restart the Aerospike server process on the cluster node.
  • Wait until data migration is complete.

Repeat this process for all nodes in the cluster.

Changing the encryption algorithm works analogously.

Key rotation​

In order to change the encryption key prior to Aerospike Server version 5.7, you will follow a similar process. Just like enabling or disabling encryption, you will move through the cluster taking servers down, updating their key file, removing the data from the device(s), and restarting servers.

With Aerospike Server version 5.7, you need not remove the data from the device(s). You must instead provide the old encryption key as the file specified by the encryption-old-key-file configuration parameter.

The files specified by the encryption-key-file and the encryption-old-key-file configuration parameters are read exactly once at server startup, just after the configuration file, aerospike.conf, has been parsed. For increased security, the encryption key file may thus reside in a RAM-backed file system instead of a file system backed by physical storage media.

Once the Aerospike server process has been started, the encryption key file and the encryption old key file - or the complete RAM-backed file system - may be discarded. In this way, the encryption key file and the encryption old key file would only temporarily be available as a file on each cluster node.

Note that the same encryption key file and encryption old key file must be provided again on the next start of the Aerospike server process. This means that the encryption key file and encryption old key file still need to be stored permanently somewhere, just not necessarily on the cluster nodes. The idea is to only make them temporarily available on the cluster nodes during Aerospike startup.

Keep your encryption key file and your encryption old key file safe. If you lose them, you lose access to your data.

Technical Details​

Encryption at rest involves two encryption keys:

  • One key that you supply in the encryption key file, which is called the user-supplied encryption key.
  • The other key is created by the system itself, which is called the data encryption key.

The encryption process is two-step:

  1. The server hashes the user-supplied encryption key contained in the encryption key file. This uses the encryption algorithm you specify in the encryption parameter, which is either 128-bit (AES-128) or 256-bit (AES-256).
  2. The server uses the resulting value as a key to encrypt a data encryption key that is stored on the device itself. To create the data encryption key, the server uses a cryptographically strong random number generator: OpenSSL’s RAND_bytes() function.

On Aerospike startup, the following happens:

  • Aerospike reads the encryption key file provided by the user and hashes it with SHA-256 (for AES-128) or SHA-512 (for AES-256). This yields the user-supplied encryption key.
  • Aerospike reads the encrypted data encryption key from each device and decrypts it using the user-supplied encryption key. This yields the data encryption key for each device.
  • Aerospike tests whether the data encryption key for a device is valid by checking a "canary" value stored in the device header. If this value is decryptable using the data encryption key, it means that the encryption key file supplied by the user is valid.
  • (Version 5.7) If the canary value is not decryptable, the process is retried with the encryption old key file supplied by the user.

Either a user-supplied encryption key file will be found valid and its contents will be used in generating the data encryption key or the user has not supplied a valid encryption key and the Aerospike server will stop.

Once a valid data encryption key is computed, it is used to write and read data to and from the device. This encryption is known as XTS-AES block cipher mode and is detailed in IEEE P1619 standard for disk encryption and NIST Special Publication (SP) 800-38E.

Disk encryption typically works on a per-sector basis. However, for better performance, Aerospike encrypts on a per-record basis. Records on disk are basically treated as variable-size disk sectors up to a size of 1 MiB. Beyond 1 MiB, records are split into 1-MiB chunks.

Performance​

The performance impact of encryption depends on the deployed hardware as well as the average size of the encrypted records. A definitive answer regarding the performance impact can thus only be obtained by benchmarking on the deployed hardware with real-life data.

However, use of Linux's OpenSSL library on Intel Xeon hardware uses on-CPU hardware acceleration. In one typical hardware configuration, we observed that:

Records smaller than 512 bytes resulted in 20% performance loss in terms of TPS.

Records with sizes of 1 KiB, 5 KiB, and 10 KiB didn’t lead to any measurable performance loss.

Last updated on