asbackup utility is used to backup namespaces, sets, or partitions from an Aerospike cluster to local storage. By default, backups are not parallelized. This can be changed with the
--parallel command line option (see the parallel flag under connection options). If a record is created or updated after its partition was backed up, the backup will not reflect that.
When run with the
asbackup creates multiple
.asb backup files in the given directory. The backup consists of all created files. Alternatively,
asbackup store the complete backup in the given single file. If
- is specified as the file,
asbackup writes the backup to
stdout. This allows for pipelines:
asbackup --output-file - [...] | gzip -1 [...] >backup.asb.gz
When running in directory mode, each worker thread creates its own backup file. If, after completing the scan, the backup file is not full (i.e. less than --file-limit MB in size), place the backup file on a queue to be reused by another backup job.
--help option of
asbackup gives an overview of all supported command line options.
$ asbackup --help
The most basic form of running
asbackup is to just specify the cluster to backup (
--host), the namespace to backup (
--namespace), as well as the local directory for the backup files (
--directory). Suppose that we have a cluster that contains a node with IP address
18.104.22.168. To backup the
test namespace on this cluster to the directory
backup_2015_08_24, we would issue the following command.
$ asbackup --host 22.214.171.124 --namespace test --directory backup_2015_08_24
Estimating the backup size
When passing the
--estimate command line option to
asbackup (and skipping
asbackup creates a temporary test backup of 10,000 records from the namespace. It then outputs, based on the observed record sizes, an estimate of the average size of a record in the backup. In order to estimate the total size of the backup file or files, multiply this size by the number of records in the namespace and add 10% for indexes and overhead.
Note: per-record filters (
node-list are not accounted for in the estimate, and using these options will have no affect on the estimate.
Before a backup-to-file is run, asbackup runs an estimate on the namespace being backed up and uses it to calculate a 99.9% confidence upper bound on the total size of the backup file. The number of estimate samples taken can be controlled with
--estimate-samples, with the default being 10000, just as in normal estimate mode.
Beginning from Server Version 3.12, timestamps can be specified so only records updated since timestamp X are backed up. An operational routine can be established to do incremental daily backups. Please see
--modified-after option in Data Options section.
Additionally, you can do incremental backups using the
--partition-list option. For more information, see here.
|127.0.0.1||The host that acts as the entry point to the cluster. Any of the cluster nodes can be specified. The remaining cluster nodes will be automatically discovered.|
|3000||Port to connect to.|
|-||User name with read permission. Mandatory if the server has security enabled.|
|-||Password to authenticate the given user. The first form passes the password on the command line. The second form prompts for the password.|
|INTERNAL||Set authentication mode when user and password are defined. Modes are (INTERNAL, EXTERNAL, EXTERNAL_INSECURE, PKI) and the default is INTERNAL. This mode must be set EXTERNAL when using LDAP.|
|1||Maximum number of scan calls to run in parallel. If only one partition range is given, or the entire namespace is being backed up, the range of partitions will be evenly divided by this number to be processed in parallel. Otherwise, each filter cannot be parallelized individually, so you may only achieve as much parallelism as there are partition filters.|
|disabled||Indicates a TLS connection should be used.|
|false||Use to connect to |
|10000||Socket timeout in milliseconds. If this value is 0, it is set to total-timeout. If both are 0, there is no socket idle time limit.|
|0||Total socket timeout in milliseconds. Default is 0, i.e. no timeout.|
|5||Maximum number of retries before aborting the current transaction.|
|0||The amount of time to sleep between retries.|
|Path to a trusted CA certificate file.|
|Path to a directory of trusted CA certificates.|
|The default TLS name used to authenticate each TLS socket connection. Note: this must also match the cluster name.|
|Set the TLS protocol selection criteria. This format is the same as Apache's SSLProtocol documented at https://httpd.apache.org/current/mod/mod_ssl#sslprotocol . If not specified the asbackup will use '-all +TLSv1.2' if has support for TLSv1.2, otherwise it will be '-all +TLSv1'.|
|Set the TLS cipher selection criteria. The format is the same as Open_sSL's Cipher List Format documented at https://www.openssl.org/manmaster/man1/ciphers.html .|
|Path to the key for mutual authentication (if Aerospike Cluster is supporting it).|
|Password to load protected tls-keyfile. It can be one of the following:|
1) Environment varaible: 'env:<VAR>'
2) File: 'file:<PATH>'
3) String: 'PASSWORD'
User will be prompted on command line if --tls-keyfile-password specified and no password is given.
|Path to the chain file for mutual authentication (if Aerospike Cluster is supporting it).|
|Path to a certificate blacklist file. The file should contain one line for each blacklisted certificate. Each line starts with the certificate serial number expressed in hex. Each entry may optionally specify the issuer name of the certificate (serial numbers are only required to be unique per issuer). Example: 867EC87482B2 /C=US/ST=CA/O=Acme/OU=Engineering/CN=TestChainCA|
|Enable CRL checking for leaf certificate. An error occurs if a valid CRL files cannot be found in tls_capath.|
|Enable CRL checking for entire certificate chain. An error occurs if a valid CRL files cannot be found in tls_capath.|
|Enable logging session information for each TLS connection.|
The tlsname is only used when connecting with a secure TLS enabled server.
The following example creates a backup of cluster nodes
126.96.36.199 using the default Aerospike port of
3000 with tls configured.
HOST is "
asbackup --host 188.8.131.52:cert1:3000,184.108.40.206:cert2:3000 --namespace test --directory backup_2015_08_24 --tls-enable --tls-cafile /cluster_name.pem --tls-protocols TLSv1.2 --tls-keyfile /cluster_name.key --tls-certfile /cluster_name.pem
|-||Directory to store the |
|-||The single file to write the backup to. |
|Must be used with the |
|-||Specified in lieu of |
|10000||Sets the number of record samples to take in a backup estimate. This also sets the number of estimate samples taken for the estimate run before backup-to-file.|
|250 MiB||File size limit (in MiB) for |
|-||Clear directory or remove output file. By default, |
|-||Clear directory or remove output file, like |
|-||Do not base-64 encode BLOB values. For better readability of backup files, |
|none||The encryption algorithm to be used on backup files as they are written. The options available are |
|none||The compression algorithm to be used on backup files as they are written. The options available are |
Namespace data selection options
|-||Namespace to backup. Mandatory.|
|All sets||The set(s) to backup. May pass in a comma-separated list of sets to back up (version 3.6.1+). Starting with asbackup 3.9.0, server version 5.2 or later is required for multi-set backup. Note: multi-set backup cannot be used with |
|All bins||The bins to backup.|
|-||Only backup record metadata (digest, TTL, generation count, key). WARNING: No data (bin contents) is backed up. Also, this is unrelated to the single-bin option in the Aerospike server configuration file.|
|-||Do not backup any record data (metadata or bin data). By default, |
|-||Do not backup any secondary index definitions. See above.|
|-||Do not backup any UDF modules. See above.|
|0 = all records.||An approximate limit for the number of records to process. Available in Server 4.9 and above. Note: this option is mutually exclusive to |
|-||Backup data with last-update-time after the specified date-time. The system's local timezone applies. Available in Server 3.12 and later. Starting with asbackup 3.9.0, server version 5.2 or later is required.|
|-||Backup data with last-update-time before the specified date-cal timezone applies. Available in Server 3.12 and later. Starting with asbackup 3.9.0, server version 5.2 or later is required.|
|-||Include only records that have no TTL; that is, persistent records. Starting with asbackup 3.9.0, server version 5.2 or later is required.|
Partition scanning backup options
-X, --partition-list <list>
Backup list of partition filters. Partition filters can be ranges, individual partitions, or records after a specific digest within a single partition.
This option is mutually exclusive with the
--after-digest option described in After specific digest,
Default number of partitions to back up: 0 to 4095: all partitions.
<begin partition>[-<partition count>]|<digest>
<begin partition>: 0 to 4095.
- Either the optional
<partition count>: 1 to 4096. Default: 1
- Or the optional
<digest>: Base64-encoded string of desired digest to start at in specified partition.
Note: when using multiple partition filters, each partition filter is a single scan call and cannot be parallelized with the
parallel option. To have more parallelizability, you can either break up the partition filters, or run a backup using only one partition filter.
When backing up only a single partition range, the range is automatically divided into
parallel segments of near-equal size, each of which is backed up in parallel.
- Backup only partition 361
- Backup partitions 361, 529, and 841
- Backup 481 partitions, starting at 361 (i.e. partitions 361 through 841)
- Backup all records after the digest
VSmeSvxNRqr46NbOqiy9gy5LTIc=in its partition (which in this case is partition 2389)
- Backup partitions 0 to 999 (1000 partitions starting from 0)
- Then backup partition 2222
- Then backup all records after the digest
EjRWeJq83vEjRRI0VniavN7xI0U=in its partition
After specific digest
Backup records after the specified record digest in that record's partition and all succeeding partitions.
This option is mutually exclusive with the
--partition-list option described in Partition filter,
<digest>format: Base64-encoded string of desired digest. This is the same encoding used for backup of digests, so you can copy-and-paste digest identifiers from backup files to use as the command-line argument with
Backups can be made of only a subset of data matching a provided Aerospike Expression. As of right now, you must provide the base-64 encoding of the filter expression, which can be generated if using the C client (
as_exp_build_b64) or the Java client (
This option is mutually exclusive with multi-set backup (i.e.
--set with more than one set specified).
To build an expression that filters for bin "name" = "bob", first, build the expression in the C client and print out its base 64 encoding:
as_exp_build_b64(b64_exp, as_exp_cmp_eq(as_exp_bin_str("name"), as_exp_str("bob")));
This should print "kwGTUQOkbmFtZaQDYm9i". Then, to run a backup with this filter expression, run
asbackup --filter-exp kwGTUQOkbmFtZaQDYm9i ...
|disabled||Enables the resumption of an interrupted backup from provided state file. All other command line arguments should match those used in the initial run (except |
|see below||Specifies where to save the backup state file to. If this points to a directory, the state file is saved within the directory using the same naming convention as backup-to-directory state files. If this does not point to a directory, the path is treated as a path to the state file.|
Default backup state file location
For backups to a file, the backup state is saved to a file with the same name and location as the backup state file with
.state appended as a postfix. For backups to a directory, the backup state is saved in the directory with name
--output-file-prefix is supplied, that is used in place of
Backup to S3
To back up files to Amazon S3, prefix file and directory names with "s3://<bucket>/<key>", where <bucket> is the name of the S3 bucket to upload to or download from, and <key> is the key of the object to download/prefix of files in the S3 "directory". If using the default S3 endpoint,
--s3-region <region> must be set to the region where the bucket is located. If using another endpoint, specify that endpoint with
Files are uploaded in parts asynchronously. The maximum number of simultaneous asynchronous upload parts across all threads is controlled with
--s3-max-async-uploads. Each connection to S3 is throttled, so this number may need adjustment to maximize throughput.
You can upload a maximum of 10,000 parts to S3. Each part must be between 5MB and 5GB, except for the last part which has no lower bound. When backing up to a directory, the value of
--file-limit is used to calculate what part size should be used (i.e. max(
file-limit / 10000, 5MB)). When backing up to a file, the estimate run before starting the backup is used in the prior equation to calculate the proper upload part size. The upload part size may also be overridden with
--s3-min-part-size, though it is unlikely this option will need to be used in practice.
|-||Sets the S3 region of the bucket being uploaded to or downloaded from. Must be set if using the default S3 endpoint.|
|-||Sets the S3 endpoint to use. Must point to an S3-compatible storage system.|
|default||Sets the S3 profile to use for credentials.|
|-||An override for the minimum S3 part size to use for file uploads. By default, this size is calculated based on the expected backup file size (found either with the value of |
|32||The maximum number of simultaneous download requests from S3.|
|16||The maximum number of simultaneous upload requests from S3.|
To backup all records from a namespace "test" to an S3 bucket "test-bucket" in region "us-west-1" under directory "test-dir", run
asbackup -n test -d s3://test-bucket/test-dir --s3-region us-west-1
Configuration file options
asbackup can be configured by using tools configuration files. See Aerospike Tools Configuration for more details. The following options affect configuration file behavior.
|disabled||Do not read any configuration file. The configuration files options |
|-||Section with these instance is read. e.g in case instance |
|-||Read this file after default configuration file.|
|-||Read only this configuration file. The configuration files options |
|disabled||Output considerably more information about the running backup.|
|-||Output machine-readable status updates to the given path, typically a FIFO.|
|0||Available only for Aerospike Server version 4.7+.|
Limit returned records per second (rps) rate for each server. If