Aerospike Benchmark (asbench)
The Aerospike Benchmark tool measures the performance of an Aerospike cluster. It can mimic real-world workloads with configurable record structures, various access patterns, and UDF calls.
The new benchmark tool is a C based benchmark tool, which replaces the Java Benchmark tool.
Usage
The --help
option of asbench
gives an overview of all supported command line options.
The -V
or --version
option prints the current version of asbench.
asbench --help
Connection Options
Option | Default | Description |
---|---|---|
-h or --hosts <host1>[:<tlsname1>][:<port1>][,...] | 127.0.0.1 | List of seed hosts. The tlsname is only used when connecting with a secure TLS enabled server. If the port is not specified, the default port is used. IPv6 addresses must be enclosed in square brackets. |
-p or --port <port> | 3000 | Set the default port on which to connect to Aerospike. |
-U or --user <user> | - | User name. Mandatory if security on the server is enabled. |
-P[<password>] | - | User's password for Aerospike servers that require authentication. If -P is set, the actual password is optional. If the password is not given, the user is prompted on the command line. If the password is given, it must be provided directly after -P with no intervening space (ie. -Pmypass). |
--auth <mode> | INTERNAL | Set the authentication mode when user/password is defined. Modes are [INTERNAL, EXTERNAL, EXTERNAL_INSECURE, PKI]. This mode must be set to EXTERNAL when using LDAP. |
-tls or --tls-enable | disabled | Use TLS/SSL sockets. |
--services-alternate | false | Use to connect to alternate-access-address when the cluster nodes publish IP addresses through access-address , which are not accessible over WAN, and alternate IP addresses accessible over WAN through alternate-access-address . |
TLS Options
Option | Default | Description |
---|---|---|
--tls-cafile=TLS_CAFILE | Path to a trusted CA certificate file. | |
--tls-capath=TLS_CAPATH | Path to a directory of trusted CA certificates. | |
--tls-name=TLS_NAME | The default TLS name used to authenticate each TLS socket connection. Note: this must also match the cluster name. | |
--tls-protocols=TLS_PROTOCOLS | Set the TLS protocol selection criteria. This format is the same as Apache's SSLProtocol documented at https://httpd.apache.org/docs/current/mod/mod_ssl.html#sslprotocol . If not specified the benchmark uses '-all +TLSv1.2' if it supports TLSv1.2, otherwise it uses '-all +TLSv1'. | |
--tls-cipher-suite=TLS_CIPHER_SUITE | Set the TLS cipher selection criteria. The format is the same as Open_SSL's Cipher List Format documented at https://www.openssl.org/docs/manmaster/man1/ciphers.html . | |
--tls-keyfile=TLS_KEYFILE | Path to the key for mutual authentication (if Aerospike Cluster supports it). | |
--tls-keyfile-password=TLS_KEYFILE_PASSWORD | Password to load protected tls-keyfile. It can be one of the following: 1) Environment variable: 'env:<VAR>' 2) File: 'file:<PATH>' 3) String: 'PASSWORD' User will be prompted on the command line if --tls-keyfile-password specified and no password is given. | |
--tls-certfile=TLS_CERTFILE <path> | Path to the chain file for mutual authentication (if Aerospike Cluster supports it). | |
--tls-cert-blacklist <path> | Path to a certificate blacklist file. The file should contain one line for each blacklisted certificate. Each line starts with the certificate serial number expressed in hex. Each entry may optionally specify the issuer name of the certificate (serial numbers are only required to be unique per issuer). Example: 867EC87482B2 /C=US/ST=CA/O=Acme/OU=Engineering/CN=TestChainCA | |
--tls-crl-check | Enable CRL checking for leaf certificate. An error occurs if a valid CRL file cannot be found in tls_capath. | |
--tls-crl-check-all | Enable CRL checking for the entire certificate chain. An error occurs if a valid CRL file cannot be found in tls_capath. | |
--tls-log-session-info | Log TLS connected session info. | |
--tls-login-only | Use TLS for node login only. |
The tlsname is only used when connecting with a secure TLS enabled server.
The following example runs the default benchmark on a cluster of nodes 1.2.3.4
and 5.6.7.8
using the default Aerospike port of 3000
with tls configured.
HOST is "host1
[:tlsname1
][:`port1`],...".
asbench --hosts 1.2.3.4:cert1:3000,5.6.7.8:cert2:3000 --namespace test --tls-enable --tls-cafile /cluster_name.pem --tls-protocols TLSv1.2 --tls-keyfile /cluster_name.key --tls-certfile /cluster_name.pem
Global Options
Option | Default | Description |
---|---|---|
-z or --threads <count> | 16 | The number of threads used to perform synchronous read/write commands. |
--compress | disabled | Enable binary data compression through the aerospike client. Internally, this sets the compression policy to true. |
--socket-timeout <ms> | 30000 | Read/Write socket timeout in milliseconds. |
--read-socket-timeout <ms> | 30000 | Read socket timeout in milliseconds. |
--write-socket-timeout <ms> | 30000 | Write socket timeout in milliseconds. |
-T or --timeout <ms> | 0 | Read/Write total timeout in milliseconds. |
--read-timeout <ms> | 0 | Read total timeout in milliseconds. |
--write-timeout <ms> | 0 | Write total timeout in milliseconds. |
--max-retries <number> | 1 | Maximum number of retries before aborting the current transaction. |
-d or --debug | disabled | Run benchmark in debug mode. |
-S or --shared | disabled | Use shared memory cluster tending. |
-C or --replica {master,any,sequence,prefer-rack} | master | Which replica to use for reads. master: Always use node containing master partition. any: Distribute reads across master and proles in round-robin fashion. sequence: Always try master first. If master fails, try proles in sequence. preferRack: Always try node on the same rack as the benchmark first. If no nodes on the same rack, use sequence. This option requires rack-id to be set. |
--rack-id <num> | Which rack this instance of the asbench resides. Required with replica policy prefer-rack. | |
-N or --read-mode-ap {one,all} | one | Read mode for AP (availability) namespaces. |
-B or --read-mode-sc {session,linearize,allowReplica,allowUnavailable} | session | Read mode for SC (strong consistency) namespaces. |
-M or --commit-level {all,master} | all | Write commit guarantee level. |
-Y or --conn-pools-per-node <num> | 1 | Number of connection pools per node. |
-D or --durable-delete | disabled | All transactions will set the durable-delete flag which indicates to the server that if the transaction results in a delete, to generate a tombstone for the deleted record. |
--send-key | disabled | Enables the key policy AS_POLICY_KEY_SEND, which sends the key value in addition to the key digest. |
--sleep-between-retries | 0 | Enables sleep between retries if a transaction fails and the timeout was not exceeded. |
-c or --async-max-commands <command count> | 50 | Maximum number of concurrent asynchronous commands that are active at any time. |
-W or --event-loops <thread count> | 1 | Number of event loops (or selector threads) when running in asynchronous mode. |
Namespace and Record Format Options
Option | Default | Description |
---|---|---|
-n or --namespace <ns> | test | The Aerospike namespace to perform all operations under. |
-s or --set <set name> | testset | The Aerospike set to perform all operations in. |
-b or --bin <bin name> | testbin | The base name to use for bins. The first bin is <bin_name> , the second is <bin_name>_2 , and so on. |
-K or --start-key <key> | 0 | Set the starting value of the working set of keys. If using an 'insert' workload, the start_value indicates the first value to write. Otherwise, the start_value indicates the smallest value in the working set of keys. |
-k or --keys <count> | 1000000 | Set the number of keys the client is dealing with. If using an 'insert' workload (detailed below), the client will write this number of keys, starting from value = start-key. Otherwise, the client will read and update randomly across the values between start-key and start-key + num_keys. start-key can be set using '-K' or '--start-key'. |
-o or --object-spec <obj_spec> | I4 | Describes a comma-separated bin specification. See object spec below for more details. |
--compression-ratio <ratio> | 1 | Sets the desired compression ratio for binary data. Causes the benchmark tool to generate binary data which will roughly compress by this proportion. Note: this is only applied to B<n> binary data, not any of the other types of record data. |
-e or --expiration-time | 0 | Set the TTL of all records written in write transactions. Options are -1 (no TTL, never expire), -2 (no change TTL, i.e. the record TTL will not be modified by this write transaction), 0 (adopt default TTL value from namespace) and >0 (the TTL of the record in seconds). |
Object Spec
The object spec is a flexible way to describe how to structure records being written to the database. It is a comma-separated list of bin specs, and each bin spec is one of the following:
Variable Scalars:
Type | Format | Description |
---|---|---|
Boolean | b | A random boolean bin/value. |
Integer | I<n> | A random integer with the lower n bytes randomized (and the rest remaining 0). n can range from 1 to 8. Note: the n th byte is guaranteed to not be 0, except in the case n=1 . |
Double | D | A random double bin/value (8 bytes). |
String | S<n> | A random string of length n of either lowercase letters a-z or numbers 0-9 . |
Binary Data | B<n> | Random binary data of length n . Note: if --compression-ratio is set, only the first ratio * n bytes are random, and the rest are 0. |
Constant Scalars:
Type | Format | Example |
---|---|---|
Const Boolean | true /T or false /F | true |
Const Integer | A decimal, hex (0x...), or octal (0...) number | 123 |
Const Double | A decimal number with a . | 123.456 |
Const String | A backslash-escaped string enclosed in double quotes | "this -> \" is a double quote\n" |
Collection Bins:
Type | Format | Notes |
---|---|---|
List | [<bin_spec>,...] | A list of one or more bin specs separated by commas. |
Map | {<scalar_bin_spec>:<bin_spec>,...} | A list of one or more mappings from a scalar bin spec (i.e. anything but a list or map) to a bin spec. These describe the key-value pairs that the map will contain. |
Multipliers
Multipliers are positive integer constants, followed by a "*", preceding a bin spec.
In the root-level object spec, multipliers indicate how many times to repeat a bin spec across separate bins. For example, the following object specs are equivalent:
I, I, I, I, S10, S10, S10 = 4*I, 3*S10
123, 123, 123, "string", "string" = 3*123, 2*"string"
In a list, multipliers indicate how many times to repeat a bin spec in the list. The following are equivalent:
[I, I, I, I, S10, S10, S10] = [4*I, 3*S10]
In a map, multipliers must precede variable scalar keys, and they indicate how many unique key-value pairs of the given format to insert into the map. Multipliers may not precede const key bin specs or value bin specs in a key-value mapping. The following are equivalent:
{I:B10, I:B10, I:B10} = {3*I:B10}
Workloads
There are four main types of workloads, which are ways in which the benchmark tool can interact with the Aerospike database. The four types are:
I
) Linear Insert: Runs over the range of keys specified and inserts a record with that key.RU,<read_pct>[,<read_pct_all_bins>[,<write_pct_all_bins>]]
) Random Read/Update: Randomly picks keys, and either writes a record with that key or reads a record from the database with that key, with probability according to the given read percentage.<read_pct>
can be anything between 0 and 100. 0 would mean to only do writes, and 100 to only do reads.- Starting with asbench 1.5 (Tools 7.1), you may optionally provide the
<read_pct_all_bins>,<write_pct_all_bins>
to indicate the percentage of reads and writes that read the entire record. Otherwise only read the first bin. The default is 100.
RR,<read_pct>[,<read_pct_all_bins>[,<replace_pct_all_bins>]]
) Random Read/Replace/Function: Same asRU
, except replaces record instead of updates.RUF,<read_pct>,<write_pct>[,<read_pct_all_bins>[,<write_pct_all_bins>]]
) Random Read/Update/Function: Same asRU
, except may also perform an apply command on the random key with a given UDF function.- The percentage of operations that are function calls (i.e. UDFs) is
100 - <read_pct> - <write_pct>
. This value must not be negative, which is checked for at initialization.
- The percentage of operations that are function calls (i.e. UDFs) is
RUD,<read_pct>,<write_pct>[,<read_pct_all_bins>[,<write_pct_all_bins>]]
) Random Read/Update/Delete: Same asRU
, except may also perform deletes on a random key.- The percentage of operations that deletes is
100 - <read_pct> - <write_pct>
. This value must not be negative, which is checked for at initialization.
- The percentage of operations that deletes is
DB
) Delete bins: Same asI
, but deletes the record with the given key from the database.
In order for DB to delete entire records, it must delete every bin that the record contains. Since bins are named based on their position in the object spec, typically you want to make sure when running this workload you are using the same object spec you used to generate the records being deleted.
DB
uses write-bins to determine which bins to delete (by default, all), so if you only want to delete a subset of bins, you may use write-bins to select which to delete
Workload Options
Option | Default | Description |
---|---|---|
--read-bins | all bins | Specifies which bins from the object-spec to load from the database on read transactions. Must be given as a comma-separated list of increasing bin numbers, starting from 1 (i.e. "1,3,4,6"). |
--write-bins | all bins | Specifies which bins from the object-spec to generate and store in the database on write transactions. Must be given as a comma-separated list of bin numbers, starting from 1 (i.e. "1,3,4,6"). |
-R or --random | disabled | Use dynamically generated random bin values for each write transaction instead of fixed values (one per thread) created at the beginning of the workload. |
-t or --duration <seconds> | 10 for RU,RUF workloads, 0 for I,DB workloads | Specifies the minimum amount of time the benchmark will run for. For random workloads with no finite amount of work needing to be done, this value must be above 0 for anything to happen. For workloads with a finite amount of work, like linear insertion/deletion, this value should be set to 0. |
-w or --workload <workload> | RU,50 | Desired workload. See workload types |
-g or --throughput <tps> | 0 | Throttle transactions per second to a maximum value. If tps is zero, do not throttle throughput. |
--batch-size <size> | 1 | Enable batch mode with a number of records to process in each batch get call. Batch mode is valid only for RU or RUF workloads. If batch size is 1, batch mode is disabled. |
-a or --async | disabled | Enable asynchronous mode, which uses the asynchronous variant of every Aerospike C Client method for transactions. |
Workload Stages
Multiple different workloads may be run in sequence using a workload stage config file, which is in YAML format. The config file should only be a list of workload stages in the following format:
- stage: 1
# required arguments
workload: <workload type>
# optional arguments
duration: <seconds>
tps : max possible with 0 (default), or specified transactions per second
object-spec: Object spec for the stage. Otherwise, inherits from the previous
stage, with the first stage inheriting the global object spec.
key-start: Key start, otherwise inheriting from the global context
key-end: Key end, otherwise inheriting from the global context
read-bins: Which bins to read if the workload includes reads
write-bins: Which bins to write to if the workload includes writes
pause: max number of seconds to pause before the stage starts. Waits a random
number of seconds between 1 and the pause.
async: when true/yes, uses asynchronous commands for this stage. Default is false
random: when true/yes, randomly generates new objects for each write. Default is false
batch-size: specifies the batch size of reads for this stage. Default is 1
- stage: 2
...
Each stage must begin with stage: <stage number>
, where stage number is the position of the stage in the list. The stages must appear in order.
When arguments say they inherit from the global context, the value they inherit either comes from a command line argument, or is the default value if no command line argument for that value was given.
Latency Histograms
There are multiple ways to record latencies measured throughout a benchmark run. All latencies are recorded in microseconds.
Option | Default | Description |
---|---|---|
--output-file | stdout | Specifies an output file to write periodic latency data, which enables the tracking of transaction latencies in microseconds in a histogram. Currently uses a default layout. The file is opened in append mode. |
-L or --latency | disabled | Enables the periodic HDR histogram summary of latency data. |
--percentiles <p1>[,<p2>[,<p3>...]] | "50,90,99,99.9,99.99" | Specifies the latency percentiles to display in the periodic latency histogram. |
--output-period <seconds> | 1 | Specifies the period between successive snapshots of the periodic latency histogram. |
--hdr-hist <path/to/output> | disabled | Enables the cumulative HDR histogram and specifies the directory to dump the cumulative HDR histogram summary. |
Periodic Latency Histogram
Periodic latency data is recorded in histograms with three ranges of fixed bucket sizes, which as of right now are not configurable. There is one histogram for reads, one for writes, and one for UDF calls. The three ranges are:
- 100us to 4000us, bucket width 100us
- 4000us to 64000us, bucket width 1000us
- 64000us to 128000us, bucket width 4000us
Format of the histogram output file:
<hist_name> <UTC time>, <period time>, <num records>, <bucket 1 lower bound>:<num records>, ...
First the name of the histogram is printed (either read_hist
, write_hist
, or udf_hist
). This is followed by the UTC-time of the event being recorded (i.e. the end of the interval), then by the length of the interval in seconds, and then by the total number of transaction latencies recorded in the interval. After this, each bucket with at least one latency recorded is displayed in ascending order of interval lower bounds, written as the lower bound of the bucket, colon, number of transaction latencies falling within the bucket's range.
HDR Histogram
Transaction latencies can also be recorded in an HDR Histogram. There is one HDR histogram for reads, one for writes, and one for UDF calls. The two ways to enable the HDR Histograms are to either use --latency
, which will display select percentiles from the HDR histograms every output-period
seconds, and to use --hdr-hist
, which will write the full HDR histograms to the given directory in both a human-readable text format (.txt) and a binary encoding of the HDR Histogram (.hdrhist).
The percentiles that are printed when --latency
is enabled can be configured by using --percentiles
followed by a comma-separated list of percentiles. This list must be in ascending order, and no percentile can be less than 0 or greater or equal to 100.
UDFs
UDF calls are made in RUF (read/update/function) workloads, being the "function" part of that workload. A key is chosen at random from the range of keys given, and an Aerospike apply call is made on that key with the given UDF function (--udf-function-name
) from the given UDF package (--udf-package-name
). Optionally, --udf-function-values
may be supplied, which takes an object spec and randomly generates arguments every call.
Note: the UDF function args follow the same rules as the object spec used on records, and only randomly generate for every call if --random
is supplied as an argument
Option | Default | Description |
---|---|---|
-upn or --udf-package-name <package_name> | - | The package name for the udf to be called. |
-ufn or --udf-function-name <function_name> | - | The name of the UDF function in the package to be called. |
-ufv or --udf-function-values <fn_vals> | none | The arguments to be passed to the udf when called, which are given as an object spec (see object spec). |