Aerospike Admin Collectinfo-Analyzer Mode Commands
Usage
To start the interactive shell in Collectinfo-analyzer mode run asadm -c -f <collectinfopath> (Introduced: 0.1.8).
The -f parameter accepts the path of a single collectinfo file or a collectinfo directory path.
Help
The help command provides a brief description of all supported commands. You can
provide the name of a command to print help for that specific command.
In the example below, we request help for the info command.
Collectinfo-analyzer> help info
Commands that display summary tables for various aspects of Aerospike
functionality.
Usage: info COMMAND
Commands:
Default Displays network, namespace, and xdr summary information
namespace Provides summary tables for various aspects of Aerospike functionality
network Displays network information for the cluster
set Displays summary information for each set
sindex Displays summary information for Secondary Indexes
xdr Displays summary information for each datacenter
Run 'help info COMMAND' for more information on a command.
List
The list command displays the collectinfo currently added to collectinfo-analyzer.
Example Output:
Collectinfo-analyzer> list
2017-03-08 07:04:33 UTC: /home/vagrant/shared/data/healthdata/remote_cluster
Info
Commands within info provide diagnostic information in a concise tabular
format. Without additional arguments info will execute network,
namespace, and xdr sub-commands. Command descending from
info will alert you of potential cluster issues by coloring suspicious test
red. If cluster principal and node ids provided in collectinfo, one node name will be green,
this node is the node expected to be the Paxos Principal node. For namespace and set
subcommands, it displays extra rows in blue, which has sum of statistics per
namespace and set.
Info network
The info network command primarily serves as a translation table between
Node name, Node ID, and IP. It also provides cluster stats such as the
cluster size, cluster key, number of client connections and uptime for each server.
Under the Node Id column asadm also indicate the node which is expected to be the Paxos Principal with an asterisks.
Example Output:
Collectinfo-analyzer> info network
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Network Information (2020-12-16 21:45:32 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Node| Node ID| IP| Build|Migrations|~~~~~~~~~~~~~~~~~~Cluster~~~~~~~~~~~~~~~~~~|Client| Uptime
| | | | |Size| Key|Integrity| Principal| Conns|
10-0-0-1.heln.qwest.net:3000| BB9010016AE4202| 10.0.0.1:3000|C-5.3.0.1| 0.000 | 5|92DCF600367B|True |BB9050016AE4202| 2|00:07:48
10-0-0-2.heln.qwest.net:3000| BB9020016AE4202| 10.0.0.2:3000|C-5.3.0.1| 0.000 | 5|92DCF600367B|True |BB9050016AE4202| 2|00:07:47
10-0-0-3.heln.qwest.net:3000| BB9030016AE4202| 10.0.0.3:3000|C-5.3.0.1| 0.000 | 5|92DCF600367B|True |BB9050016AE4202| 2|00:07:46
10-0-0-4.heln.qwest.net:3000| BB9040016AE4202| 10.0.0.4:3000|C-5.3.0.1| 0.000 | 5|92DCF600367B|True |BB9050016AE4202| 3|00:07:46
10-0-0-5.heln.qwest.net:3000|*BB9050016AE4202| 10.0.0.5:3000|C-5.3.0.1| 0.000 | 5|92DCF600367B|True |BB9050016AE4202| 3|00:07:45
Number of rows: 5
Info namespace
The info namespace command displays a summary of important namespace
statistics for each namespace defined on each node ordered by Namespace and
Node. It displays an extra row which is an aggregate of some of the statistics.
When the primary index or secondary index is stored on device (not shmem) extra
usage statistics are displayed similar to the "Memory" columns in the following table.
From 0.1.14 onwards it displays information in two separate tables:
- Usage: Namespace usage related details
- Object: Namespace object related details
Example Output:
Collectinfo-analyzer> info namespace
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Namespace Usage Information (2023-03-21 23:44:05 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Namespace| Node|Evictions| Stop|~Device~|~~~~~~~~~~~~Memory~~~~~~~~~~~|~Primary Index~~|~Secondary Index~
| | |Writes| HWM%| Used| Used%| HWM%| Stop%| Type| Used| Type| Used
bar |172.17.0.3:3000| 0.000 |False | 0.0 %| 0.000 B | 0.0 %|0.0 %|90.0 %|shmem| 0.000 B |shmem | 0.000 B
bar |172.17.0.4:3000| 0.000 |False | 0.0 %| 0.000 B | 0.0 %|0.0 %|90.0 %|shmem| 0.000 B |shmem | 0.000 B
bar |172.17.0.5:3000| 0.000 |False | 0.0 %| 0.000 B | 0.0 %|0.0 %|90.0 %|shmem| 0.000 B |shmem | 0.000 B
bar | | 0.000 | | | 0.000 B | 0.0 %| | | | 0.000 B | | 0.000 B
test |172.17.0.3:3000| 0.000 |False | 0.0 %|16.169 MB|0.39 %|0.0 %|90.0 %|shmem|103.125 KB|shmem | 16.000 MB
test |172.17.0.4:3000| 0.000 |False | 0.0 %|16.164 MB|0.39 %|0.0 %|90.0 %|shmem| 99.625 KB|shmem | 16.000 MB
test |172.17.0.5:3000| 0.000 |False | 0.0 %|16.179 MB|0.39 %|0.0 %|90.0 %|shmem|108.812 KB|shmem | 16.000 MB
test | | 0.000 | | |48.511 MB|0.39 %| | | |311.562 KB| | 48.000 MB
Number of rows: 6
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Namespace Object Information (2023-03-21 23:44:05 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Namespace| Node|Rack| Repl|Expirations| Total|~~~~~~~~~~Objects~~~~~~~~~~|~~~~~~~~~Tombstones~~~~~~~~|~~~~Pending~~~~
| | ID|Factor| |Records| Master| Prole|Non-Replica| Master| Prole|Non-Replica|~~~~Migrates~~~
| | | | | | | | | | | | Tx| Rx
bar |172.17.0.3:3000| 0| 2| 0.000 |0.000 |0.000 |0.000 | 0.000 |0.000 |0.000 | 0.000 |0.000 |0.000
bar |172.17.0.4:3000| 0| 2| 0.000 |0.000 |0.000 |0.000 | 0.000 |0.000 |0.000 | 0.000 |0.000 |0.000
bar |172.17.0.5:3000| 0| 2| 0.000 |0.000 |0.000 |0.000 | 0.000 |0.000 |0.000 | 0.000 |0.000 |0.000
bar | | | | 0.000 |0.000 |0.000 |0.000 | 0.000 |0.000 |0.000 | 0.000 |0.000 |0.000
test |172.17.0.3:3000| 0| 1| 0.000 |1.650 K|1.650 K|0.000 | 0.000 |0.000 |0.000 | 0.000 |0.000 |0.000
test |172.17.0.4:3000| 0| 1| 0.000 |1.594 K|1.594 K|0.000 | 0.000 |0.000 |0.000 | 0.000 |0.000 |0.000
test |172.17.0.5:3000| 0| 1| 0.000 |1.741 K|1.741 K|0.000 | 0.000 |0.000 |0.000 | 0.000 |0.000 |0.000
test | | | | 0.000 |4.985 K|4.985 K|0.000 | 0.000 |0.000 |0.000 | 0.000 |0.000 |0.000
Number of rows: 6
Info set
The info set command displays a summary of important set
statistics for each set defined on each namespace on all nodes ordered by Set and
Namespace. If configured, it displays details about your storage quotas. It includes an extra row which displays an aggregate of grouped rows.
Example Output:
Collectinfo-analyzer> info set
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Set Information (2023-03-21 23:18:54 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Namespace| Set| Node| Memory| Disk|~~~~~~Quota~~~~~~~| Objects| Stop| Disable| Set
| | | Used| Used| Total| Used%| |Writes|Eviction|Index
| | | | | | | | Count| |
test |testset|172.17.0.3:3000| 37.534 KB|0.000 B | 48.828 KB|76.87 %|882.000 | 0|False |No
test |testset|172.17.0.4:3000| 37.326 KB|0.000 B | 48.828 KB|76.44 %|877.000 | 0|False |No
test |testset|172.17.0.5:3000| 38.353 KB|0.000 B | 48.828 KB|78.55 %|901.000 | 0|False |No
test |testset| |113.213 KB|0.000 B |146.484 KB|77.29 %| 2.660 K| | |
test |ufodata|172.17.0.3:3000| 32.640 KB|0.000 B | 0.000 B | --|768.000 | 0|False |No
test |ufodata|172.17.0.4:3000| 30.479 KB|0.000 B | 0.000 B | --|717.000 | 0|False |No
test |ufodata|172.17.0.5:3000| 35.700 KB|0.000 B | 0.000 B | --|840.000 | 0|False |No
test |ufodata| | 98.818 KB|0.000 B | 0.000 B | 0.0 %| 2.325 K| | |
Number of rows: 6
Further statistics for a set can be displayed using the show statistics command for specific sets:
Collectinfo-analyzer> show statistics sets for test1 testset
Info sindex
The info sindex command displays a summary of important sindex
statistics for each sindex defined on each namespace on all nodes ordered by Sindex and
Node.
Example Output:
Collectinfo-analyzer> info sindex
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Secondary Index Information (2020-12-16 23:10:06 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Index Name|Namespace| Set| Node| Bins| Bin|State|Keys|~~~~~~~~~~Entries~~~~~~~~~~|~~~~Storage~~~~~|~~~~Queries~~~~~|~~~~Updates~~~~~| Context
| | | | | Type| | | Total| Avg Per| Avg Per| Type| Used|Requests|Avg Num| Writes|Deletes|
| | | | | | | | | Rec| Bin Val| | | Recs| | | |
name-sindex|bar |testset|10.0.0.1:3000| name|STRING|RW | 2| 1.000 K | 1.000 | 0.500 K |shmem| 16.000 MB| 0.000 |0.000 | 5.000 |0.000 |--
name-sindex|bar |testset|10.0.0.3:3000| name|STRING|RW | 2| 1.000 K | 1.000 | 0.500 K |shmem| 16.000 MB| 0.000 |0.000 | 5.000 |0.000 |--
name-sindex|bar |testset|10.0.0.4:3000| name|STRING|RW | 2| 1.000 K | 1.000 | 0.500 K |shmem| 16.000 MB| 0.000 |0.000 | 3.000 |0.000 |--
name-sindex|bar |testset|10.0.0.5:3000| name|STRING|RW | 2| 1.000 K | 1.000 | 0.500 K |shmem| 16.000 MB| 0.000 |0.000 | 4.000 |0.000 |--
name-sindex|bar |testset|10.0.0.6:3000| name|STRING|RW | 2| 1.000 K | 1.000 | 0.500 K |shmem| 16.000 MB| 0.000 |0.000 | 3.000 |0.000 |--
|bar |testset| | | | | | 5.000 K | | 2.500 K | | 80.000 MB| 0.000 |0.000 |20.000 |0.000 |--
age-sindex |test |testset|10.0.0.3:3000| age|STRING|RW | 0| 0.000 | 1.000 | 0.000 |shmem| 16.000 MB| 0.000 |0.000 | 0.000 |0.000 |[list_index(-1), map_key(<string#11>)]
age-sindex |test |testset|10.0.0.1:3000| age|STRING|RW | 0| 0.000 | 1.000 | 0.000 |shmem| 16.000 MB| 0.000 |0.000 | 0.000 |0.000 |[list_index(-1), map_key(<string#11>)]
age-sindex |test |testset|10.0.0.4:3000| age|STRING|RW | 0| 0.000 | 1.000 | 0.000 |shmem| 16.000 MB| 0.000 |0.000 | 0.000 |0.000 |[list_index(-1), map_key(<string#11>)]
age-sindex |test |testset|10.0.0.5:3000| age|STRING|RW | 0| 0.000 | 1.000 | 0.000 |shmem| 16.000 MB| 0.000 |0.000 | 0.000 |0.000 |[list_index(-1), map_key(<string#11>)]
age-sindex |test |testset|10.0.0.6:3000| age|STRING|RW | 0| 0.000 | 1.000 | 0.000 |shmem| 16.000 MB| 0.000 |0.000 | 0.000 |0.000 |[list_index(-1), map_key(<string#11>)]
|test |testset| | | | | | 0.000 | 1.000 | 0.000 | | 80.000 MB| 0.000 |0.000 | 0.000 |0.000 |[list_index(-1), map_key(<string#11>)]
Number of rows: 10
Further statistics for a secondary index can be displayed using the show statistics command for specific sindex:
Collectinfo-analyzer> show statistics sindex for test test_str_idx
Info xdr
The info xdr command shows the current performance characteristics of XDR on each node.
Example Output:
Collectinfo-analyzer> info xdr
~~~~~~~~~~~~~~~~~~~~~~~~~~~~XDR Information DC1 (2020-12-17 00:11:48 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Node|Success|~~~~~~~~Retry~~~~~~~~~|Recoveries| Lag| Avg|Throughput
| |Connection|Destination| Pending|(hh:mm:ss)|Latency| (rec/s)
| | Reset| | | | (ms)|
10-0-0-3.heln.qwest.net:3000| 224| 0| 0| 0| 00:00:00| 0| 1078
10-0-0-5.heln.qwest.net:3000| 223| 0| 0| 0| 00:00:00| 0| 970
| | | | 0| | 0|
Number of rows: 2
~~~~~~~~~~~~~~~~~~~~~~~~~~~~XDR Information DC2 (2020-12-17 00:11:48 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Node|Success|~~~~~~~~Retry~~~~~~~~~|Recoveries| Lag| Avg|Throughput
| |Connection|Destination| Pending|(hh:mm:ss)|Latency| (rec/s)
| | Reset| | | | (ms)|
10-0-0-4.heln.qwest.net:3000| 199| 0| 0| 0| 00:00:00| 0| 898
10-0-0-6.heln.qwest.net:3000| 201| 0| 0| 0| 00:00:00| 0| 988
| | | | 0| | 0|
Number of rows: 2
Info dc
(Introduced: 0.0.16)
The info dc command displays a summary of important datacenter
statistics for each datacenter.
This feature is replaced by info xdr on servers versions 5.0 and above.
In the example below, DC information for only one snapshot is displayed as second collectinfo is created by old version of asadm so it does not have dc stats in it.
Collectinfo-analyzer> info dc
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~DC Information (2020-12-18 18:12:25 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Node| DC| DC Type|Namespaces| Lag|Records| Avg| Status
| | | | |Shipped|Latency|
| | | | | | (ms)|
10-0-0-1.heln.qwest.net:3000|aerospike_b|aerospike|test |00:00:00| 44452| 50|CLUSTER_UP
10-0-0-2.heln.qwest.net:3000|aerospike_b|aerospike|test |00:00:00| 45307| 52|CLUSTER_UP
10-0-0-1.heln.qwest.net:3000|aerospike_c|aerospike|test |00:00:00| 44452| 54|CLUSTER_UP
10-0-0-2.heln.qwest.net:3000|aerospike_c|aerospike|test |00:00:00| 45307| 56|CLUSTER_UP
Number of rows: 4
Show
The show commands generally provide a very verbose output about the
requested component. Most support the like modifier.
Best Practices
(Introduced: 2.4.0)
This command is supported in server v. 5.7 and later.
The show best-practices command is used to display violations of Aerospike's best-practices.
An explanation of each of Aerospike's best-practices can be found at Best-Practices
In the example below node BB9010016AE4202 is violating two best-practices, swappiness and thp-enabled, which will be displayed in red. Node
BB9030016AE4202, BB9040016AE4202 are not violating any best-practices and display ok in green.
Collectinfo-analyzer> show best-practices
~Best Practices (2021-09-21 23:55:09 UTC)~
Node|Response
BB9010016AE4202|swappiness, thp-enabled
BB9030016AE4202|ok
BB9040016AE4202|ok
Number of rows: 3
Following Aerospike's best-practices are required for optimal stability and performance.
Configuration
The show config command is used to display Aerospike configuration
settings. By default the command lists all server configuration parameters
for security (added in tools 7.0, otherwise joined with service), service,
network, and namespace but you may also provide one of the sub-commands:
xdr, security, service, network, and namespace to limit the output to just those contexts.
See below for details and additional subcommands for show config xdr.
In the example below, we request all network configuration parameters containing the words heartbeat or mesh.
Collectinfo-analyzer> show config network like heartbeat mesh
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Network Configuration (2020-12-17 01:07:36 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Node | 10-0-0-1.heln.qwest.net:3000| 10-0-0-2.heln.qwest.net:3000| 10-0-0-4.heln.qwest.net:3000| 10-0-0-5.heln.qwest.net:3000| 10-0-0-6.heln.qwest.net:3000
heartbeat.connect-timeout-ms|500 |500 |500 |500 |500
heartbeat.interval |150 |150 |150 |150 |150
heartbeat.mode |multicast |multicast |multicast |multicast |multicast
heartbeat.mtu |1500 |1500 |1500 |1500 |1500
heartbeat.multicast-group |239.1.99.200 |239.1.99.200 |239.1.99.200 |239.1.99.200 |239.1.99.200
heartbeat.port |9918 |9918 |9918 |9918 |9918
heartbeat.protocol |v3 |v3 |v3 |v3 |v3
heartbeat.timeout |10 |10 |10 |10 |10
Number of rows: 9
We can use the diff modifier with show config commands. To show differences
between node configurations.
Example Output:
Collectinfo-analyzer> show config diff
~~~~~~~~~~~~~~~~~~~~~~~~~~Service Configuration (2020-12-17 01:09:07 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~
Node | 10-0-0-1.heln.qwest.net:3000| 10-0-0-2.heln.qwest.net:3000| 10-0-0-4.heln.qwest.net:3000
pidfile|/var/run/aerospike/asd0.pid |/var/run/aerospike/asd1.pid |/var/run/aerospike/asd2.pid
Number of rows: 2
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Network Configuration (2020-12-17 01:09:07 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Node | 10-0-0-1.heln.qwest.net:3000| 10-0-0-2.heln.qwest.net:3000| 10-0-0-4.heln.qwest.net:3000
heartbeat-address|192.168.120.110 |192.168.120.112 |192.168.120.113
Number of rows: 2
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~test Namespace Configuration (2020-12-17 01:09:07 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Node | 10-0-0-1.heln.qwest.net:3000| 10-0-0-2.heln.qwest.net:3000| 10-0-0-4.heln.qwest.net:3000
migrate-rx-partitions-initial|4036 |3904 |3614
migrate-tx-partitions-initial|3362 |4096 |4096
Number of rows: 3
~~~~~~~~~~~~~~~~~~~~bar Namespace Configuration (2020-12-17 01:09:07 UTC)~~~~~~~~~~~~~~~~~~~~~~~~
Node| 10-0-0-1.heln.qwest.net:3000| 10-0-0-2.heln.qwest.net:3000| 10-0-0-4.heln.qwest.net:3000
Number of rows: 1
For large clusters we can use -flip option to flip output table for simplicity and ease of understanding.
Example Output:
Collectinfo-analyzer> show config namespace like partition -flip
~~~~~~~~~~test Namespace Configuration (2020-12-17 01:19:14 UTC)~~~~~~~~~~
Node|partition-tree-sprigs|sindex.num-partitions
10-0-0-1.heln.qwest.net:3000| 256| 32
10-0-0-2.heln.qwest.net:3000| 256| 32
10-0-0-4.heln.qwest.net:3000| 256| 32
10-0-0-5.heln.qwest.net:3000| 256| 32
10-0-0-6.heln.qwest.net:3000| 256| 32
Number of rows: 5
~~~~~~~~~~bar Namespace Configuration (2020-12-17 01:19:14 UTC)~~~~~~~~~~~
Node|partition-tree-sprigs|sindex.num-partitions
10-0-0-1.heln.qwest.net:3000| 256| 32
10-0-0-2.heln.qwest.net:3000| 256| 32
10-0-0-4.heln.qwest.net:3000| 256| 32
10-0-0-5.heln.qwest.net:3000| 256| 32
10-0-0-6.heln.qwest.net:3000| 256| 32
Number of rows: 5
XDR Configuration
The show config xdr command displays all the available configuration information related to xdr. By default,
this command displays XDR configuration, XDR datacenter configuration, and XDR namespace configuration. You may also provide one of the sub-commands: dc, namespace, and filter to limit the output to a specific context. For example, to see configuration parameters for only namespace, use show config xdr namespace. All of the commands support the use of the for, like, and diff modifier.
show config xdr subcommands dc, namespace, and filter were added in asadm Tools package 8.2 (asadm 2.13).
The show config xdr dc command displays a new table for each configured datacenter. The command also supports the for modifier to filter by datacenter.
In the example below we get XDR datacenter configuration parameters that contain "max" for datacenter dc2:
Collectinfo-analyzer> show config xdr dc for dc2 like max
~~~~~~~~~XDR dc2 DC Configuration (2023-02-16 22:37:00 UTC)~~~~~~~~~
Node |10.0.0.4:3000|10.0.0.5:3000|10.0.0.6:3000
max-recoveries-interleaved|0 |0 |0
max-used-service-threads |0 |0 |0
Number of rows: 3
The show config xdr namespace command displays a new table for each configured XDR namespace. The command also supports the for modifier to filter first by namespace and then by datacenter.
In the example below we get xdr namespace configuration parameters that contain "sets" for datacenter dc2:
Collectinfo-analyzer> show config xdr namespace for test dc2 like sets
~~~~XDR test Namespace Configuration (2023-02-16 22:41:12 UTC)~~~~
Datacenter |dc2 |dc2 |dc2
Node |10.0.0.4:3000|10.0.0.5:3000|10.0.0.6:3000
ignored-sets |testset |testset |testset
ship-only-specified-sets|false |false |false
shipped-sets | | |
Number of rows: 5
The show config xdr filter command displays the xdr-filters that are set for a given namespace and datacenter. The command also supports the for modifier to filter first by datacenter and then by namespace.
Collectinfo-analyzer> show config xdr filter
~~~~~~~~~~~~~~~~~~~~~~~XDR Filters (2023-02-16 22:55:02 UTC)~~~~~~~~~~~~~~~~~~~~~~~
Namespace|Datacenter| Base64 Expression| Expression
bar |dc2 |null |null
test |dc2 |kxGRSJMEk1ECo2FnZRU|or(is_tombstone(), ge(bin_int("age"), 21))
Number of rows: 2
Distribution
(Introduced: 0.1.12)
The show distribution displays histograms
and supports the object_size, time_to_live and eviction histograms. Output will only
be displayed if histogram information is provided in collectinfo.
For object_size, we can set parameter -b to get bytewise distribution.
Example Output:
Collectinfo-analyzer> show distribution time_to_live
~~~~~~~~~~~~~~~~~~~~test - TTL Distribution in Seconds (2020-12-18 02:14:24 UTC)~~~~~~~~~~~~~~~~~~~~
Percentage of records having ttl less than or equal to value measured in Seconds
Node| 10%| 20%| 30%| 40%| 50%| 60%| 70%| 80%| 90%| 100%
10-0-0-1.heln.qwest.net:3000|427100|427100|427100|427100|427100|427100|427100|427100|427100|427100
10-0-0-2.heln.qwest.net:3000|427100|427100|427100|427100|427100|427100|427100|427100|427100|427100
10-0-0-3.heln.qwest.net:3000|427100|427100|427100|427100|427100|427100|427100|427100|427100|427100
10-0-0-4.heln.qwest.net:3000|427100|427100|427100|427100|427100|427100|427100|427100|427100|427100
10-0-0-6.heln.qwest.net:3000|427100|427100|427100|427100|427100|427100|427100|427100|427100|427100
Number of rows: 5
~~~~~~~~~~~~~~~~~~~~bar - TTL Distribution in Seconds (2020-12-18 02:14:24 UTC)~~~~~~~~~~~~~~~~~~~~~
Percentage of records having ttl less than or equal to value measured in Seconds
Node| 10%| 20%| 30%| 40%| 50%| 60%| 70%| 80%| 90%| 100%
10-0-0-1.heln.qwest.net:3000|425500|425500|425500|425500|425500|425500|425500|425500|425500|425500
10-0-0-2.heln.qwest.net:3000|425500|425500|425500|425500|425500|425500|425500|425500|425500|425500
10-0-0-3.heln.qwest.net:3000|425500|425500|425500|425500|425500|425500|425500|425500|425500|425500
10-0-0-4.heln.qwest.net:3000|425500|425500|425500|425500|425500|425500|425500|425500|425500|425500
10-0-0-6.heln.qwest.net:3000|425500|425500|425500|425500|425500|425500|425500|425500|425500|425500
Number of rows: 5
Jobs
(Introduced: 2.5.0)
The show jobs [scan|query|sindex-builder] command displays current and past jobs running on the aerospike cluster and should be used
in conjunction with the manage jobs controller. To make viewing easier, run the pager on command first.
By default all job modules are shown. Each module table is organized in a number of ways for easier viewing. It groups the jobs by their Namespace and Type.
Groups are separated by horizontal dashes. Jobs are further organized left to right by their Progress % and Time Since Done.
Note: SIndex-builder jobs were removed in server v. 5.7 and after.
Collectinfo-analyzer> show jobs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Scan Jobs (2021-10-20 23:08:14 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Node |10.0.0.3:3000 |10.0.0.2:3000 |10.0.0.1:3000
Namespace |bar |bar |bar
Module |scan |scan |scan
Type |basic |basic |basic
Progress % |100.0 |100.0 |100.0
Transaction ID |1583278212325152813 |1554763604191518487 |1554763604191518487
Time Since Done |00:33:26 |00:34:42 |00:34:43
active-threads |0 |0 |0
from |10.0.22.1+52252 |10.0.22.1+34048 |10.0.22.1+40340
n-pids-requested |1.366 K |1.365 K |1.365 K
net-io-bytes |37.940 MB |8.505 MB |8.048 MB
priority |0 |0 |0
recs-failed |0.000 |0.000 |0.000
recs-filtered-bins|0.000 |0.000 |0.000
recs-filtered-meta|0.000 |0.000 |0.000
recs-succeeded |333.874 K |75.826 K |71.779 K
recs-throttled |333.874 K |75.826 K |71.779 K
rps |0.000 |0.000 |0.000
run-time |00:00:05 |00:00:01 |00:00:01
socket-timeout |00:00:30 |00:00:30 |00:00:30
status |done(ok) |done(abandoned-response-timeout)|done(abandoned-response-timeout)
-------------------------------------------------------------------------------------------------------------------
Node |10.0.0.3:3000 |10.0.0.2:3000 |10.0.0.1:3000
Namespace |test |test |test
Module |scan |scan |scan
Type |basic |basic |basic
Progress % |100.0 |100.0 |100.0
Transaction ID |17709699727074092152|17709699727074092152 |17709699727074092152
Time Since Done |00:47:59 |00:47:59 |00:47:59
active-threads |0 |0 |0
from |10.0.22.1+51868 |10.0.22.1+33716 |10.22.22.1+40008
n-pids-requested |1.366 K |1.365 K |1.365 K
net-io-bytes |438.377 KB |443.145 KB |442.441 KB
priority |0 |0 |0
recs-failed |0.000 |0.000 |0.000
recs-filtered-bins|0.000 |0.000 |0.000
recs-filtered-meta|0.000 |0.000 |0.000
recs-succeeded |3.308 K |3.349 K |3.343 K
recs-throttled |3.308 K |3.349 K |3.343 K
rps |0.000 |0.000 |0.000
run-time |00:00:00 |00:00:00 |00:00:00
socket-timeout |00:00:30 |00:00:30 |00:00:30
status |done(ok) |done(ok) |done(ok)
Number of rows: 42
~~~~~~~~~~~~~~~~~~~~~Query Jobs (2021-10-20 23:08:14 UTC)~~~~~~~~~~~~~~~~~~~~~
Node |10.0.0.1:3000 |10.0.0.3:3000 |10.0.0.2:3000
Namespace |bar |bar |bar
Module |query |query |query
Progress % |0.0 |0.0 |0.0
Transaction ID |2143237531128163351|2143237531128163351|2143237531128163351
Time Since Done |00:00:00 |00:00:00 |00:00:00
active-threads |0 |0 |0
net-io-bytes |2.400 MB |2.087 MB |2.681 MB
priority |10 |10 |10
recs-failed |0.000 |0.000 |0.000
recs-filtered-bins|0.000 |0.000 |0.000
recs-filtered-meta|0.000 |0.000 |0.000
recs-succeeded |32.558 K |29.274 K |36.467 K
recs-throttled |0.000 |0.000 |0.000
rps |0.000 |0.000 |0.000
run-time |00:00:07 |00:00:07 |00:00:07
set |testset |testset |testset
sindex-name |a-bar-index |a-bar-index |a-bar-index
socket-timeout |00:00:00 |00:00:00 |00:00:00
status |active |active |active
Number of rows: 20
Latencies
(Introduced: 0.7.0)
The show latencies command displays latencies characteristics of reads, writes,
queries, replication, and udf. Output will only be displayed if latency information is provided in collectinfo.
In the below example we look at the latency of writes_master.
Collectinfo-analyzer> show latencies -v like write-master
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Latency (2020-12-17 19:19:12 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Namespace| Histogram| Node|ops/sec|>1ms|>2ms|>4ms|>8ms|>16ms|>32ms|>64ms|>128ms|>256ms|>512ms|>1024ms|>2048ms|>4096ms|>8192ms|>16384ms|>32768ms|>65536ms
test |write-master| 10.0.0.1:3000|0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0
test |write-master| 10.0.0.2:3000|0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0
test |write-master| 10.0.0.4:3000|0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0
test |write-master| 10.0.0.5:3000|0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0
test |write-master| 10.0.0.6:3000|0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0
| | |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0
Number of rows: 5
Latency
(Introduced: 0.1.15)
(Removed: 0.7.0)
The show latency command displays latency characteristics of reads, writes,
and proxies. Output will only be displayed if latency information is provided in collectinfo.
In the below example we look at the latency of writes_master.
Collectinfo-analyzer> show latency -v like writes_master
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Latency (2020-12-17 02:08:39 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Namespace| Histogram| Node|ops/sec|>1ms|>2ms|>4ms|>8ms|>16ms|>32ms|>64ms|>128ms|>256ms|>512ms|>1024ms|>2048ms|>4096ms|>8192ms|>16384ms|>32768ms|>65536ms
test |write-master| 10-0-0-1.heln.qwest.net:3000|0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0
test |write-master| 10-0-0-2.heln.qwest.net:3000|0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0
test |write-master| 10-0-0-4.heln.qwest.net:3000|0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0
test |write-master| 10-0-0-5.heln.qwest.net:3000|0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0
test |write-master| 10-0-0-6.heln.qwest.net:3000|0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0
| | |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0 |0.0
Number of rows: 5
Pmap
(Introduced: 0.1.12)
The show pmap command displays partition map analysis of the Aerospike cluster.
Output will only be displayed if partition information is provided in collectinfo.
The following example shows the output of the show pmap command.
Primary Partitions: Total number of primary partitions for a specific namespace on that node.
Secondary Partitions: Total number of secondary partitions for a specific namespace on that node.
Unavailable Partitions: The number of partitions that are unavailable when roster nodes are missing.
Dead Partitions: The number of partitions that are unavailable when all roster nodes are present.
Collectinfo-analyzer> show pmap
~~~~~~~~~~~~~~~~~~~~Partition Map Analysis (2020-12-18 01:12:36 UTC)~~~~~~~~~~~~~~~~~~~~
Namespace| Node| Cluster Key|~~~~~~~~~~~~Partitions~~~~~~~~~~~~
| | |Primary|Secondary|Unavailable|Dead
bar | 10-0-0-1.heln.qwest.net:3000|33718FC58CD6| 791| 799| 0| 0
bar | 10-0-0-2.heln.qwest.net:3000|33718FC58CD6| 868| 822| 0| 0
bar | 10-0-0-3.heln.qwest.net:3000|33718FC58CD6| 839| 862| 0| 0
bar | 10-0-0-4.heln.qwest.net:3000|33718FC58CD6| 800| 780| 0| 0
bar | 10-0-0-6.heln.qwest.net:3000|33718FC58CD6| 798| 833| 0| 0
bar | | | 4096| 4096| 0| 0
test | 10-0-0-1.heln.qwest.net:3000|33718FC58CD6| 791| 799| 0| 0
test | 10-0-0-2.heln.qwest.net:3000|33718FC58CD6| 868| 822| 0| 0
test | 10-0-0-3.heln.qwest.net:3000|33718FC58CD6| 839| 862| 0| 0
test | 10-0-0-4.heln.qwest.net:3000|33718FC58CD6| 800| 780| 0| 0
test | 10-0-0-6.heln.qwest.net:3000|33718FC58CD6| 798| 833| 0| 0
test | | | 4096| 4096| 0| 0
Number of rows: 10
Racks
(Introduced: 2.5.0)
The show racks command displays a namespaces' rack-ids and the nodes assigned to each.
This is particularly useful in rack-aware configurations.
Collectinfo-analyzer> show racks
~~~~~~~~~~~~~~~~Racks (2021-10-21 20:33:28 UTC)~~~~~~~~~~~~~~~~~
Namespace|Rack| Nodes
| ID|
bar |4 |BB9040016AE4202, BB9020016AE4202, BB9010016AE4202
test |2 |BB9040016AE4202, BB9010016AE4202
Number of rows: 2
Roles
(Introduced: 2.1.0)
The show roles command displays roles along with associated privileges, allowlist,
and quotas as returned by the principal node.
Collectinfo-analyzer> show roles
~~~~~~~~~~~~~~~~~~~~~~~~~~~Roles (2021-04-21 22:28:01 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Role| Privileges|Allowlist|~~~Quotas~~
| | | Read|Write
read | read| --|-- |--
read-write | read-write| --|-- |--
read-write-udf| read-write-udf| --|-- |--
reader | read| 1.1.1.1|10000|1
root | user-admin, sys-admin, data-admin, read-write| --|-- |--
superuser |user-admin, sys-admin, data-admin, read-write-udf| --|-- |--
sys-admin | sys-admin| --|-- |--
user-admin | user-admin| --|-- |--
write | write| --|-- |--
writer | read-write| 2.2.2.2|1 |10000
Number of rows: 10
Roster
(Introduced: 2.5.0)
The show roster command displays the current and pending roster as well as the observed nodes.
To make viewing easier, run the pager on command first. To filter output based on namespace, use the for modifier.
To filter output based on node use the with modifier. To display any differences between values
in any given column use the diff modifier.
Collectinfo-analyzer> show roster
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Roster (2021-10-21 20:12:29 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Node| Node ID|Namespace| Current Roster| Pending Roster| Observed Nodes
10.0.0.1:3000|BB9010016AE4202 |bar |BB9040016AE4202@4, BB9020016AE4202@4, BB9010016AE4202@4|BB9040016AE4202@4, BB9020016AE4202@4, BB9010016AE4202@4|BB9040016AE4202@4, BB9020016AE4202@4, BB9010016AE4202@4
10.0.0.2:3000|BB9020016AE4202 |bar |BB9040016AE4202@4, BB9020016AE4202@4, BB9010016AE4202@4|BB9040016AE4202@4, BB9020016AE4202@4, BB9010016AE4202@4|BB9040016AE4202@4, BB9020016AE4202@4, BB9010016AE4202@4
10.0.0.4:3000|*BB9040016AE4202|bar |BB9040016AE4202@4, BB9020016AE4202@4, BB9010016AE4202@4|BB9040016AE4202@4, BB9020016AE4202@4, BB9010016AE4202@4|BB9040016AE4202@4, BB9020016AE4202@4, BB9010016AE4202@4
10.0.0.1:3000|BB9010016AE4202 |test |BB9040016AE4202@2, BB9020016AE4202@2, BB9010016AE4202@2|BB9040016AE4202@2, BB9020016AE4202@2, BB9010016AE4202@2|BB9040016AE4202@2, BB9020016AE4202@2, BB9010016AE4202@2
10.0.0.2:3000|BB9020016AE4202 |test |BB9040016AE4202@2, BB9020016AE4202@2, BB9010016AE4202@2|BB9040016AE4202@2, BB9020016AE4202@2, BB9010016AE4202@2|BB9040016AE4202@2, BB9020016AE4202@2, BB9010016AE4202@2
10.0.0.4:3000|*BB9040016AE4202|test |BB9040016AE4202@2, BB9020016AE4202@2, BB9010016AE4202@2|BB9040016AE4202@2, BB9020016AE4202@2, BB9010016AE4202@2|BB9040016AE4202@2, BB9020016AE4202@2, BB9010016AE4202@2
Number of rows: 6
Secondary indexes
(Introduced: 2.1.0)
The show sindex command displays secondary indexes and associated static metadata as
returned by the principal node.
Collectinfo-analyzer> show sindex
~~~~~~Secondary Indexes (2021-01-22 23:04:49 UTC)~~~~~~
Index Name|Namespace| Set| Bin| Bin| Index|State
| | | | Type| Type|
name-sindex|bar |NULL|name|STRING |NONE |RW
age-index |test |NULL| age|NUMERIC|MAPVALUES|RW
job-index |test |NULL| age|STRING |MAPVALUES|RW
Number of rows: 3
Secondary index on a nested context (Introduced: 6.1.0)
A secondary index may be created on a Map (document) or List data type bin to identify a value of that type to index. The show sindex command displays the nested index as follows, with a Context that identifies the base64 encoded value. In this example, a bin report has a Map type, within it there’s a map key shape holding a list of strings:
Collectinfo-analyzer> show sindex
~~~~~~~~~~~~~~~~Secondary Indexes (2023-01-11 00:58:31 UTC)~~~~~~~~~~~~~~~~
Index|Namespace| Set| Bin| Bin|Index|State| Context
Name| | | | Type| Type| |
shape-idx|sandbox |ufodata|report|string|list |RW |[map_key(<string#5>)]
Number of rows: 1
See Use an SI query in your application.
Statistics
The show statistics command displays all server statistics from several
server components. By default it returns statistics for bin, set, service, and namespace but the
sub-commands bins, namespace, service, sets, sindex, xdr, and dc will confine the output to
a single context. See below for details and additional subcommands for show config xdr. Also, we can set -t parameter to get an extra aggregate column for total across columns. Total column displays
sum of statistics with numeric values.
The example below displays service level statistics while filtering for metric containing the token "batch" and displaying a total column:
Collectinfo-analyzer> show statistics service like batch -t
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Service Statistics (2020-12-18 01:33:36 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Node | 10-0-0-1.heln.qwest.net:3000| 10-0-0-2.heln.qwest.net:3000| 10-0-0-3.heln.qwest.net:3000|
batch_index_complete |0 |0 |0 |0
batch_index_created_buffers |0 |0 |0 |0
batch_index_delay |0 |0 |0 |0
batch_index_destroyed_buffers |0 |0 |0 |0
batch_index_error |0 |0 |0 |0
batch_index_huge_buffers |0 |0 |0 |0
batch_index_initiate |0 |0 |0 |0
batch_index_proto_compression_ratio |1.0 |1.0 |1.0 |3.0
batch_index_proto_uncompressed_pct |0.0 |0.0 |0.0 |0.0
batch_index_queue |0:0,0:0 |0:0,0:0 |0:0,0:0 |
batch_index_timeout |0 |0 |0 |0
batch_index_unused_buffers |0 |0 |0 |0
early_tsvc_batch_sub_error |0 |0 |0 |0
early_tsvc_from_proxy_batch_sub_error|0 |0 |0 |0
Number of rows: 15
For large clusters we can use the -flip option to flip the output for readability.
Example Output:
Collectinfo-analyzer> show statistics namespace for test like migrate_tx_partitions_i -flip
~~~~~~~~~~~~~~~~~~~~test Namespace Statistics (2020-12-18 02:08:43 UTC)~~~~~~~~~~~~~~~~~~~~~
Node|migrate_tx_partitions_imbalance|migrate_tx_partitions_initial
10-0-0-1.heln.qwest.net:3000| 0| 390
Number of rows: 1
XDR Statistics
The show statistics xdr command displays all the available statistics information related to xdr. By default,
this command displays xdr datacenter statistics and xdr namespace statistics. You may also provide one of the sub-commands: dc and namespace to limit the output to a specific context.
show statstics xdr subcommands dc and namespace were added in asadm Tools package 8.2 (asadm 2.13).
The show statistics xdr dc command displays a new table for each configured datacenter. The command also supports the for modifier to filter by datacenter.
Collectinfo-analyzer> show statistics xdr dc for dc2 like retry
~~~~~~~~~XDR dc2 DC Statistics (2023-02-16 23:56:28 UTC)~~~~~~~~~~
Node |172.17.0.4:3000|172.17.0.5:3000|172.17.0.6:3000
retry_conn_reset|0 |0 |0
retry_dest |0 |0 |0
retry_no_node |0 |0 |0
Number of rows: 4
The show statistics xdr namespace command displays a new table for each configured xdr namespace. The command also supports the for modifier to filter first by namespace and then by datacenter.
Collectinfo-analyzer> show statistics xdr namespace like retry
~~~~~~~~~~~~~~~~~~~~~~XDR test Namespace Statistics (2023-02-16 23:57:32 UTC)~~~~~~~~~~~~~~~~~~~~~~~
Datacenter |dc1 |dc1 |dc1 |dc2 |dc2 |dc2
Node |10.0.0.4:3000|10.0.0.5:3000|10.0.0.6:3000|10.0.0.4:3000|10.0.0.5:3000|10.0.0.6:3000
retry_conn_reset|0 |0 |0 |0 |0 |0
retry_dest |0 |0 |0 |0 |0 |0
retry_no_node |0 |0 |0 |0 |0 |0
Number of rows: 5
~~~~~~~~~~~~~~~~~~~~~~XDR bar Namespace Statistics (2023-02-16 23:57:32 UTC)~~~~~~~~~~~~~~~~~~~~~~~
Datacenter |dc1 |dc1 |dc1 |dc2 |dc2 |dc2
Node |10.0.0.4:3000|10.0.0.5:3000|10.0.0.6:3000|10.0.0.4:3000|10.0.0.5:3000|10.0.0.6:3000
retry_conn_reset|0 |0 |0 |0 |0 |0
retry_dest |0 |0 |0 |0 |0 |0
retry_no_node |0 |0 |0 |0 |0 |0
Number of rows: 5
To instead display a new table for each configured datacenter use the --by-dc flag.
Collectinfo-analyzer> show statistics xdr namespace like retry --by-dc
~~~~~~~~~~~~~~~~~~~~~~XDR dc1 Namespace Statistics (2023-02-16 23:57:32 UTC)~~~~~~~~~~~~~~~~~~~~~~~
Namespace |test |test |test |bar |bar |bar
Node |10.0.0.4:3000|10.0.0.5:3000|10.0.0.6:3000|10.0.0.4:3000|10.0.0.5:3000|10.0.0.6:3000
retry_conn_reset|0 |0 |0 |0 |0 |0
retry_dest |0 |0 |0 |0 |0 |0
retry_no_node |0 |0 |0 |0 |0 |0
Number of rows: 5
~~~~~~~~~~~~~~~~~~~~~~XDR dc2 Namespace Statistics (2023-02-16 23:57:32 UTC)~~~~~~~~~~~~~~~~~~~~~~~
Namespace |test |test |test |bar |bar |bar
Node |10.0.0.4:3000|10.0.0.5:3000|10.0.0.6:3000|10.0.0.4:3000|10.0.0.5:3000|10.0.0.6:3000
retry_conn_reset|0 |0 |0 |0 |0 |0
retry_dest |0 |0 |0 |0 |0 |0
retry_no_node |0 |0 |0 |0 |0 |0
Number of rows: 5
Stop Writes
(Introduced: 2.15.0)
The show stop-writes command in the asadm tool provides comprehensive information about stop-writes configuration parameters, metrics, and their associated context, i.e. namespace and test. This command is particularly helpful in determining the proximity to reaching the stop-writes threshold at different levels: service context (global), namespace context, or set context. It also assists in identifying the reasons for being in the stop-writes state.
show stop-writes displays the following table which is ordered based on the proximity to breaching the configured stop-writes threshold. For instance, the stop-writes-count configuration for the namespace test and set testset is closest to reaching the limit of 10,000 records and is positioned at the bottom of the table. This arrangement helps in effectively addressing the issue by providing the relevant configuration details and the metric that might potentially exceed the threshold. Additionally, the table presents the current proximity to the configured threshold, actual usage, and the threshold itself, offering a clear understanding of the current status. A -- threshold means none is configured.
Collectinfo-analyzer> show stop-writes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Stop Writes (2023-05-23 23:01:01 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Show all stop writes - add 'for <namespace> [<set>]' for a shorter list.
Config|Namespace| Set| Node|Stop-Writes| Metric| Usage%| Usage|Threshold
stop-writes-size |test |testset|172.17.0.5:3000|False |memory_data_bytes | --|123.005 KB| --
stop-writes-size |test |testset|172.17.0.4:3000|False |memory_data_bytes | --|123.373 KB| --
stop-writes-size |test |testset|172.17.0.3:3000|False |memory_data_bytes | --|123.246 KB| --
-- |test |-- |172.17.0.5:3000|False |cluster_clock_skew_ms| --| 00:00:00| --
-- |bar |-- |172.17.0.5:3000|False |cluster_clock_skew_ms| --| 00:00:00| --
-- |test |-- |172.17.0.4:3000|False |cluster_clock_skew_ms| --| 00:00:00| --
-- |bar |-- |172.17.0.4:3000|False |cluster_clock_skew_ms| --| 00:00:00| --
-- |test |-- |172.17.0.3:3000|False |cluster_clock_skew_ms| --| 00:00:00| --
-- |bar |-- |172.17.0.3:3000|False |cluster_clock_skew_ms| --| 00:00:00| --
stop-writes-pct |bar |-- |172.17.0.3:3000|False |memory_used_bytes | 0.0 %| 0.000 B | 3.600 GB
stop-writes-pct |bar |-- |172.17.0.4:3000|False |memory_used_bytes | 0.0 %| 0.000 B | 3.600 GB
stop-writes-pct |bar |-- |172.17.0.5:3000|False |memory_used_bytes | 0.0 %| 0.000 B | 3.600 GB
stop-writes-pct |test |-- |172.17.0.5:3000|False |memory_used_bytes | 1.74 %|728.567 KB|40.960 MB
stop-writes-pct |test |-- |172.17.0.3:3000|False |memory_used_bytes | 1.74 %|729.996 KB|40.960 MB
stop-writes-pct |test |-- |172.17.0.4:3000|False |memory_used_bytes | 1.74 %|730.748 KB|40.960 MB
stop-writes-sys-memory-pct|bar |-- |172.17.0.3:3000|False |system_free_mem_pct |28.89 %| 26.0 %| 90.0 %
stop-writes-sys-memory-pct|test |-- |172.17.0.3:3000|False |system_free_mem_pct |28.89 %| 26.0 %| 90.0 %
stop-writes-sys-memory-pct|bar |-- |172.17.0.4:3000|False |system_free_mem_pct |28.89 %| 26.0 %| 90.0 %
stop-writes-sys-memory-pct|test |-- |172.17.0.4:3000|False |system_free_mem_pct |28.89 %| 26.0 %| 90.0 %
stop-writes-sys-memory-pct|bar |-- |172.17.0.5:3000|False |system_free_mem_pct |28.89 %| 26.0 %| 90.0 %
stop-writes-sys-memory-pct|test |-- |172.17.0.5:3000|False |system_free_mem_pct |28.89 %| 26.0 %| 90.0 %
stop-writes-count |test |testset|172.17.0.5:3000|False |objects |96.89 %| 9.689 K| 10.000 K
stop-writes-count |test |testset|172.17.0.3:3000|False |objects |97.08 %| 9.708 K| 10.000 K
stop-writes-count |test |testset|172.17.0.4:3000|False |objects |97.18 %| 9.718 K| 10.000 K
Number of rows: 24
User Defined Functions
(Introduced: 2.1.0)
The show udfs command displays user defined functions (udfs) modules as
returned by the principal node.
Collectinfo-analyzer> show udfs
~~~~~~~~UDF Modules (2021-01-22 23:12:29 UTC)~~~~~~~~~
Filename| Hash|Type
abc.123 |dceaf7f1acddf1d6e12a1752d499d80cfadfc24b|LUA
bar.lua |591d2536acb21a329040beabfd9bfaf110d35c18|LUA
foo.lua |f6eaf2b22d8b29b3597ef1ad9113d0907425ecd0|LUA
Users
(Introduced: 2.1.0)
Access Control Permissions: user-admin
The show users [user] command displays users along with their associated roles as
returned by the principal node. Optionally, you can display a single user by providing a username as the first argument.
User runtime statistics were moved to the show users statistics command in Tools package 8.4.0 (Asadm 2.15.0).
In asadm 2.2.0 to 2.14.0 inclusive, runtime statistics were located in the show users table if quotas were enabled but only accounted for the principal node.
Collectinfo-analyzer> show users
~~Users (2023-05-24 20:52:11 UTC)~~
To see users statistics run 'show
user statistics'
User| Roles|~Read~|~Write~
| | Quota| Quota
admin |user-admin|0 |0
reader | reader|10000 |1
root | root|0 |0
superuser| superuser|0 |0
writer | writer|1 |10000
Number of rows: 5
Users Statistics
(Introduced: 2.15.0)
Access Control Permissions: user-admin
The show users statistics [user] command displays users, number of user connections, and quota-related metrics across all nodes in the cluster.
Use this to see the activity of users and find out which users might be close to or exceeding their assigned quotas. In addition
to viewing users per node, there is also an additional aggregate line to display usage for the entire cluster. Optionally, you can retrieve a single user
by providing a username as the first argument.
Collectinfo-analyzer> show users stat
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Users Statistics (2023-05-24 21:49:04 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
User| Node|Connections|~~~~~~~~~~~~~~~~~~~~Read~~~~~~~~~~~~~~~~~~~|~~~~~~~~~~~~~~~~~~~Write~~~~~~~~~~~~~~~~~~~
| | | Quota|Usage%| Single| PI/SI| PI/SI| Quota|Usage%| Single| PI/SI| PI/SI
| | | | | Record| Query| Query| | | Record| Query| Query
| | | | | TPS|Limited|Limitless| | | TPS|Limited|Limitless
| | | | | | RPS| | | | | RPS|
admin |172.17.0.3:3000| 2.000 | 0.000 | --| 0.000 |0.000 | 0.000 | 0.000 | --| 0.000 |0.000 | 0.000
admin |172.17.0.4:3000| 2.000 | 0.000 | --| 0.000 |0.000 | 0.000 | 0.000 | --| 0.000 |0.000 | 0.000
admin |172.17.0.5:3000| 2.000 | 0.000 | --| 0.000 |0.000 | 0.000 | 0.000 | --| 0.000 |0.000 | 0.000
admin | | 6.000 | 0.000 | 0.0 %| 0.000 |0.000 | 0.000 | 0.000 | 0.0 %| 0.000 |0.000 | 0.000
reader |172.17.0.3:3000| 13.000 |10.000 K|5.93 %|593.000 |0.000 | 0.000 | 1.000 | 0.0 %| 0.000 |0.000 | 0.000
reader |172.17.0.4:3000| 13.000 |10.000 K|5.15 %|515.000 |0.000 | 0.000 | 1.000 | 0.0 %| 0.000 |0.000 | 0.000
reader |172.17.0.5:3000| 13.000 |10.000 K| 4.7 %|470.000 |0.000 | 0.000 | 1.000 | 0.0 %| 0.000 |0.000 | 0.000
reader | | 39.000 |30.000 K|5.26 %| 1.578 K|0.000 | 0.000 | 3.000 | 0.0 %| 0.000 |0.000 | 0.000
root |172.17.0.3:3000| --| 0.000 | --| 0.000 |0.000 | 0.000 | 0.000 | --| 0.000 |0.000 | 0.000
root |172.17.0.4:3000| --| 0.000 | --| 0.000 |0.000 | 0.000 | 0.000 | --| 0.000 |0.000 | 0.000
root |172.17.0.5:3000| --| 0.000 | --| 0.000 |0.000 | 0.000 | 0.000 | --| 0.000 |0.000 | 0.000
root | | --| 0.000 | 0.0 %| 0.000 |0.000 | 0.000 | 0.000 | 0.0 %| 0.000 |0.000 | 0.000
superuser|172.17.0.3:3000| 14.000 | 0.000 | --|263.000 |0.000 | 0.000 | 0.000 | --|271.000 |0.000 | 0.000
superuser|172.17.0.4:3000| 12.000 | 0.000 | --|225.000 |0.000 | 0.000 | 0.000 | --|257.000 |0.000 | 0.000
superuser|172.17.0.5:3000| 14.000 | 0.000 | --|227.000 |0.000 | 0.000 | 0.000 | --|226.000 |0.000 | 0.000
superuser| | 40.000 | 0.000 | 0.0 %|715.000 |0.000 | 0.000 | 0.000 | 0.0 %|754.000 |0.000 | 0.000
writer |172.17.0.3:3000| 14.000 | 1.000 | 0.0 %| 0.000 |0.000 | 0.000 |10.000 K|5.29 %|529.000 |0.000 | 0.000
writer |172.17.0.4:3000| 12.000 | 1.000 | 0.0 %| 0.000 |0.000 | 0.000 |10.000 K|4.56 %|456.000 |0.000 | 0.000
writer |172.17.0.5:3000| 14.000 | 1.000 | 0.0 %| 0.000 |0.000 | 0.000 |10.000 K|4.45 %|445.000 |0.000 | 0.000
writer | | 40.000 | 3.000 | 0.0 %| 0.000 |0.000 | 0.000 |30.000 K|4.77 %| 1.430 K|0.000 | 0.000
Number of rows: 15
Features
The features command displays features used in cluster. It supports like modifier.
Example Output:
Collectinfo-analyzer> features
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Features (2020-12-18 02:09:28 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Node | 10-0-0-1.heln.qwest.net:3000| 10-0-0-2.heln.qwest.net:3000| 10-0-0-3.heln.qwest.net:3000
AGGREGATION |NO |NO |NO
BATCH |NO |NO |NO
INDEX-ON-DEVICE|NO |NO |NO
INDEX-ON-PMEM |NO |NO |NO
KVS |YES |YES |YES
LDT |NO |NO |NO
QUERY |NO |NO |NO
RACK-AWARE |NO |NO |NO
SC |NO |NO |NO
SCAN |NO |NO |NO
SECURITY |NO |NO |NO
SINDEX |NO |NO |NO
TLS (FABRIC) |NO |NO |NO
TLS (HEARTBEAT)|NO |NO |NO
TLS (SERVICE) |NO |NO |NO
UDF |NO |NO |NO
XDR DESTINATION|NO |NO |NO
XDR SOURCE |NO |NO |NO
Number of rows: 19
Summary
(Introduced: 0.1.9)
The summary command displays summary of cluster. If the UDA was collected when the collectinfo was created using --agent-host and --agent-port you may
optionally use the --agent-unstable flag to include values in the Usage (Unique Data) value where the cluster was reportedly unstable.
Tools package 7.1.1 or later is required to use asadm's integration with the UDA
Example Output:
Collectinfo-analyzer> summary -l
Cluster
=======
1. Cluster Name : bob-cluster-a
2. Server Version : E-5.6.0.3
3. OS Version :
4. Cluster Size : 2
5. Devices : Total 2, per-node 1
6. Memory : Total 2.000 GB, 0.00% used (0.000 B), 100.00% available (2.000 GB)
7. Pmem Index : Total 16.000 GB, 0.00% used (0.000 B), 100.00% available (16.000 GB)
8. Disk : Total 8.000 GB, 0.00% used (0.000 B), 99.00% available contiguous space (7.920 GB)
9. Usage (Unique Data): None
10. Active Namespaces : 0 of 1
11. Features : Index-on-pmem, SC, Security, TLS (Service)
Namespaces
==========
test
====
1. Devices : Total 2, per-node 1
2. Memory : Total 2.000 GB, 0.00% used (0.000 B), 100.00% available (2.000 GB)
3. Pmem Index : Total 16.000 GB, 0.00% used (0.000 B), 100.00% available (16.000 GB)
4. Disk : Total 8.000 GB, 0.00% used (0.000 B), 99.00% available contiguous space (7.920 GB)
5. Replication Factor : 2
6. Rack-aware : False
7. Master Objects : 0.000
8. Compression-ratio : 1.0
Pager
The pager command sets pager for output. For output which can not fit in
output console, this command gives option to scroll each output table vertically as well as horizontally.