Skip to main content

Managing Queries

info

Prior to Server 6.0, primary index (PI) queries were called scans and secondary index (SI) queries were called queries. Both types of query are managed through a unified interface and run on a common subsystem.

Queries can be initiated in parallel, and each query uses its own dedicated threads, so that queries will not interfere with each other.

Long queries can be independently throttled, via a specified requested records per second (rps) value.

Tuning Queries

Aerospike allows you to tune various parameters that control the query subsystem, including:

For more information regarding these configuration parameters and their default values, please see Configuration Reference.

Updating Query Settings

The above parameters can be dynamically set in the cluster using Aerospike Admin (asadm), using the following command:

note

Tools package 6.0.x or later is required to use asadm's manage config commands. Otherwise, use the equivalent asinfo - set-config command.

asadm -e "enable; manage config service param <name> to <value>"

Where <name> is the configuration parameter name and <value> is the parameter value.

Query Job Management

Query job management is only relevant to long queries.

List Queries

Use the following command to list long queries with asadm in Tools package 6.2.x or later:

asadm -e 'show jobs queries'

Configure the number of completed queries to track with the query-max-done parameter. Active long queries are always tracked. Not specifying a trid (transaction id) will list all active queries and up to query-max-done most recently completed long queries.

Example for an SI query returning a single record:

Admin+> show jobs queries trid 15021089193528544137
~~~~~~~~~~~~~Query Jobs (2022-11-30 23:21:42 UTC)~~~~~~~~~~~~~
Node |mycluster-1:3000 |172.17.0.2:3000
Namespace |test |test
Module |query |query
Type |basic |basic
Progress % |100.0 |100.0
Transaction ID |15021089193528544137|15021089193528544137
Time Since Done |00:12:51 |00:12:51
active-threads |0 |0
from |127.0.0.1+60036 |172.17.0.3+55372
n-pids-requested |2.048 K |2.048 K
net-io-bytes |30.000 B |149.000 B
net-io-time |00:00:00 |00:00:00
recs-failed |0.000 |0.000
recs-filtered-bins|0.000 |0.000
recs-filtered-meta|0.000 |0.000
recs-succeeded |0.000 |1.000
recs-throttled |0.000 |0.000
rps |0.000 |0.000
run-time |00:00:00 |00:00:00
set |testset |testset
sindex-name |mysindex |mysindex
socket-timeout |00:00:30 |00:00:30
status |done(ok) |done(ok)
Number of rows: 23

List queries with asinfo against Server 6.0 or later:

asinfo -v 'query-show'
asinfo -v 'query-show:trid=<jobid>'

Examples (against a 6.2 server -- version 6.1 does not display the sindex-name):

  • This example shows a PI query that times out on the client side. The default timeout of 1 second is used on aql causing the server to fail returning all the records. Instead, the query returns a subset of the 1M records that were on the namespace:
Admin+> asinfo -v 'query-show:trid=15648753051941266254'
mycluster-1:3000 (172.17.0.3) returned:
trid=15648753051941266254:job-type=basic:ns=test:n-pids-requested=2048:rps=0:active-threads=0:status=done(abandoned-response-timeout):job-progress=100.00:run-time=1066:time-since-done=14817072:recs-throttled=0:recs-filtered-meta=0:recs-filtered-bins=0:recs-succeeded=250319:recs-failed=0:net-io-bytes=17826771:net-io-time=856:socket-timeout=30000:from=127.0.0.1+59856

172.17.0.2:3000 (172.17.0.2) returned:
trid=15648753051941266254:job-type=basic:ns=test:n-pids-requested=2048:rps=0:active-threads=0:status=done(abandoned-response-timeout):job-progress=100.00:run-time=1055:time-since-done=14816025:recs-throttled=0:recs-filtered-meta=0:recs-filtered-bins=0:recs-succeeded=276896:recs-failed=0:net-io-bytes=20972559:net-io-time=854:socket-timeout=30000:from=172.17.0.3+55192
  • This example shows an SI query that returns a single record:
Admin+> asinfo -v 'query-show:trid=15021089193528544137'
mycluster-1:3000 (172.17.0.3) returned:
trid=15021089193528544137:job-type=basic:ns=test:set=testset:sindex-name=mysindex:n-pids-requested=2048:rps=0:active-threads=0:status=done(ok):job-progress=100.00:run-time=29:time-since-done=502877:recs-throttled=0:recs-filtered-meta=0:recs-filtered-bins=0:recs-succeeded=0:recs-failed=0:net-io-bytes=30:net-io-time=0:socket-timeout=30000:from=127.0.0.1+60036

172.17.0.2:3000 (172.17.0.2) returned:
trid=15021089193528544137:job-type=basic:ns=test:set=testset:sindex-name=mysindex:n-pids-requested=2048:rps=0:active-threads=0:status=done(ok):job-progress=100.00:run-time=24:time-since-done=502880:recs-throttled=0:recs-filtered-meta=0:recs-filtered-bins=0:recs-succeeded=1:recs-failed=0:net-io-bytes=149:net-io-time=0:socket-timeout=30000:from=172.17.0.3+55372

Abort Queries

Kill a Query Job:

Kill a running query with Tools package 6.2.x or later:

asadm -e 'enable; manage jobs kill trid <jobid>'

Important Statistics to Monitor

Admin> show stat namespace for test like query

Query Histograms

An overall query histogram is written to the log file every 10 seconds. For more details refer histograms page.

Jun 16 2022 17:02:22 GMT: INFO (info): (hist.c:321) histogram dump: {test}-pi-query (1 total) msec
Jun 16 2022 17:02:22 GMT: INFO (info): (hist.c:340) (07: 0000000001)
Jun 16 2022 17:02:22 GMT: INFO (info): (hist.c:321) histogram dump: {test}-pi-query-rec-count (1 total) count
Jun 16 2022 17:02:22 GMT: INFO (info): (hist.c:340) (10: 0000000001)
Jun 16 2022 17:02:22 GMT: INFO (info): (hist.c:321) histogram dump: {test}-si-query (1 total) msec
Jun 16 2022 17:02:22 GMT: INFO (info): (hist.c:340) (02: 0000000001)
Jun 16 2022 17:02:22 GMT: INFO (info): (hist.c:321) histogram dump: {test}-si-query-rec-count (1 total) count
Jun 16 2022 17:02:22 GMT: INFO (info): (hist.c:340) (10: 0000000001)

SI Query Microbenchmarks

SI query microbenchmarks are not supported from Server version 6.0 onwards.

Recommendations

Enable tracking only if you run long-running SI queries or for debugging.

By default, query_in_transaction_thread and query_req_in_query_thread are set to '0'. Set them to '1' when the database is in memory or when the SI query returns fewer records.

asadm – Killing Jobs
asinfo - Reference