Query

Query

Aerospike Query operations perform value-based searches using secondary indexes (SI). A Query object, created by calling Client#query, is used to execute queries on the specified namespace and set (optional). Queries can return a set of records as a RecordStream or be processed using Aeorspike User-Defined Functions (UDFs) before returning to the client.

For more information, please refer to the section on ⇑Queries in the Aerospike technical documentation.

To scan all records in a database namespace or set, it is more efficient to use operations, which provide more fine-grained control over execution priority, concurrency, etc.

SI Filters

With a SI, the following queries can be made:

See module:aerospike/filter for a list of all supported secondary index filter.

Before a secondary index filter can be applied, a SI needs to be created on the bins which the index filter matches on. Using the Node.js client, a SI can be created using Client#createIndex.

Currently, only a single SI index filter is supported for each query. To do more advanced filtering, a expressions can be applied to the query using policy (see below). Alternatively, User-Defined Functions (UDFs) can be used to further process the query results on the server.

Previously, predicate filtering was used to perform secondary index queries. SI filter predicates have been deprecated since server 5.2, and obsolete since server 6.0.

For more information about Predicate Filtering, please refer to the ⇑Predicate Filtering documentation in the Aerospike Feature Guide.

Selecting Bins

Using Query#select it is possible to select a subset of bins which should be returned by the query. If no bins are selected, then the whole record will be returned. If the Query#nobins property is set to true the only the record meta data (ttl, generation, etc.) will be returned.

Executing a Query

A query is executed using Query#foreach. The method returns a RecordStream which emits a data event for each record returned by the query. The query can be aborted at any time by calling RecordStream#abort.

Applying User-Defined Functions

User-defined functions (UDFs) can be used to filter, transform, and aggregate query results. Stream UDFs can process a stream of data by defining a sequence of operations to perform. Stream UDFs perform read-only operations on a collection of records. Use Query#setUdf to set the UDF parameters (module name, function name and optional list of arguments) before executing the query using Query#foreach.

The feature guides on ⇑User-Defined Functions and ⇑Stream UDFs contain more detailed information and examples.

Query Aggregation using Stream UDFs

Use Aerospike Stream UDFs to aggregate query results using Query#apply. Aggregation queries work similar to a MapReduce system and return a single result value instead of stream of records. Aggregation results can be basic data types (string, number, byte array) or collection types (list, map).

Please refer to the technical documentation on ⇑Aggregation for more information.

Executing Record UDFs using Background Queries

Record UDFs perform operations on a single record such as updating records based on a set of parameters. Using Query#background you can run a Record UDF on the result set of a query. Queries using Records UDFs are run in the background on the server and do not return the records to the client.

For additional information please refer to the section on ⇑Record UDFs in the Aerospike technical documentation.

Query pagination

Query pagination allows for queries return records in pages rather than all at once. To enable query pagination, the query property Query#paginate must be true and the previously stated query property Query#maxRecords must be set to a nonzero positive integer in order to specify a maximum page size.

When a page is complete, RecordStream event RecordStream#event:error will emit a Query#queryState object containing a serialized version of the query. This serialized query, if be assigned back to Query#queryState, allows the query to retrieve the next page of records in the query upon calling Query#foreach. If Query#queryState is undefined, pagination is not enabled or the query has completed. If RecordStream#event:error emits an undefined object, either Query#paginate is not true, or the query has successfully returned all the specified records.

For additional information and examples, please refer to the Query#paginate section below.

Constructor

new Query(client, ns, set, optionsopt)

Source:
See:
Example
const Aerospike = require('aerospike')
const namespace = 'test'
const set = 'demo'

Aerospike.connect((error, client) => {
  if (error) throw error
  var index = {
    ns: namespace,
    set: set,
    bin: 'tags',
    index: 'tags_idx',
    type: Aerospike.indexType.LIST,
    datatype: Aerospike.indexDataType.STRING
  }
  client.createIndex(index, (error, job) => {
    if (error) throw error
    job.waitUntilDone((error) => {
      if (error) throw error

      var query = client.query('test', 'demo')
      const queryPolicy = { filterExpression: exp.keyExist('uniqueExpKey') }
      query.select('id', 'tags')
      query.where(Aerospike.filter.contains('tags', 'green', Aerospike.indexType.LIST))
      var stream = query.foreach(queryPolicy)
      stream.on('error', (error) => {
        console.error(error)
        throw error
      })
      stream.on('data', (record) => {
        console.info(record)
      })
      stream.on('end', () => {
        client.close()
      })
    })
  })
})
Parameters:
Name Type Attributes Description
client Client

A client instance.

ns string

The namescape.

set string

The name of a set.

options object <optional>

Query parameters.

Properties
Name Type Attributes Default Description
filters Array.<object> <optional>

List of index filters to apply to the query. See Query#where.

select Array.<string> <optional>

List of bin names to select. See Query#select.

nobins boolean <optional>
false

Whether only meta data should be returned. See Query#nobins.

ttl boolean <optional>
0

The time-to-live (expiration) of the record in seconds. See Query#ttl.

Members

filters :Array.<object>

Description:
  • Filters to apply to the query.

    Note: Currently, a single index filter is supported. To do more advanced filtering, you need to use a user-defined function (UDF) to process the result set on the server.

Source:
See:

Filters to apply to the query.

Note: Currently, a single index filter is supported. To do more advanced filtering, you need to use a user-defined function (UDF) to process the result set on the server.

Type:
  • Array.<object>

maxRecords :number

Description:
  • Approximate number of records to return to client.

    When query#paginate is true, then maxRecords will be the page size if there are enough records remaining in the query to fill the page size.

    When query#paginate is false, this number is divided by the number of nodes involved in the scan, and actual number of records returned may be less than maxRecords if node record counts are small and unbalanced across nodes.

Source:

Approximate number of records to return to client.

When query#paginate is true, then maxRecords will be the page size if there are enough records remaining in the query to fill the page size.

When query#paginate is false, this number is divided by the number of nodes involved in the scan, and actual number of records returned may be less than maxRecords if node record counts are small and unbalanced across nodes.

Type:
  • number

nobins :boolean

Description:
  • If set to true, the query will return only meta data, and exclude bins.

Source:

If set to true, the query will return only meta data, and exclude bins.

Type:
  • boolean

ns :string

Description:
  • Namespace to query.

Source:

Namespace to query.

Type:
  • string

paginate :boolean

Description:
  • If set to true, paginated queries are enabled. In order to receive paginated results, the query#maxRecords property must assign a nonzero integer value.

Source:

If set to true, paginated queries are enabled. In order to receive paginated results, the query#maxRecords property must assign a nonzero integer value.

Type:
  • boolean
Examples

Asynchronous pagination over a set of thirty records with Query#foreach.

const Aerospike = require('./lib/aerospike');
// Define host configuration
let config = {
  hosts: '34.213.88.142:3000',
  policies: {
    batchWrite : new Aerospike.BatchWritePolicy({socketTimeout : 0, totalTimeout : 0}),
  }
};

var batchRecords = []
for(let i = 0; i < 30; i++){
  batchRecords.push({
    type: Aerospike.batchType;.BATCH_WRITE,
    key: new Aerospike.Key('test', 'demo', 'key' + i),
    ops:[Aerospike.operations.write('exampleBin', i)]
  })
}

;(async function() {
  try {
    client = await Aerospike.connect(config)
    await client.truncate('test', 'demo', 0)
    await client.batchWrite(batchRecords, {socketTimeout : 0, totalTimeout : 0})

    const query = client.query('test', 'demo', { paginate: true, maxRecords: 10})
    do {
      const stream = query.foreach()
      stream.on('error', (error) => { throw error })
      stream.on('data', (record) => {
        console.log(record.bins)
      })
      await new Promise(resolve => {
        stream.on('end', (queryState) => {
          query.queryState = queryState
          resolve()
        })
      })
    } while (query.queryState !== undefined)

  } catch (error) {
    console.error('An error occurred at some point.', error)
    process.exit(1)
  } finally {
    if (client) client.close()
  }
})()

Asynchronous pagination over a set of thirty records with Query#results

const Aerospike = require('./lib/aerospike');
// Define host configuration
let config = {
  hosts: '34.213.88.142:3000',
  policies: {
    batchWrite : new Aerospike.BatchWritePolicy({socketTimeout : 0, totalTimeout : 0}),
  }
};

var batchRecords = []
for(let i = 0; i < 30; i++){
  batchRecords.push({
    type: Aerospike.batchType.BATCH_WRITE,
    key: new Aerospike.Key('test', 'demo', 'key' + i),
    ops:[Aerospike.operations.write('exampleBin', i)]
  })
}


;(async function() {
  try {
    client = await Aerospike.connect(config)
    await client.truncate('test', 'demo', 0)
    await client.batchWrite(batchRecords, {socketTimeout : 0, totalTimeout : 0})

    const query = client.query('test', 'demo', { paginate: true, maxRecords: 11})

    let allResults = []
    let results = await query.results()
    allResults = [...allResults, ...results]


    results = await query.results()
    allResults = [...allResults, ...results]

    results = await query.results()
    allResults = [...allResults, ...results]

    console.log("Records returned in total: " + allResults.length)  // Should be 30 records
  } catch (error) {
    console.error('An error occurred at some point.', error)
    process.exit(1)
  } finally {
    if (client) client.close()
  }
})()

queryState :Object

Description:
  • If set to a valid serialized query, calling query#foreach will allow the next page of records to be queried while preserving the progress of the previous query. If set to null, calling query#foreach will begin a new query.

Source:

If set to a valid serialized query, calling query#foreach will allow the next page of records to be queried while preserving the progress of the previous query. If set to null, calling query#foreach will begin a new query.

Type:
  • Object

selected :Array.<string>

Description:
  • List of bin names to be selected by the query. If a query specifies bins to be selected, then only those bins will be returned. If no bins are selected, then all bins will be returned (unless Query#nobins is set to true).

Source:
See:

List of bin names to be selected by the query. If a query specifies bins to be selected, then only those bins will be returned. If no bins are selected, then all bins will be returned (unless Query#nobins is set to true).

Type:
  • Array.<string>

set :string

Description:
  • Name of the set to query.

Source:

Name of the set to query.

Type:
  • string

ttl :number

Description:
  • The time-to-live (expiration) of the record in seconds. There are also special values that can be set in the record TTL:

    0 (defined Aerospike.ttl.NAMESPACE_DEFAULT), which means that the record will adopt the default TTL value from the namespace.

    -1 (defined Aerospike.ttl.NEVER_EXIRE), which means that the record will get an internal "void_time" of zero, and thus will never expire.

    -2 (defined Aerospike.ttl.DONT_UPDATE), which means that the record ttl will not change when the record is updated.

    Note that the TTL value will be employed ONLY on background query writes.

Source:

The time-to-live (expiration) of the record in seconds. There are also special values that can be set in the record TTL:

0 (defined Aerospike.ttl.NAMESPACE_DEFAULT), which means that the record will adopt the default TTL value from the namespace.

-1 (defined Aerospike.ttl.NEVER_EXIRE), which means that the record will get an internal "void_time" of zero, and thus will never expire.

-2 (defined Aerospike.ttl.DONT_UPDATE), which means that the record ttl will not change when the record is updated.

Note that the TTL value will be employed ONLY on background query writes.

Type:
  • number

udf :Object

Description:
  • User-defined function parameters to be applied to the query executed using Query#foreach.

Source:

User-defined function parameters to be applied to the query executed using Query#foreach.

Type:
  • Object

Methods

apply(udfModule, udfFunction, udfArgsopt, policyopt, callbackopt) → (nullable) {Promise}

Applies a user-defined function (UDF) to aggregate the query results.

Description:
  • The aggregation function is called on both server and client (final reduce). Therefore, the Lua script files must also reside on both server and client.

Source:
Parameters:
Name Type Attributes Description
udfModule string

UDF module name.

udfFunction string

UDF function name.

udfArgs Array.<*> <optional>

Arguments for the function.

policy QueryPolicy <optional>

The Query Policy to use for this operation.

callback QueryaggregationResultCallback <optional>

The function to call when the operation completes.

Returns:

If no callback function is passed, the function returns a Promise that resolves to the aggregation results.

Type
Promise

background(udfModule, udfFunction, udfArgsopt, policyopt, queryIDopt, callbackopt) → (nullable) {Promise}

Applies a user-defined function (UDF) on records that match the query filter. Records are not returned to the client.

Description:
  • When a background query is initiated, the client will not wait for results from the database. Instead a Job instance will be returned, which can be used to query the query status on the database.

Source:
Parameters:
Name Type Attributes Description
udfModule string

UDF module name.

udfFunction string

UDF function name.

udfArgs Array.<*> <optional>

Arguments for the function.

policy WritePolicy <optional>

The Write Policy to use for this operation.

queryID number <optional>

Job ID to use for the query; will be assigned randomly if zero or undefined.

callback jobCallback <optional>

The function to call when the operation completes.

Returns:

If no callback function is passed, the function returns a Promise that resolves to a Job instance.

Type
Promise

foreach(policyopt, dataCbopt, errorCbopt, endCbopt) → {RecordStream}

Asynchronously executes the query and returns each result item through the stream.

Description:
  • Applying a Stream UDF to the query results

    A stream UDF can be applied to the query to filter, transform and aggregate the query results. The UDF parameters need to be set on the query object using Query#setUdf before the query is executed.

    If a UDF is applied to the query, the resulting stream will return the results of the UDF stream function. Record meta data and the record keys will not be returned.

    For aggregation queries that return a single result value instead of a stream of values, you should use the Query#apply method instead.

Source:
Parameters:
Name Type Attributes Description
policy QueryPolicy <optional>

The Query Policy to use for this operation.

dataCb recordCallback <optional>

The function to call when the operation completes with the results of the operation; if no callback function is provided, the method returns a Promise instead.

errorCb errorCallback <optional>

Callback function called when there is an error.

endCb doneCallback <optional>

Callback function called when an operation has completed.

Returns:
Type
RecordStream

hasNextPage() → {boolean}

Checks compiliation status of a paginated query.

Description:
  • If false is returned, there are no more records left in the query, and the query is complete. If true is returned, calling Query#foreach will continue from the state specified by Query#queryState.

Source:
Returns:
Type
boolean

nextPage(…state)

Sets query#queryState to the value specified by the state argument.

Description:
Source:
Parameters:
Name Type Attributes Description
state object <repeatable>

serialized query emitted from the RecordStream#event:error event.

operate(operations, policyopt, queryIDopt, callbackopt) → (nullable) {Promise}

Applies write operations to all matching records.

Description:
  • Performs a background query and applies one or more write operations to all records that match the query filter(s). Neither the records nor the results of the operations are returned to the client. Instead a Job instance will be returned, which can be used to query the query status.

    This method requires server >= 3.7.0.

Source:
Since:
  • v3.14.0
Example

Increment count bin on all matching records using a background query

const Aerospike = require('aerospike')

Aerospike.connect().then(async (client) => {
  const query = client.query('namespace', 'set')
  query.where(Aerospike.filter.range('age', 18, 42))
  const ops = [Aerospike.operations.incr('count', 1)]
  const job = await query.operate(ops)
  await job.waitUntilDone()
  client.close()
})
Parameters:
Name Type Attributes Description
operations Array.<module:aerospike/operations~Operation>

List of write operations to perform on the matching records.

policy QueryPolicy <optional>

The Query Policy to use for this operation.

queryID number <optional>

Job ID to use for the query; will be assigned randomly if zero or undefined.

callback jobCallback <optional>

The function to call when the operation completes.

Returns:

If no callback function is passed, the function returns a Promise that resolves to a Job instance.

Type
Promise

partitions(begin, count, digest)

Specify the begin and count of the partitions to be queried by the Query foreach op.

Description:
  • If a Query specifies partitions begin and count, then only those partitons will be queried and returned. If no partitions are specified, then all partitions will be queried and returned.

Source:
Parameters:
Name Type Description
begin number

Start partition number to query.

count number

Number of partitions from the start to query.

digest string

Start from this digest if it is specified.

results(policyopt) → {Promise.<Array.<RecordObject>>}

Executes the query and collects the results into an array. On paginated queries, preparing the next page is also handled automatically.

Description:
  • This method returns a Promise that contains the query results as an array of records, when fulfilled. It should only be used if the query is expected to return only few records; otherwise it is recommended to use Query#foreach, which returns the results as a RecordStream instead.

    If pagination is enabled, the data emitted from the RecordStream#event:error event will automatically be assigned to Query#queryState, allowing the next page of records to be queried if Query#foreach or Query#results is called.

Source:
Parameters:
Name Type Attributes Description
policy QueryPolicy <optional>

The Query Policy to use for this operation.

Returns:
Type
Promise.<Array.<RecordObject>>

select(…bins)

Specify the names of bins to be selected by the query.

Description:
  • If a query specifies bins to be selected, then only those bins will be returned. If no bins are selected, then all bins will be returned. (Unless Query#nobins is set to true.)

Source:
Parameters:
Name Type Attributes Description
bins string <repeatable>

List of bin names to return.

setUdf(udfModule, udfFunction, udfArgsopt)

Set user-defined function parameters to be applied to the query.

Source:
Parameters:
Name Type Attributes Description
udfModule string

UDF module name.

udfFunction string

UDF function name.

udfArgs Array.<*> <optional>

Arguments for the function.

where(index)

Applies a SI to the query.

Description:
  • Use a SI to limit the results returned by the query. This method takes SI created using the filter module as argument.

Source:
See:
Example

Applying a SI filter to find all records where the 'tags' list bin contains the value 'blue':

const Aerospike = require('aerospike')

Aerospike.connect().then(client => {
  let query = client.query('test', 'demo')

  let tagsFilter = Aerospike.filter.contains('tags', 'blue', Aerospike.indexType.LIST)
  query.where(tagsFilter)

  let stream = query.foreach()
  stream.on('data', record => { console.info(record.bins.tags) })
  stream.on('error', error => { throw error })
  stream.on('end', () => client.close())
})
Parameters:
Name Type Description
index module:aerospike/filter~SindexFilterPredicate

filter - The index filter to apply to the function.