Scan

Scan

The scan object created by calling Client#scan is used for executing record scans on the specified namespace and set (optional). Scans can return a set of records as a RecordStream or apply an Aerospike UDF (user-defined function) on each of the records on the server.

Scan is obsolete in server 6.0

Use query methods implemented by Client#query. For more information, please refer to the section on ⇑Historical evolution of scan features in the Aerospike technical documentation.

Selecting Bins

Using Scan#select it is possible to select a subset of bins which should be returned by the query. If no bins are selected, then the whole record will be returned. If the Scan#nobins property is set to true the only the record meta data (ttl, generation, etc.) will be returned.

Executing a Scan

A scan is executed using Scan#foreach. The method returns a RecordStream which emits a data event for each record returned by the scan. The scan can be aborted at any time by calling RecordStream#abort.

Executing Record UDFs using Background Scans

Record UDFs perform operations on a single record such as updating records based on a set of parameters. Using Scan#background you can run a Record UDF on the result set of a scan. Scans using Records UDFs are run in the background on the server and do not return the records to the client.

For additional information please refer to the section on ⇑Record UDFs in the Aerospike technical documentation.

Scan pagination

Scan pagination allows for queries return records in pages rather than all at once. To enable scan pagination, the scan property Scan#paginate must be true and the previously stated scan policy ScanPolicy#maxRecords must be set to a nonzero positive integer in order to specify a maximum page size.

When a page is complete, RecordStream event RecordStream#event:error will emit a Scan#scanState object containing a serialized version of the scan. This serialized scan, if be assigned back to Scan#scanState, allows the scan to retrieve the next page of records in the scan upon calling Scan#foreach. If RecordStream#event:error emits an undefined object, either Scan#paginate is not true, or the scan has successfully returned all the specified records.

For additional information and examples, please refer to the Scan#paginate section below.

Constructor

new Scan(client, ns, set, optionsopt)

Source:
Since:
  • v2.0
Deprecated:
  • since server 6.0
See:
Example
const Aerospike = require('aerospike')

// INSERT HOSTNAME AND PORT NUMBER OF AEROSPIKE SERVER NODE HERE!
var config = {
  hosts: '192.168.33.10:3000',
  // Timeouts disabled, latency dependent on server location. Configure as needed.
  policies: {
    scan : new Aerospike.ScanPolicy({socketTimeout : 0, totalTimeout : 0}),
   }
}

Aerospike.connect(config, (error, client) => {
  if (error) throw error

  const scan = client.scan('test', 'demo')
  let recordsSeen = 0
  const stream = scan.foreach()
  stream.on('error', (error) => { throw error })
  stream.on('end', () => client.close())
  stream.on('data', (record) => {
    console.log(record)
    recordsSeen++
    if (recordsSeen > 100) stream.abort() // We've seen enough!
  })
})
Parameters:
Name Type Attributes Description
client Client

A client instance.

ns string

The namescape.

set string

The name of a set.

options object <optional>

Scan parameters.

Properties
Name Type Attributes Default Description
select Array.<string> <optional>

List of bin names to select. See Scan#select.

nobins boolean <optional>
false

Whether only meta data should be returned. See Scan#nobins.

concurrent boolean <optional>
false

Whether all cluster nodes should be scanned concurrently. See Scan#concurrent.

ttl boolean <optional>
0

The time-to-live (expiration) of the record in seconds. See Scan#ttl.

Members

concurrent :boolean

Description:
  • If set to true, all cluster nodes will be scanned in parallel.

Source:

If set to true, all cluster nodes will be scanned in parallel.

Type:
  • boolean

nobins :boolean

Description:
  • If set to true, the scan will return only meta data, and exclude bins.

Source:

If set to true, the scan will return only meta data, and exclude bins.

Type:
  • boolean

ns :string

Description:
  • Namespace to scan.

Source:

Namespace to scan.

Type:
  • string

paginate :boolean

Description:
  • If set to true, paginated queries are enabled. In order to receive paginated results, the ScanPolicy#maxRecords property must assign a nonzero integer value.

Source:

If set to true, paginated queries are enabled. In order to receive paginated results, the ScanPolicy#maxRecords property must assign a nonzero integer value.

Type:
  • boolean
Examples

Asynchronous pagination over a set of thirty records with Scan#foreach.

const Aerospike = require('./lib/aerospike');
// Define host configuration
let config = {
  hosts: '34.213.88.142:3000',
  policies: {
    batchWrite : new Aerospike.BatchWritePolicy({socketTimeout : 0, totalTimeout : 0}),
  }
};

var batchRecords = []
for(let i = 0; i < 30; i++){
  batchRecords.push({
    type: Aerospike.batchType.BATCH_WRITE,
    key: new Aerospike.Key('test', 'demo', 'key' + i),
    ops:[Aerospike.operations.write('exampleBin', i)]
  })
}

;(async function() {
  try {
    client = await Aerospike.connect(config)
    await client.truncate('test', 'demo', 0)
    await client.batchWrite(batchRecords, {socketTimeout : 0, totalTimeout : 0})

    const scan = client.scan('test', 'demo', {paginate: true})
    do {
      const stream = scan.foreach({maxRecords: 11})
      stream.on('error', (error) => { throw error })
      stream.on('data', (record) => {
        console.log(record.bins)
      })
      await new Promise(resolve => {
        stream.on('end', (scanState) => {
          scan.nextPage(scanState)
          console.log(scan.scanState)
          resolve()
        })
      })
    } while (scan.hasNextPage())

  } catch (error) {
    console.error('An error occurred at some point.', error)
    process.exit(1)
  } finally {
    if (client) client.close()
  }
})()

Asynchronous pagination over a set of thirty records with Scan#foreach.

const Aerospike = require('./lib/aerospike');
// Define host configuration
let config = {
  hosts: '34.213.88.142:3000',
  policies: {
    batchWrite : new Aerospike.BatchWritePolicy({socketTimeout : 0, totalTimeout : 0}),
  }
};

var batchRecords = []
for(let i = 0; i < 30; i++){
  batchRecords.push({
    type: Aerospike.batchType.BATCH_WRITE,
    key: new Aerospike.Key('test', 'demo', 'key' + i),
    ops:[Aerospike.operations.write('exampleBin', i)]
  })
}

;(async function() {
  try {
    client = await Aerospike.connect(config)
    await client.truncate('test', 'demo', 0)
    await client.batchWrite(batchRecords, {socketTimeout : 0, totalTimeout : 0})

    const scan = client.scan('test', 'demo', {paginate: true})
    let allResults = []
    let results = await scan.results({maxRecords: 11})
    allResults = [...allResults, ...results]


    results = await scan.results({maxRecords: 11})
    allResults = [...allResults, ...results]

    results = await scan.results({maxRecords: 11})
    allResults = [...allResults, ...results]

    console.log("Records returned in total: " + allResults.length)  // Should be 30 records

  } catch (error) {
    console.error('An error occurred at some point.', error)
    process.exit(1)
  } finally {
    if (client) client.close()
  }
})()

scanState :number

Description:
  • If set to a valid serialized scan, calling scan#foreach will allow the next page of records to be queried while preserving the progress of the previous scan. If set to null, calling scan#foreach will begin a new scan.

Source:

If set to a valid serialized scan, calling scan#foreach will allow the next page of records to be queried while preserving the progress of the previous scan. If set to null, calling scan#foreach will begin a new scan.

Type:
  • number

selected :Array.<string>

Description:
  • List of bin names to be selected by the scan. If a scan specifies bins to be selected, then only those bins will be returned. If no bins are selected, then all bins will be returned (unless Scan#nobins is set to true).

Source:
See:

List of bin names to be selected by the scan. If a scan specifies bins to be selected, then only those bins will be returned. If no bins are selected, then all bins will be returned (unless Scan#nobins is set to true).

Type:
  • Array.<string>

set :string

Description:
  • Name of the set to scan.

Source:

Name of the set to scan.

Type:
  • string

ttl :number

Description:
  • The time-to-live (expiration) of the record in seconds. There are also special values that can be set in the record TTL:

    0 (defined Aerospike.ttl.NAMESPACE_DEFAULT), which means that the record will adopt the default TTL value from the namespace.

    -1 (defined Aerospike.ttl.NEVER_EXIRE), which means that the record will get an internal "void_time" of zero, and thus will never expire.

    -2 (defined Aerospike.ttl.DONT_UPDATE), which means that the record ttl will not change when the record is updated.

    Note that the TTL value will be employed ONLY on background scan writes.

Source:

The time-to-live (expiration) of the record in seconds. There are also special values that can be set in the record TTL:

0 (defined Aerospike.ttl.NAMESPACE_DEFAULT), which means that the record will adopt the default TTL value from the namespace.

-1 (defined Aerospike.ttl.NEVER_EXIRE), which means that the record will get an internal "void_time" of zero, and thus will never expire.

-2 (defined Aerospike.ttl.DONT_UPDATE), which means that the record ttl will not change when the record is updated.

Note that the TTL value will be employed ONLY on background scan writes.

Type:
  • number

Methods

background(udfModule, udfFunction, udfArgsopt, policyopt, scanIDopt, callbackopt) → (nullable) {Promise}

Perform a read-write background scan and apply a Lua user-defined function (UDF) to each record.

Description:
  • When a background scan is initiated, the client will not wait for results from the database. Instead a Job instance will be returned, which can be used to query the scan status on the database.

Source:
Parameters:
Name Type Attributes Description
udfModule string

UDF module name.

udfFunction string

UDF function name.

udfArgs Array.<*> <optional>

Arguments for the function.

policy ScanPolicy <optional>

The Scan Policy to use for this operation.

scanID number <optional>

Job ID to use for the scan; will be assigned randomly if zero or undefined.

callback jobCallback <optional>

The function to call when the operation completes.

Returns:

If no callback function is passed, the function returns a Promise that resolves to a Job instance.

Type
Promise

foreach(policyopt, dataCbopt, errorCbopt, endCbopt) → {RecordStream}

Performs a read-only scan on each node in the cluster. As the scan iterates through each partition, it returns the current version of each record to the client.

Source:
Parameters:
Name Type Attributes Description
policy ScanPolicy <optional>

The Scan Policy to use for this operation.

dataCb recordCallback <optional>

The function to call when the operation completes with the results of the operation; if no callback function is provided, the method returns a Promise instead.

errorCb errorCallback <optional>

Callback function called when there is an error.

endCb doneCallback <optional>

Callback function called when an operation has completed.

Returns:
Type
RecordStream

hasNextPage() → {boolean}

Checks compiliation status of a paginated scan.

Description:
  • If false is returned, there are no more records left in the scan, and the scan is complete. If true is returned, calling Scan#foreach will continue from the state specified by Scan#scanState.

Source:
Returns:
Type
boolean

nextPage(…state)

Sets scan#scanState to the value specified by the state argument.

Description:
Source:
Parameters:
Name Type Attributes Description
state object <repeatable>

serialized scan emitted from the RecordStream#event:error event.

operate(operations, policyopt, scanIDopt, callbackopt) → (nullable) {Promise}

Applies write operations to all matching records.

Description:
  • Performs a background scan and applies one or more write operations to all records. Neither the records nor the results of the operations are returned to the client. Instead a Job instance will be returned, which can be used to query the scan status.

    This method requires server >= 3.7.0.

Source:
Since:
  • v3.14.0
Example

Increment count bin on all records in set using a background scan

const Aerospike = require('aerospike')

// INSERT HOSTNAME AND PORT NUMBER OF AEROSPIKE SERVER NODE HERE!
var config = {
  hosts: '192.168.33.10:3000',
  // Timeouts disabled, latency dependent on server location. Configure as needed.
  policies: {
    scan : new Aerospike.ScanPolicy({socketTimeout : 0, totalTimeout : 0})
   }
}

Aerospike.connect(config).then(async (client) => {
  const scan = client.scan('namespace', 'set')
  const ops = [Aerospike.operations.incr('count', 1)]
  const job = await scan.operate(ops)
  await job.waitUntilDone()
  client.close()
})
Parameters:
Name Type Attributes Description
operations Array.<module:aerospike/operations~Operation>

List of write operations to perform on the matching records.

policy ScanPolicy <optional>

The Scan Policy to use for this operation.

scanID number <optional>

Job ID to use for the scan; will be assigned randomly if zero or undefined.

callback jobCallback <optional>

The function to call when the operation completes.

Returns:

If no callback function is passed, the function returns a Promise that resolves to a Job instance.

Type
Promise

partitions(begin, count, digest)

Specify the begin and count of the partitions to be scanned by the scan foreach op.

Description:
  • If a scan specifies partitions begin and count, then only those partitons will be scanned and returned. If no partitions are specified, then all partitions will be scanned and returned.

Source:
Parameters:
Name Type Description
begin number

Start partition number to scan.

count number

Number of partitions from the start to scan.

digest string

Start from this digest if it is specified.

results(policyopt) → {Promise.<Array.<RecordObject>>}

Executes the Scan and collects the results into an array. On paginated queries, preparing the next page is also handled automatically.

Description:
  • This method returns a Promise that contains the scan results as an array of records, when fulfilled. It should only be used if the scan is expected to return only few records; otherwise it is recommended to use Scan#foreach, which returns the results as a RecordStream instead.

    If pagination is enabled, the data emitted from the RecordStream#event:error event will automatically be assigned to Scan#scanState, allowing the next page of records to be queried if Scan#foreach or Scan#results is called.

Source:
Parameters:
Name Type Attributes Description
policy ScanPolicy <optional>

The Scan Policy to use for this operation.

Returns:
Type
Promise.<Array.<RecordObject>>

select(…bins)

Specify the names of bins to be selected by the scan.

Description:
  • If a scan specifies bins to be selected, then only those bins will be returned. If no bins are selected, then all bins will be returned. (Unless Scan#nobins is set to true.)

Source:
Parameters:
Name Type Attributes Description
bins string <repeatable>

List of bin names to return.