Skip to main content
Loading

Managing Sets in a Namespace

An Aerospike set is similar to a table in a relational database, except you do not need to define a schema for a set. Sets are created dynamically and implicitly on the first record insertion in set. For more see Data Model.

Protecting a set from evictions​

A set can be protected from evictions with a dynamic configuration. If evictions are in effect for a namespace, sets with evictions disabled will be skipped, regardless of the void-time of the records.

The asadm command to dynamically disable set evictions is:

asadm -e "enable; manage config namespace test set demo param disable-eviction to true"

Or using the older info command:

asadm -e "enable; asinfo -v 'set-config:context=namespace;id=test;set=demo;disable-eviction=true' "
note

Configuration parameters are applied per-node. asadm sends the same info command disabling evictions to all the nodes in the cluster.

Capping the size of a set​

Starting with Aerospike 6.3, you can use the stop-writes-size configuration parameter to specify a size limit for a set. If this threshold is breached, client writes to the set are refused. Deletions, replica writes, and migration writes are still allowed when the set is in stop-writes mode.

Once the limit is reached, the server does not allow any additional writes, even those that would decrease the size of a record. There are two ways to get under the stop-writes-size limit: increase or remove the limit, or delete records. However, record deletions via UDF, delete-all-bins ops, or background ops queries will also be rejected – only regular record deletes (including batch deletes) and nsup deletes will be allowed.

The asadm command to dynamically cap the size of a set is:

asadm -e "enable; manage config namespace test set demo param stop-writes-size to 250M"

Or using the older info command:

asadm -e "enable; asinfo -v 'set-config:context=namespace;id=test;set=demo;stop-writes-size=250M' "
note

Configuration parameters are applied per-node. asadm sends the same info command limiting the set size to all the nodes in the cluster. The cluster-wide size quota for the set is number-of-cluster-nodes x stop-writes-size.

The threshold is checked and stop-writes triggered independently on each node.

Capping the number of records in a set​

A set can be capped to a maximum number of records. If this threshold is breached, client writes to the set are refused. Deletions, replica writes, and migration writes are still allowed when the set is in stop-writes mode.

The asadm command to dynamically cap the number of records in a set is:

asadm -e "enable; manage config namespace test set demo param stop-writes-count to 1000000"

Or using the older info command:

asadm -e "enable; asinfo -v 'set-config:context=namespace;id=test;set=demo;stop-writes-count=1000000' "
note

Configuration parameters are applied per-node. asadm sends the same info command limiting the number of records in the set to all the nodes in the cluster. The cluster-wide record-count quota for the set is number-of-cluster-nodes x stop-writes-count.

The threshold is checked and stop-writes triggered independently on each node.

Adding and Removing a Set Index​

Introduced in Aerospike Database 5.6, a set index can be dynamically configured in the set context through declaring enable-index to be true. This may significantly improve performance when scanning the records in the set.

asadm -e "enable; manage config namespace test set demo param enable-index to true"

Or using the older info command:

asadm -e "enable; asinfo -v 'set-config:context=namespace;id=test;set=demo;enable-index=true' "

Removing the set index is simply a matter of changing the value to false, which is also the default for each set.

asadm -e "enable; manage config namespace test set demo param enable-index to false"

Or using the older info command:

asadm -e "enable; asinfo -v 'set-config:context=namespace;id=test;set=demo;enable-index=false' "

Monitoring Set Indexes​

The memory used for all set indexes in a namespace is available via the memory_used_set_index_bytes namespace statistic and also via the "memory-usage" server log ticker line.

Clearing a Set​

There is a limit of 1023 on the total number of sets in a namespace. Deleting a set metadata (to reclaim space from this 1023 limit) requires durably deleting all the data in the set and cold restarting the node. See the following How to clear up set names when they exceed the limit for details.

Truncating a Set in a Namespace​

caution

Server versions 4.3.1.11, 4.4.0.11, 4.5.0.6 and 4.5.1.5 added 2 different commands for truncation:

  • truncate for truncating sets.
  • truncate-namespace for truncating namespaces.

    It is recommended to use the asadm manage truncate command due to the clear separation between truncating a set and truncating a namespace (added in Tools package 6.0).

To initiate a set truncation, the asadm, asinfo, or aql tools can be used to issue a truncate command to one Aerospike cluster node at a time.

To truncate a set, use either of the following commands.

Using the asadm command-line interface:

asadm
Seed:        [('127.0.0.1', 3000, None)]
Aerospike Interactive Shell, version 2.11.0

Found 1 nodes
Online:  127.0.0.1:3000

Admin> info set
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Set Information (2023-03-26 22:59:40 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~
Namespace| Set|           Node|  Memory|    Disk|~~~~~Quota~~~~~~| Objects|   Stop| Disable|  Set
         |    |               |    Used|    Used|     Total|Used%|        | Writes|Eviction|Index
         |    |               |        |        |          |     |        |  Count|        |     
test     |demo|127.0.0.1:3000 |1.236 MB|0.000 B |250.000 MB|0.0 %|99.711 K|1000000|True    |No   
test     |demo|               |1.236 MB|0.000 B |250.000 MB|0.0 %|99.711 K|       |        |     
Number of rows: 1

Admin> enable
Admin+> manage truncate ns test set demo
You're about to truncate up to 0 records from set demo for namespace test
Confirm that you want to proceed by typing 6cdc8c, or cancel by typing anything else.
6cdc8c
Successfully started truncation for set demo of namespace test
Admin+> info set
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Set Information (2023-03-26 23:04:08 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~
Namespace| Set|           Node|  Memory|    Disk|~~~~~Quota~~~~~~|Objects|   Stop| Disable|  Set
         |    |               |    Used|    Used|     Total|Used%|       | Writes|Eviction|Index
         |    |               |        |        |          |     |       |  Count|        |     
test     |demo|127.0.0.1:3000 |0.000 B |0.000 B |250.000 MB|0.0 %|0.000  |1000000|True    |No   
test     |demo|               |0.000 B |0.000 B |250.000 MB|0.0 %|0.000  |       |        |     
Number of rows: 1

Or in non-interactive mode:

asadm --enable -e "manage truncate ns test set demo --no-warn"

Or with asinfo send truncate a single node. The truncate operation is then communicated to the other nodes as shared metadata (SMD):

asinfo -h 127.0.0.1 -v "truncate:namespace=test;set=demo"

Or with aql:

aql -c "TRUNCATE <namespace_name>.<set_name>"

Mechanism​

The mechanism involved in truncating a namespace or set within a namespace is distinct from NSUP.

Before server 6.3, the truncate info command started a single background thread on each of the nodes in the cluster. From server 6.3, the truncate info command starts a number of threads for each truncation job equal to the value of the truncate-threads configuraiton parameter. The truncate threads then traverse the primary index of the namespace and remove the necessary records.

From server 6.3, set truncation is optimized to use set indexes to rapidly find all the records belonging to the set being truncated. This optimization significantly increases the speed of set truncation, with the caveat that it is skipped if the set has tombstones.

The truncate command takes an optional lut (last-update-time) threshold parameter, which limits the truncation to records with an LUT earlier than the specified timestamp . If no timestamp is specified, the now timestamp of the receiving node is used.

Set truncation can be issued repeatedly. It updates the effective last-update-time of affected set, and is reflected in the lut field of the set information.

For truncation on sets with secondary indexes, each record data is actively read, and any secondary index entries associated with the record are also removed.

info

Read the truncate info command for advise on optimally truncating in the presence of secondary indexes.

In the Enterprise Edition, truncation is durable and preserves record deletions through a cold-restart. In the Community Edition, similar to record deletes, records in previously truncated sets are not durable and deletes can return through a cold-start.

note

The truncate-undo and truncate-namespace-undo commands can be used to remove the truncate related entry(ies) in SMD to allow for potential recovery of inadvertently truncated records upon a subsequent Cold Restart. Truncated records that would have been already overwritten on persistent storage would not be recoverable.

Last updated on