Flat JSON Format

Flat JSON Serialization Format

This document specifies the Flat JSON data serialization format used by Aerospike Connect. It describes how Aerospike notification messages are represented as Flat JSON objects. Each message represents an event that occurred in an Aerospike database cluster. Different kinds of messages are used to describe events such as record creation/update and record deletions.

Key Format

The record key has the following properties

Field	Type	Description
namespace	string	Namespace of the Aerospike record.
set	string	Set of the Aerospike record. Maybe missing in some cases.
userKey	number, bytes (Base64 encoded) or string	The user key of the Aerospike record. Is available if user key is stored on the Aerospike server.
digest	bytes (Base64 encoded)	The digest of the Aerospike record.

Example Key

{
    "namespace": "users",
    "set": "premium",
    "userKey": "id123",
    "digest": "IBlTW5m3UGqbFxfrsoDxCrLkKYQ="
}

Message Format

Each message is a JSON object with the metadata included in the specifed metadata-key in the config and all the bins are present as top level objects.

Message metadata

The message metadata properties are:

Metadata	Type	Description
msg	string	Write/Delete operation.
namespace	string	Namespace of the Aerospike record.
set	string	Set of the Aerospike record.
userKey	long, double, bytes or string	The user key of the Aerospike record. Present only if it is a write operation and the user key is stored on the Aerospike server.
digest	bytes	The digest of the Aerospike record.
gen	int	The generation of the Aerospike record.
lut	long	Time when the record was last updated, in milliseconds since the Unix epoch. It is available whenever the Aerospike server ships last-update time. [1][2]
exp	int	Time when the record will expire, in seconds since the Unix epoch. Zero means the record will not expire. Present only in write operation.
durable	boolean	Whether the delete is durable. Present only in delete operation.

note

All metadata is affected by both delete and write operations, except where the description indicates otherwise.

[1] When the Aerospike server does not ship lut, then the following versions of these outbound connectors ship lut as zero:

• JMS outbound connector, versions earlier than 3.0.0
• Kafka outbound connector, versions earlier than 4.0.0
• Pulsar outbound connector, versions earlier than 2.0.0

[2] Breaking Change When the Aerospike server ships lut, then the following versions of these outbound connectors ship lut as a value of the data type "integer":

• JMS outbound connector, versions earlier than 3.0.0
• Kafka outbound connector, versions earlier than 4.0.0
• Pulsar outbound connector, versions earlier than 2.0.0

Example Write Message

Record with bins color and size.

{
    "metadata": {
        "msg": "write",
        "namespace": "users",
        "set": "premium",
        "userKey": 7612,
        "gen": 4,
        "lut": 1617167159548,
        "digest": "IBlTW5m3UGqbFxfrsoDxCrLkKYQ=",
        "exp": 1682797792
    },
    "color": "red",
    "size": 123
}

Example Delete Message

{
    "metadata": {
        "msg": "delete",
        "namespace": "users",
        "digest": "IBlTW5m3UGqbFxfrsoDxCrLkKYQ=",
        "gen": 4,
        "lut": 1617167159548,
        "durable": false
    }
}

Batch Flat JSON Serialization Format

If batching is configured then the records in the batch will be a JSON array of Flat JSON formatted records.

Example batch of Flat JSON formatted records.

[
  {
    "metadata": {
      "msg": "write",
      "namespace": "users",
      "set": "premium",
      "userKey": 7612,
      "gen": 4,
      "lut": 1617167159548,
      "digest": "IBlTW5m3UGqbFxfrsoDxCrLkKYQ=",
      "exp": 1682797792
    },
    "color": "red",
    "size": 123
  },
  {
    "metadata": {
      "msg": "delete",
      "namespace": "users",
      "digest": "IBlTW5m3UGqbFxfrsoDxCrLkKYQ=",
      "gen": 4,
      "lut": 1617167159548,
      "durable": false
    }
  }
]

If the keys of a batch are configured to be concatenated, then the payload key of the batch will be a JSON array of Flat JSON formatted keys.

Example batch of Flat JSON formatted keys

[
  {
    "namespace": "users",
    "set": "premium",
    "userKey": "id123",
    "digest": "IBlTW5m3UGqbFxfrsoDxCrLkKYQ="
  },
  {
    "namespace": "users",
    "set": "premium",
    "userKey": "id124",
    "digest": "JQlTW5m3UGqbFxfxsoDxCrLkKYQ="
  }
]