Configuring Log Streams

This document shows how to stream log messages by context into separate log sinks.

note

The destination for a log stream is called a log sink. Some examples of log sinks are files on the server, and a stream-oriented Unix domain socket used by logstash, or journald.

Default logging configuration

The following default configuration is used for logging, unless a user overwrites it with an explicit logging configuration block.

logging {
    console {
        context any info
    }
}

• console streams log messages to STDERR. A Linux OS with systemd captures these streaming log messages from STDERR to journald. You can use the journalctl command to examine these logs. For Aerospike instances running in a Docker container, the log stream is captured in the logger defined by the Docker daemon or Kubernetes.
• context any means "capture the log messages of all Aerospike logging contexts." A log context is associated with log messages originating in a particular server module, such as tls, drv_ssd, or fabric.

Logging contexts

To print a full list of logging contexts:

asinfo -v "log/" -l

misc:INFO
alloc:INFO
arenax:INFO
hardware:INFO
msg:INFO
os:INFO
rbuffer:INFO
socket:INFO
tls:INFO
vault:INFO
vmapx:INFO
xmem:INFO
aggr:INFO
appeal:INFO
as:INFO
audit:INFO
batch:INFO
batch-sub:INFO
bin:INFO
config:INFO
clustering:INFO
drv_pmem:INFO
drv_ssd:INFO
exchange:INFO
exp:INFO
fabric:INFO
flat:INFO
geo:INFO
hb:INFO
health:INFO
hlc:INFO
index:INFO
info:INFO
info-port:INFO
job:INFO
key-busy:DETAIL
migrate:INFO
mon:INFO
namespace:INFO
nsup:INFO
particle:INFO
partition:INFO
paxos:INFO
proto:INFO
proxy:INFO
proxy-divert:INFO
query:INFO
record:INFO
roster:INFO
rw:INFO
rw-client:INFO
security:INFO
service:INFO
service-list:INFO
sindex:INFO
skew:INFO
smd:INFO
storage:INFO
truncate:INFO
tsvc:INFO
udf:INFO
xdr:INFO
xdr-client:INFO

See the server log reference for examples of log messages by context.

Severity levels

The severity level controls how much information is logged. INFO is the default logging severity level for each logging context. Choose from the standard severity levels in the following table:

Severity	Description
critical	Critical error messages
warning	Warning messages
info	Informational messages
debug	Debugging information
detail	Very verbose debugging information

The severity levels are listed in the table from the least and most urgent amount of information logged, to the most verbose, which includes non-urgent messages about routine operations. Every severity level includes all higher levels. For example, info includes warning and critical messages.

The debug level is useful for troubleshooting. The detail level records a large volume of information, including the keys of records being read and written. For production systems, consider carefully the amount of logging information you need and set the severity level appropriately.

Defining log sinks

Aerospike log sinks are static configuration subcontexts of the logging context, which means that they must be defined in the configuration file ( /etc/aerospike/aerospike.conf), and then the Aerospike server node must be restarted.

note

Aerospike supports a maximum of eight log sinks.

Creating a file log sink

systemd journald aggregates all system logs into a binary format in /var/log/journal/, and you must use journalctl to view and filter the information you want. journalctl has a lot of filtering options, so a separate Aerospike log file is not necessary. However, a separate log file stores the logging stream in plain text, and is easier to browse.

The following example configures a separate log file for Aerospike. The directory that contains the log file must already exist:

logging {
        file /var/log/aerospike/aerospike.log {
            context any info
        }

        console {
            context any info
        }
}

Save your changes and restart Aerospike:

systemctl restart aerospike

When asd starts up, Aerospike creates the log sinks based on the logging configuration context. Use following command to verify the enumerated list of log sinks:

asinfo -v 'logs' -l

Expected output:

0:/var/log/aerospike/aerospike.log
1:stderr

Creating a log sink for debugging

You can direct a debugging log stream to a separate file. The following example configures a separate log file, tls_debug.log, only for debug level messages from the tls context:

logging {
        file /var/log/aerospike/tls_debug.log {
            context any critical
            context tls debug
        }

        file /var/log/aerospike/aerospike.log {
            context any info
        }

        console {
            context any info
        }
}

Creating a syslog socket sink for audit trail

Server 6.3 adds syslog, a new kind of log sink in addition to file and console. syslog is Unix domain socket that is compatible with the syslog protocol. You can direct the audit trail log stream to a socket with the following configuration:

logging {
        file /var/log/aerospike/tls_debug.log {
            context any critical
            context tls debug
        }

        file /var/log/aerospike/aerospike.log {
            context any info
        }

        syslog {
            facility local0
            tag aerospike-audit
            context any critical
            context audit info
        }
}

You must also enable the audit trail in your security configuration context, then save the configuration file and restart Aerospike.

Dynamically changing the severity level

Use asinfo to temporarily change the logging level of a context to help debug a problem. The following example shows the syntax:

asinfo -v "log-set:id=SINK-ID;CONTEXT=LOG-LEVEL"

• The SINK-ID is the log identifier, also called the logging sink. A logging sink is the destination of a logging stream. Use asinfo to identify your sinks:

  asinfo -v "logs" -l

  The example output shows there are two sinks:

  0:/var/log/aerospike/aerospike.log
  1:stderr

  Sink 0 is the log file on disk, sink 1 is stderr.

• Use CONTEXT=LOG-LEVEL to specify the context to change, and the severity level to set. For example, the following command sets the logging level to debug on the tls context:

  asinfo -v "log-set:id=0;tls=debug"

  Expected output:

  requested value log-set:id=0;tls=debug
  ok

• When you are finished debugging, change it back:

  asinfo -v "log-set:id=0;tls=info"

All other contexts remain at their configured logging levels.