Skip to main content

Install Aerospike SQL Powered by Starburst

Aerospike SQL Powered by Starburst provides a tightly-integrated SQL processing engine to the Aerospike Real-time Data Platform. Powered by Starburst Enterprise, Aerospike SQL provides an enterprise-grade implementation of the open source Trino (formerly Presto SQL) that is installed, managed and supported by Aerospike.

The combination of the Aerospike Database and the Starburst SQL engine offers a rapid and cost effective way for SQL users to generate valuable insights in dashboards, ad-hoc queries, or Python applications.

Aerospike SQL enables you to run Trino on a single machine, a cluster of machines, on-prem, or in the cloud. Aerospike SQL uses the Aerospike Presto (Trino) connector to reconcile the data model differences which manifest themselves while accessing a NoSQL database such as Aerospike (ASDB) using SQL via an SQL engine such as Trino.

Overview

A deployment of Aerospike SQL consists of four components.

  1. Starburst Admin: This is a collection of Ansible playbooks for installing and managing Starburst Enterprise platform (SEP) or Trino clusters.
  2. Starburst Coordinator node and Starburst Worker node(s): More information about these here.
  3. Aerospike Connect for Presto (Trino).
  4. Aerospike Database Cluster.

This image illustrates the architectural view. This guide walks you through the setup and configuration of this deployment.

Aerospike SQL architecture overview

Installing Aerospike SQL broadly consists of the following four steps:

  1. Install the Aerospike SQL package on Starburst Admin node.
  2. Install the following components using Ansible scripts in the aerospike-sql package:
    • Starburst Enterprise
    • Aerospike Connect for Presto (Trino)
  3. Place the license files.
  4. Start the deployment.

Prerequisites

Aerospike SQL is supported on Debian, CentOS 7, and Red Hat Enterprise Linux (RHEL) 8 with standard system utilities installed, including git, rpm, rsync, ssh, tar, wget, and zip/unzip.

danger

Before you proceed, verify passwordless SSH connectivity between the Starburst Admin to Starburst coordinator and worker(s).

Starburst Admin node requirements:

Starburst Coordinator and Worker node requirements:

Pre-Flight Checklist

caution

Go through the following table and ensure you meet each requirement. The installation process will fail if the requirements are not met.

LocationRequirement
Starburst Admin node
Python 3
Java JDK 11
Ansible
Able to SSH without a password to and from the coordinator and worker nodes.
Starburst Coordinator and Worker nodes
Python 3 (If using CentOS 7/RHEL 8)
Python (If using Debian)
Java JDK 11
jq command-line JSON processor
Aerospike nodesAerospike database version compatible with your Presto Trino connector. See this page for details.

Install the Aerospike SQL Package

Download the Aerospike SQL package to the Starburst admin node.

Install the Aerospike SQL package:

  • CentOS 7/8 and RHEL 7/8: sudo rpm -i aerospike-sql-1.0.7-1.linux.x86_64.rpm
  • Debian: sudo dpkg -i aerospike-sql-1.0.0-linux.x86_64.deb

Update Configurations

After the package is installed, update the configurations.

Go to the installed Aerospike SQL directory:

cd /opt/aerospike/aerospike-sql

Edit the following lines in the playbooks/host file. Replace:

  • [coordinator IP address] with the private IP address of the Starburst coordinator node.
  • [worker IP address] with the private IP address of the Starburst worker node(s).
  • [UID] with the UID of your Starburst Admin user id.
[coordinator]
[coordinator IP address] ansible_user=[UID] ansible_ssh_private_key_file=~/.ssh/id_rsa

[worker]
[worker_1_IP_address] ansible_user=[UID] ansible_ssh_private_key_file=~/.ssh/id_rsa
# Add all the worker nodes as applicable
#[worker_x_IP_address] ansible_user=[UID] ansible_ssh_private_key_file=~/.ssh/id_rsa
#[worker_y_IP_address] ansible_user=[UID] ansible_ssh_private_key_file=~/.ssh/id_rsa

Edit the following lines in the files/catalog/aerospike.properties file. Replace:

  • [aerospike_node_1_IP_addr], [aerospike_node_2_IP_addr], and [aerospike_node_3_IP_addr] with the public IP for the Aerospike nodes.
  • [UID] with the default user ID for your OS.
connector.name=aerospike
# Replace with your public IP for all aerospike nodes in the cluster
aerospike.hostlist=[aerospike_node_1_IP_addr]:3000,[aerospike_node_2_IP_addr]:3000,[aerospike_node_3_IP_addr]:3000
aerospike.split-number=8
aerospike.strict-schemas=false
aerospike.record-key-hidden=false
aerospike.enable-statistics=true
aerospike.insert-require-key=true
#Change [UID] as per your OS(eg: default user for CentOS/Fedora/RHEL on aws is “ec2-user”)
aerospike.table-desc-dir=/home/[UID]/
aerospike.clientpolicy.tls.enabled=false
aerospike.case-insensitive-identifiers=true

Use Ansible to Install Components

In the next stage, use the Ansible playbooks from the Aerospike SQL package to install Starburst Enterprise and the Aerospike Trino connector.

Use Ansible to Install Starburst Enterprise

Run the following Ansible command on the Starburst admin node to install Starburst Enterprise:

ansible-playbook   playbooks/install.yml

This downloads and installs Starburst Enterprise and Starburst Trino. This process should take less than 10 minutes.

When the installation successfully completes, the tail of the output looks like:


PLAY RECAP *****************************************************************************
172.31.2.13 : ok=14 changed=4 unreachable=0 failed=0 skipped=15 rescued=0 ignored=0
172.31.5.45 : ok=14 changed=4 unreachable=0 failed=0 skipped=15 rescued=0 ignored=0

For more information refer to the Starburst Trino configuration guide.

Use Ansible to Install the Aerospike Trino Connector

Run the following Ansible command on the Starburst admin node to install the Aerospike Trino Connector:

ansible-playbook   playbooks/aerospike.yml

This downloads and installs the Aerospike Trino connector.

When the installation successfully completes, the tail of the output looks like:

PLAY RECAP *****************************************************************************
172.31.2.13 : ok=9 changed=3 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
172.31.5.45 : ok=9 changed=3 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0

For more information please refer to the Aerospike Connector configuration guide

Place the License Files

In this stage of the deployment, place the Starburst and Aerospike license files as specified.

Starburst

Copy the Starburst license file (starburstdata.licence) to the /etc/starburst directory on the Starburst coordinator server.

Please contact your Aerospike Sales Representative to get the Starburst license file

Aerospike EE

Follow the instructions in our documentation to place the Aerospike Feature Key file on the Aerospike instance.

Start the Aerospike SQL Deployment

Update the Configurations

Use the following Ansible command on the Starburst admin node to generate the configuration file for the coordinator and workers, and distribute these files to all hosts:

ansible-playbook   playbooks/push-configs.yml

When this process is complete, the final output is:

PLAY RECAP *****************************************************************************
172.31.2.13 : ok=25 changed=17 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
172.31.5.45 : ok=25 changed=17 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Start Aerospike SQL

Use Ansible on the Starburst admin node to start the Aerospike SQL deployment:

ansible-playbook   playbooks/start.yml

When this process is complete, the final output is:

PLAY RECAP *****************************************************************************
172.31.2.13 : ok=3 changed=1 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
172.31.5.45 : ok=3 changed=1 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0

Verify the Deployment on Coordinator

Use the following command to show the active Starburst process and verify it is running:

ps -ef | grep starburst

The output of this command looks like:

starbur+ 13120     1 34 05:30 ?        00:00:28 java -cp /usr/lib/starburst/lib/* -server -Xmx16G -XX:-UseBiasedLocking -XX:+UseG1GC -XX:G1HeapRegionSize=32M -XX:+ExplicitGCInvokesConcurrent -XX:+ExitOnOutOfMemoryError -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -XX:ReservedCodeCacheSize=512M -XX:PerMethodRecompilationCutoff=10000 -XX:PerBytecodeRecompilationCutoff=10000 -Djdk.attach.allowAttachSelf=true -Djdk.nio.maxCachedBufferSize=2000000 -Dnode.launcher-log-file=/var/log/starburst/launcher.log -Dnode.environment=production -Dlog.enable-console=false -Dlog.levels-file=/etc/starburst/log.properties -Dlog.output-file=/var/log/starburst/server.log -Dnode.server-log-file=/var/log/starburst/server.log -Dcatalog.config-dir=/etc/starburst/catalog -Dconfig=/etc/starburst/config.properties -Dnode.data-dir=/var/lib/starburst -Dnode.id=1b176fe8-6f89-5bba-b826-df4bbdb4cf99 com.starburstdata.presto.StarburstTrinoServer
root 14054 13742 0 05:32 pts/0 00:00:00 grep --color=auto starburst

Visit the Aerospike SQL Query Editor Admin Panel

The Aerospike SQL Query Editor Admin Panel lets you view and edit your queries using a web browser.

Open a browser and go to the landing page: http://[coordinator_IP_addr]:8080/ui/insights/ide

Create a user ( trino in the example below), then use that user account to log in.

Log In

Log in to Aerospike SQL UI

Select Query editor

Select Aerospike SQL Query Editor

Query Editor:

The Aerospike SQL Query Editor

Manage the deployment

You can issue the following Ansible commands from the Starburst admin node to manage the deployment.

Stop services:

ansible-playbook playbooks/stop.yml   

Restart services:

ansible-playbook playbooks/restart.yml

Uninstall Starburst Enterprise:

ansible-playbook playbooks/uninstall.yml