Loading
Configure the Elasticsearch Client for Aerospike Connect for Elasticsearch
The es-client section of the aerospike-elasticsearch-outbound.yaml file specifies the configuration parameters for creating Elasticsearch Java Client. This client is used for all communication from Aerospike Connect for Elasticsearch to the Elasticsearch cluster.
Cluster configuration
The cluster-config section defines the Elasticsearch cluster type.
| Option | Required | Description | Details |
|---|---|---|---|
| type | Yes | The type of the Elasticsearch cluster. | The supported options are on-prem and cloud. |
Each type has a specific required configuration.
| Type | Option | Description | Details |
|---|---|---|---|
| on-prem | nodes | A list of Elasticsearch nodes to connect to. All valid properties of Node can be configured. | Choose this option if you manage the Elasticsearch cluster, and you have access to the cluster nodes. |
| cloud | cloud-id | A valid elastic cloud cloudId that will route to a cluster. The cloudId is located in your Elastic Cloud user console | Choose this option if you are using the Elastic Cloud hosted service. |
Default Headers
An optional default-headers section is a set of key-value pairs that sets the default request headers which are sent with each request.
Example
...
es-client:
...
default-headers:
environment: prod
...
...
Auth configuration
The auth-config section sets options for authenticating with the Elasticsearch cluster.
| Option | Required | Default | Description | Details |
|---|---|---|---|---|
| type | Yes | none | The type of authentication method. | The supported options are basic, token, api-key, and none. |
Some of these types have extra configuration options.
| Config Type | Option | Required | Description |
|---|---|---|---|
| basic | username | Yes | Username for basic auth. |
| basic | password-file | Yes | File path of the file containing the password for basic authentication. |
| token | token-file | Yes | File path of the file containing the authentication token. |
| api-key | api-key-id-file | Yes | File path of the file containing the API key ID. |
| api-key | api-key-secret-file | Yes | File path of the file containing the API key Secret. |
Transport options
The transport-options section is optional. It sets configurations for all application-specific or request-specific options, including headers and query parameters.
| Option | Default | Description |
|---|---|---|
| headers | empty map | All request specific headers. |
| parameters | empty map | All request specific headers query parameters. |
Request Config
The elasticsearch-request-config section sets options for customizing the underlying HTTP client at request level. RequestConfig has more details on some of these options:
| Option | Default | Description |
|---|---|---|
| expect-continue-enabled | false | Determines whether the 'Expect: 100-Continue' handshake is enabled for entity enclosing methods. Refer to RequestConfig for more details. |
| proxy | null | HTTP proxy for request execution. All valid properties of HttpHost can be configured. |
| local-address | null | Local address for request execution. |
| stale-connection-check-enabled | false | Whether or not to use stale connection check. |
| cookie-spec | null | The name of the cookie specification for HTTP state management. |
| redirects-enabled | true | Whether or not redirects are handled automatically. |
| relative-redirects-allowed | true | Whether or not relative redirects are rejected. HTTP specification requires the location value be an absolute URI. |
| circular-redirects-allowed | false | Whether or not circular redirects (redirects to the same location) are allowed. |
| max-redirects | 50 | The maximum number of redirects to be followed. The limit on number of redirects is intended to prevent infinite loops. |
| authentication-enabled | true | Whether or not authentication is handled automatically. |
| target-preferred-auth-schemes | null | The order of preference for supported authentication schemes in the form of a list when authenticating with the target host. |
| proxy-preferred-auth-schemes | null | The order of preference for supported authentication schemes in the form of a list when authenticating with the proxy host. |
| connection-request-timeout | -1 | Timeout in milliseconds used when requesting a connection from the connection manager. |
| connect-timeout | -1 | Timeout in milliseconds until a connection is established. |
| socket-timeout | -1 | Socket timeout SO_TIMEOUT in milliseconds. The timeout for waiting for data, specifically, the maximum period of inactivity between two consecutive data packets. |
| content-compression-enabled | true | Whether or not the target server is asked to compress content. |
| normalize-uri | true | Whether or not the client should normalize URIs in requests. |
Other configurations
You can also configure a few optional parameters:
| Option | Default | Description |
|---|---|---|
| enable-compression | false | Whether the REST client should compress requests using gzip content encoding and add the "Accept-Encoding: gzip" header to receive compressed responses. |
| path-prefix | Empty String | Sets the path's prefix for every request used by the HTTP client. |
| meta-header-enabled | false | Whether to send a X-Elastic-Client-Meta header that describes the runtime environment. |
| strict-deprecation-mode-enabled | false | Whether the REST client should return any response containing at least one warning header as a failure. |
| tls-config | null | An optional TLS configuration for connecting to the Elasticsearch cluster. The configuration options here are the same as Service section's TLS config. |
| max-conn-total | 0 | Maximum total connections to the Elasticsearch cluster. |
| max-conn-per-route | 0 | Maximum connection per route to the Elasticsearch cluster. |
| conn-time-to-live-millis | -1 | Maximum time to live in milliseconds for persistent connections. |
| io-thread-count | No. of available processors | The number of I/O dispatch threads to be used by the I/O reactor. |
| user-agent | Defaults to aerospike-elasticsearch-outbound:{connector-version} where connector-version is the version of the deployed aerospike-elasticsearch-outbound connector. | The User-agent to be set with every request. |
| cookie-management-disabled | false | Disable state (cookie) management. |
| auth-caching-disabled | false | Disable authentication scheme caching. |
| connection-state-disabled | false | Disable connection state tracking. |
Example
...
es-client:
cluster-config:
type: cloud
cloud-id: abc123
default-headers:
env: dev
enable-compression: true
path-prefix: /my/prefix
meta-header-enabled: true
strict-deprecation-mode-enabled: true
auth-config:
type: basic
username: elastic
password-file: /etc/aerospike-elasticsearch-outbound/secret/basic-auth-password.txt
tls-config:
trust-store:
store-file: /etc/aerospike-elasticsearch-outbound/secret/tls/ca.my-domain.com.truststore.jks
store-password-file: /etc/aerospike-elasticsearch-outbound/secret/tls/storepass
transport-options:
headers:
over-tls: true
parameters:
my-param1: hello-world
elasticsearch-request-config:
expect-continue-enabled: true
proxy:
hostname: localhost
port: 1234
scheme: http
local-address: 192.168.123.234
stale-connection-check-enabled: true
cookie-spec: test-cookie-spec
redirects-enabled: false
relative-redirects-allowed: false
circular-redirects-allowed: false
max-redirects: 5
authentication-enabled: true
target-preferred-auth-schemes:
- Basic
- NTLM
proxy-preferred-auth-schemes:
- Kerberos
- CredSSP
connection-request-timeout: 2500
connect-timeout: 1000
socket-timeout: 10000
content-compression-enabled: false
normalize-uri: false
max-conn-total: 10
max-conn-per-route: 20
conn-time-to-live-millis: 30000
io-thread-count: 30
user-agent: my-test-webapp
cookie-management-disabled: false
auth-caching-disabled: true
connection-state-disabled: true
...