Skip to main content
Loading

Modifying an Existing Configuration of the Pulsar Inbound Connector

You can modify the configuration of the Pulsar inbound connector even after the connector is deployed.

To view the current configuration that is being used by the instances of the connector on all nodes of the Pulsar Function cluster, run this command either from a Pulsar Functions worker node or, if you include the optional flag --admin-url, from a separate system on your network. The command shown here can be run from a Pulsar Functions worker node:

<pulsar-dir>/bin/pulsar-admin sinks get \
    --name "aerospike-pulsar-sink" \
    --tenant <tenant> \
    --namespace <namespace>
  • <pulsar-dir>: The directory where Pulsar is installed.
  • --tenant: The Pulsar tenant in which the connector is installed.
  • --namespace: The Pulsar namespace (within the tenant) in which the connector is installed.
  • --admin-url: The Pulsar Admin URL (webServiceUrl): Include this optional flag if you want to run the command outside of your cluster on a system that is in your network.

The configs section in the output of the command shows the keys and values that are set in the configuration file, as in this example output:

{
  "tenant": "public",
  "namespace": "default",
  "name": "aerospike-pulsar-sink",
  "className": "com.aerospike.connect.pulsar.inbound.PulsarSinkConnector",
  "inputSpecs": {
    "players": {
      "isRegexPattern": false,
      "schemaProperties": {},
      "consumerProperties": {}
    },
    "retired-players": {
      "isRegexPattern": false,
      "schemaProperties": {},
      "consumerProperties": {}
    }
  },
  "configs": {
    "topics": {
      "players": {
        "aerospike-operation": {
          "type": "write",
          "total-timeout": 0
        },
        "mapping": {
          "bins": {
            "type": "multi-bins",
            "map": {
              "name": {
                "source": "value-field",
                "field-name": "name"
              }
            }
          },
          "key-field": {
            "source": "key"
          },
          "namespace": {
            "mode": "dynamic",
            "source": "value-field",
            "field-name": "namespace"
          },
          "ttl": {
            "mode": "dynamic",
            "source": "value-field",
            "field-name": "ttl"
          }
        }
      },
      "retired-players": {
        "aerospike-operation": {
          "type": "delete"
        },
        "mapping": {
          "key-field": {
            "source": "key"
          },
          "namespace": {
            "mode": "static",
            "value": "test"
          }
        }
      }
    },
    "aerospike": {
      "seeds": [
        {
          "192.168.0.0": {
            "port": 49316
          }
        },
        {
          "192.168.0.0": {
            "port": 49312
          }
        }
      ]
    }
  },
  "parallelism": 1,
  "processingGuarantees": "EFFECTIVELY_ONCE",
  "retainOrdering": true,
  "autoAck": false,
  "archive": "builtin://aerospike"
}

Procedure​

Edit the configuration file aerospike-pulsar-inbound.yml​

  1. Configure the topics section.
  2. Configure the aerospike section.

Request that the instances of the connector use the changed configuration​

The sinks update command accepts all the arguments accepted by sinks create command. As with the sinks create and sinks get commands, you can run this command either from a Pulsar Functions worker node or, if you include the optional flag --admin-url, from a separate system on your network. The command shown here can be run from a Pulsar Functions worker node:

<pulsar-dir>/bin/pulsar-admin sinks update \
    --tenant <value> \
    --namespace <value> \
    --name "aerospike-pulsar-sink" \
    --sink-type aerospike \
    --inputs <value> \
    --sink-config-file <value> \
    --parallelism <value>
  • <pulsar-dir>: The directory where Pulsar is installed.
  • --tenant: The Pulsar tenant in which the connector is installed.
  • --namespace: The Pulsar namespace (within the tenant) in which the connector is installed.
  • --inputs: The Pulsar topic or topics to use as input to the connector. You can specify multiple topics as a comma-separated list. All topics that you list here must be listed in the aerospike-pulsar-inbound.yml config file.
  • --sink-config-file: The path to a aerospike-pulsar-inbound.yml configuration file. If you are running the command on a system outside of the cluster, you must specify the path to a copy of the configuration file on that system.
  • --parallelism: The number of connector instances to run. For example, if you set the value to 3, three instances of the connector run in the cluster.
  • --admin-url: The Pulsar Admin URL (webServiceUrl): Include this optional flag if you want to run the command outside of your cluster on a system that is in your network.

Verify that the instances of the connector are using the changed configuration​

To verify, run the same command that you ran to view the configuration:

<pulsar-dir>/bin/pulsar-admin sinks get \
    --name "aerospike-pulsar-sink" \
    --tenant <tenant> \
    --namespace <namespace>
  • <pulsar-dir>: The directory where Pulsar is installed.
  • --tenant: The Pulsar tenant in which the connector is installed.
  • --namespace: The Pulsar namespace (within the tenant) in which the connector is installed.
  • --admin-url: The Pulsar Admin URL (webServiceUrl): Include this optional flag if you want to run the command outside of your cluster on a system that is in your network.
Last updated on