ZooKeeper
Since Camel 2.9
Both producer and consumer are supported
The ZooKeeper component allows interaction with a ZooKeeper cluster and exposes the following features to Camel:
-
Creation of nodes in any of the ZooKeeper create modes.
-
Get and Set the data contents of arbitrary cluster nodes (data being set must be convertible to
byte[]
). -
Create and retrieve the list the child nodes attached to a particular node.
Maven users will need to add the following dependency to their pom.xml
for this component:
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-zookeeper</artifactId>
<version>x.x.x</version>
<!-- use the same version as your Camel core version -->
</dependency>
URI format
zookeeper://zookeeper-server[:port][/path][?options]
The path from the URI specifies the node in the ZooKeeper server (a.k.a. znode) that will be the target of the endpoint:
Options
The ZooKeeper component supports 11 options, which are listed below.
Name | Description | Default | Type |
---|---|---|---|
listChildren (common) |
Whether the children of the node should be listed |
false |
boolean |
timeout (common) |
The time interval to wait on connection before timing out. |
5000 |
int |
backoff (consumer) |
The time interval to backoff for after an error before retrying. |
5000 |
long |
bridgeErrorHandler (consumer) |
Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. |
false |
boolean |
repeat (consumer) |
Should changes to the znode be 'watched' and repeatedly processed. |
false |
boolean |
sendEmptyMessageOnDelete (consumer) |
Upon the delete of a znode, should an empty message be send to the consumer |
true |
boolean |
create (producer) |
Should the endpoint create the node if it does not currently exist. |
false |
boolean |
createMode (producer) |
The create mode that should be used for the newly created node. There are 4 enums and the value can be one of: PERSISTENT, PERSISTENT_SEQUENTIAL, EPHEMERAL, EPHEMERAL_SEQUENTIAL |
EPHEMERAL |
String |
lazyStartProducer (producer) |
Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. |
false |
boolean |
autowiredEnabled (advanced) |
Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc. |
true |
boolean |
configuration (advanced) |
To use a shared ZooKeeperConfiguration |
ZooKeeperConfiguration |
The ZooKeeper endpoint is configured using URI syntax:
zookeeper:serverUrls/path
with the following path and query parameters:
Path Parameters (2 parameters):
Name | Description | Default | Type |
---|---|---|---|
serverUrls |
Required The zookeeper server hosts (multiple servers can be separated by comma) |
String |
|
path |
Required The node in the ZooKeeper server (aka znode) |
String |
Query Parameters (12 parameters):
Name | Description | Default | Type |
---|---|---|---|
listChildren (common) |
Whether the children of the node should be listed |
false |
boolean |
timeout (common) |
The time interval to wait on connection before timing out. |
5000 |
int |
backoff (consumer) |
The time interval to backoff for after an error before retrying. |
5000 |
long |
bridgeErrorHandler (consumer) |
Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. |
false |
boolean |
repeat (consumer) |
Should changes to the znode be 'watched' and repeatedly processed. |
false |
boolean |
sendEmptyMessageOnDelete (consumer) |
Upon the delete of a znode, should an empty message be send to the consumer |
true |
boolean |
exceptionHandler (consumer) |
To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this option is not in use. By default the consumer will deal with exceptions, that will be logged at WARN or ERROR level and ignored. |
ExceptionHandler |
|
exchangePattern (consumer) |
Sets the exchange pattern when the consumer creates an exchange. There are 3 enums and the value can be one of: InOnly, InOut, InOptionalOut |
ExchangePattern |
|
create (producer) |
Should the endpoint create the node if it does not currently exist. |
false |
boolean |
createMode (producer) |
The create mode that should be used for the newly created node. There are 4 enums and the value can be one of: PERSISTENT, PERSISTENT_SEQUENTIAL, EPHEMERAL, EPHEMERAL_SEQUENTIAL |
EPHEMERAL |
String |
lazyStartProducer (producer) |
Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. |
false |
boolean |
synchronous (advanced) |
Sets whether synchronous processing should be strictly used, or Camel is allowed to use asynchronous processing (if supported). |
false |
boolean |
Use cases
Reading from a znode
The following snippet will read the data from the znode
/somepath/somenode/
provided that it already exists. The data
retrieved will be placed into an exchange and passed onto
the rest of the route:
from("zookeeper://localhost:39913/somepath/somenode").to("mock:result");
If the node does not yet exist then a flag can be supplied to have the endpoint await its creation:
from("zookeeper://localhost:39913/somepath/somenode?awaitCreation=true").to("mock:result");
Reading from a znode
When data is read due to a WatchedEvent
received from the ZooKeeper
ensemble, the CamelZookeeperEventType
header holds ZooKeeper’s
EventType
value from that WatchedEvent
. If the data is read initially (not
triggered by a WatchedEvent
) the CamelZookeeperEventType
header will not
be set.
Writing to a znode
The following snippet will write the payload of the exchange into the
znode at /somepath/somenode/
provided that it already exists:
from("direct:write-to-znode")
.to("zookeeper://localhost:39913/somepath/somenode");
For flexibility, the endpoint allows the target znode to be specified
dynamically as a message header. If a header keyed by the string
CamelZooKeeperNode
is present then the value of the header will be
used as the path to the znode on the server. For instance using the same
route definition above, the following code snippet will write the data
not to /somepath/somenode
but to the path from the header
/somepath/someothernode
.
the testPayload must be convertible
to byte[] as the data stored in ZooKeeper is byte based.
|
Object testPayload = ...
template.sendBodyAndHeader("direct:write-to-znode", testPayload, "CamelZooKeeperNode", "/somepath/someothernode");
To also create the node if it does not exist the create
option should
be used.
from("direct:create-and-write-to-znode")
.to("zookeeper://localhost:39913/somepath/somenode?create=true");
It is also possible to delete a node using the
header CamelZookeeperOperation
by setting it to DELETE
:
from("direct:delete-znode")
.setHeader(ZooKeeperMessage.ZOOKEEPER_OPERATION, constant("DELETE"))
.to("zookeeper://localhost:39913/somepath/somenode");
or equivalently:
<route>
<from uri="direct:delete-znode" />
<setHeader name="CamelZookeeperOperation">
<constant>DELETE</constant>
</setHeader>
<to uri="zookeeper://localhost:39913/somepath/somenode" />
</route>
ZooKeeper nodes can have different types; they can be 'Ephemeral' or
'Persistent' and 'Sequenced' or 'Unsequenced'. For further information
of each type you can check
here.
By default endpoints will create unsequenced, ephemeral nodes, but the
type can be easily manipulated via a uri config parameter or via a
special message header. The values expected for the create mode are
simply the names from the CreateMode
enumeration:
-
PERSISTENT
-
PERSISTENT_SEQUENTIAL
-
EPHEMERAL
-
EPHEMERAL_SEQUENTIAL
For example to create a persistent znode via the URI config:
from("direct:create-and-write-to-persistent-znode")
.to("zookeeper://localhost:39913/somepath/somenode?create=true&createMode=PERSISTENT");
or using the header CamelZookeeperCreateMode
.
the testPayload must be convertible to byte[] as the data stored in
ZooKeeper is byte based.
|
Object testPayload = ...
template.sendBodyAndHeader("direct:create-and-write-to-persistent-znode", testPayload, "CamelZooKeeperCreateMode", "PERSISTENT");
Spring Boot Auto-Configuration
When using zookeeper with Spring Boot make sure to use the following Maven dependency to have support for auto configuration:
<dependency>
<groupId>org.apache.camel.springboot</groupId>
<artifactId>camel-zookeeper-starter</artifactId>
<version>x.x.x</version>
<!-- use the same version as your Camel core version -->
</dependency>
The component supports 57 options, which are listed below.
Name | Description | Default | Type |
---|---|---|---|
camel.component.zookeeper.autowired-enabled |
Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc. |
true |
Boolean |
camel.component.zookeeper.backoff |
The time interval to backoff for after an error before retrying. |
5000 |
Long |
camel.component.zookeeper.bridge-error-handler |
Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. |
false |
Boolean |
camel.component.zookeeper.cluster.service.attributes |
Custom service attributes. |
Map |
|
camel.component.zookeeper.cluster.service.auth-info-list |
List of AuthInfo objects with scheme and auth. |
List |
|
camel.component.zookeeper.cluster.service.base-path |
The base path to store in ZooKeeper. |
String |
|
camel.component.zookeeper.cluster.service.connection-timeout |
Connection timeout. |
15000 |
Long |
camel.component.zookeeper.cluster.service.connection-timeout-unit |
Connection timeout TimeUnit. Default is TimeUnit.MILLISECONDS. |
TimeUnit |
|
camel.component.zookeeper.cluster.service.curator-framework |
Zookeeper CuratorFramework-style client. |
CuratorFramework |
|
camel.component.zookeeper.cluster.service.enabled |
Sets if the zookeeper cluster service should be enabled or not, default is false. |
false |
Boolean |
camel.component.zookeeper.cluster.service.id |
Cluster Service ID |
String |
|
camel.component.zookeeper.cluster.service.max-close-wait |
Time to wait during close to join background threads. |
1000 |
Long |
camel.component.zookeeper.cluster.service.max-close-wait-unit |
MaxCloseWait TimeUnit. Default is TimeUnit.MILLISECONDS. |
TimeUnit |
|
camel.component.zookeeper.cluster.service.namespace |
ZooKeeper namespace. If a namespace is set here, all paths will get pre-pended with the namespace. |
String |
|
camel.component.zookeeper.cluster.service.nodes |
The Zookeeper server hosts (multiple servers can be separated by comma). |
List |
|
camel.component.zookeeper.cluster.service.order |
Service lookup order/priority. |
Integer |
|
camel.component.zookeeper.cluster.service.reconnect-base-sleep-time |
Initial amount of time to wait between retries. |
0 |
Long |
camel.component.zookeeper.cluster.service.reconnect-base-sleep-time-unit |
ReconnectBaseSleepTime TimeUnit. Default is TimeUnit.MILLISECONDS. |
TimeUnit |
|
camel.component.zookeeper.cluster.service.reconnect-max-retries |
Max number of times to retry. |
3 |
Integer |
camel.component.zookeeper.cluster.service.reconnect-max-sleep-time |
Max time to sleep on each retry. Default is Integer.MAX_VALUE. |
Long |
|
camel.component.zookeeper.cluster.service.reconnect-max-sleep-time-unit |
ReconnectMaxSleepTimeUnit TimeUnit. Default is TimeUnit.MILLISECONDS. |
TimeUnit |
|
camel.component.zookeeper.cluster.service.retry-policy |
Retry policy to use. |
RetryPolicy |
|
camel.component.zookeeper.cluster.service.session-timeout |
Session timeout. |
60000 |
Long |
camel.component.zookeeper.cluster.service.session-timeout-unit |
Session timeout TimeUnit. Default is TimeUnit.MILLISECONDS. |
TimeUnit |
|
camel.component.zookeeper.configuration |
To use a shared ZooKeeperConfiguration. The option is a org.apache.camel.component.zookeeper.ZooKeeperConfiguration type. |
ZooKeeperConfiguration |
|
camel.component.zookeeper.create |
Should the endpoint create the node if it does not currently exist. |
false |
Boolean |
camel.component.zookeeper.create-mode |
The create mode that should be used for the newly created node |
EPHEMERAL |
String |
camel.component.zookeeper.enabled |
Whether to enable auto configuration of the zookeeper component. This is enabled by default. |
Boolean |
|
camel.component.zookeeper.lazy-start-producer |
Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. |
false |
Boolean |
camel.component.zookeeper.list-children |
Whether the children of the node should be listed |
false |
Boolean |
camel.component.zookeeper.repeat |
Should changes to the znode be 'watched' and repeatedly processed. |
false |
Boolean |
camel.component.zookeeper.send-empty-message-on-delete |
Upon the delete of a znode, should an empty message be send to the consumer |
true |
Boolean |
camel.component.zookeeper.service-registry.attributes |
Custom service attributes. |
Map |
|
camel.component.zookeeper.service-registry.auth-info-list |
List of AuthInfo objects with scheme and auth. |
List |
|
camel.component.zookeeper.service-registry.base-path |
The base path to store in ZooKeeper. |
String |
|
camel.component.zookeeper.service-registry.connection-timeout |
Connection timeout. |
15000 |
Long |
camel.component.zookeeper.service-registry.connection-timeout-unit |
Connection timeout TimeUnit. Default is TimeUnit.MILLISECONDS. |
TimeUnit |
|
camel.component.zookeeper.service-registry.curator-framework |
Zookeeper CuratorFramework-style client. |
CuratorFramework |
|
camel.component.zookeeper.service-registry.deregister-services-on-stop |
Should we remove all the registered services know by this registry on stop ? Default is true. |
true |
Boolean |
camel.component.zookeeper.service-registry.enabled |
Sets if the zookeeper service registry should be enabled or not, default is false. |
false |
Boolean |
camel.component.zookeeper.service-registry.id |
Service Registry ID |
String |
|
camel.component.zookeeper.service-registry.max-close-wait |
Time to wait during close to join background threads. |
1000 |
Long |
camel.component.zookeeper.service-registry.max-close-wait-unit |
MaxCloseWait TimeUnit. Default is TimeUnit.MILLISECONDS. |
TimeUnit |
|
camel.component.zookeeper.service-registry.namespace |
ZooKeeper namespace. If a namespace is set here, all paths will get pre-pended with the namespace. |
String |
|
camel.component.zookeeper.service-registry.nodes |
The Zookeeper server hosts (multiple servers can be separated by comma). |
List |
|
camel.component.zookeeper.service-registry.order |
Service lookup order/priority. |
Integer |
|
camel.component.zookeeper.service-registry.override-service-host |
Should we override the service host if given ? Default is true. |
true |
Boolean |
camel.component.zookeeper.service-registry.reconnect-base-sleep-time |
Initial amount of time to wait between retries. |
0 |
Long |
camel.component.zookeeper.service-registry.reconnect-base-sleep-time-unit |
ReconnectBaseSleepTime TimeUnit. Default is TimeUnit.MILLISECONDS. |
TimeUnit |
|
camel.component.zookeeper.service-registry.reconnect-max-retries |
Max number of times to retry. |
3 |
Integer |
camel.component.zookeeper.service-registry.reconnect-max-sleep-time |
Max time to sleep on each retry. Default is Integer.MAX_VALUE. |
Long |
|
camel.component.zookeeper.service-registry.reconnect-max-sleep-time-unit |
ReconnectMaxSleepTimeUnit TimeUnit. Default is TimeUnit.MILLISECONDS. |
TimeUnit |
|
camel.component.zookeeper.service-registry.retry-policy |
Retry policy to use. |
RetryPolicy |
|
camel.component.zookeeper.service-registry.service-host |
Service host. |
String |
|
camel.component.zookeeper.service-registry.session-timeout |
Session timeout. |
60000 |
Long |
camel.component.zookeeper.service-registry.session-timeout-unit |
Session timeout TimeUnit. Default is TimeUnit.MILLISECONDS. |
TimeUnit |
|
camel.component.zookeeper.timeout |
The time interval to wait on connection before timing out. |
5000 |
Integer |