Aggregate

The Aggregate EIP supports 27 options which are listed below:

The aggregate EIP will always use a worker pool, that is used to process all the outgoing messages from the aggregator. The worker pool is determined accordingly:

The AggregationStrategy is used for aggregating the old (lookup by its correlation id) and the new exchanges together into a single exchange. Possible implementations include performing some kind of combining or delta processing, such as adding line items together into an invoice or just using the newest exchange and removing old exchanges such as for state tracking or market data prices; where old values are of little use.

Notice the aggregation strategy is a mandatory option and must be provided to the aggregator.

Here are a few example AggregationStrategy implementations that should help you create your own custom strategy.

When aggregation Exchanges at some point you need to indicate that the aggregated exchanges is complete, so they can be send out of the aggregator. Camel allows you to indicate completion in various ways as follows:

Notice that all the completion ways are per correlation key. And you can combine them in any way you like. It’s basically the first which triggers that wins. So you can use a completion size together with a completion timeout. Only completionTimeout and completionInterval cannot be used at the same time.

Notice the completion is a mandatory option and must be provided to the aggregator. If not provided Camel will thrown an Exception on startup.

There can be use-cases where you want the incoming Exchange to determine if the correlation group should pre-complete, and then the incoming Exchange is starting a new group from scratch. o determine this the AggregationStrategy must override the canPreComplete method which has to return true.

If the preComplete method returns true, then the existing groups is completed (without aggregating the incoming exchange (newExchange). And then the newExchange is used to start the correlation group from scratch so the group would contain only that new incoming exchange. This is known as pre-completion mode. And when the aggregation is in pre-completion mode, then only the following completions are in use

The aggregator provides a pluggable repository which you can implement your own org.apache.camel.spi.AggregationRepository.
If you need persistent repository then you can use either Camel LevelDB, or SQL Component components.

If your aggregation strategy implements TimeoutAwareAggregationStrategy, then Camel will invoke the timeout method when the timeout occurs. Notice that the values for index and total parameters will be -1, and the timeout parameter will be provided only if configured as a fixed value. You must not throw any exceptions from the timeout method.

If your aggregation strategy implements CompletionAwareAggregationStrategy, then Camel will invoke the onComplete method when the aggregated Exchange is completed. This allows you to do any last minute custom logic such as to cleanup some resources, or additional work on the exchange as it’s now completed.
You must not throw any exceptions from the onCompletion method.

The AggregationStrategy can now included a property on the returned Exchange that contains a boolean to indicate if the current group should be completed. This allows to overrule any existing completion predicates / sizes / timeouts etc, and complete the group.

For example the following logic (from an unit test) will complete the group if the message body size is larger than 5. This is done by setting the exchange property Exchange.AGGREGATION_COMPLETE_CURRENT_GROUP to true.

The AggregationStrategy can now included a property on the returned Exchange that contains a boolean to indicate if all previous groups should be completed. This allows to overrule any existing completion predicates / sizes / timeouts etc, and complete all the existing previous group.

For example the following logic (from an unit test) will complete all the previous group when a new aggregation group is started. This is done by setting the property Exchange.AGGREGATION_COMPLETE_ALL_GROUPS to true.

You can manually trigger completion of all current aggregated exchanges by sending an exchange containing the exchange property Exchange.AGGREGATION_COMPLETE_ALL_GROUPS set to true. The message is considered a signal message only, the message headers/contents will not be processed otherwise.

You can alternatively set the exchange property Exchange.AGGREGATION_COMPLETE_ALL_GROUPS_INCLUSIVE to true to trigger completion of all groups after processing the current message.

If you want to aggregate some value from the messages <V> into a List<V> then we have added a org.apache.camel.processor.aggregate.AbstractListAggregationStrategy abstract class that makes this easier. The completed Exchange that is sent out of the aggregator will contain the List<V> in the message body.

For example to aggregate a List<Integer> you can extend this class as shown below, and implement the getValue method:

The org.apache.camel.processor.aggregate.AggregateController allows you to control the aggregate at runtime using Java or JMX API. This can be used to force completing groups of exchanges, or query its current runtime statistics.

The aggregator provides a default implementation if no custom have been configured, which can be accessed using getAggregateController() method. Though it may be easier to configure a controller in the route using aggregateController as shown below:

Then there is API on AggregateController to force completion. For example to complete a group with key foo

The number return would be the number of groups completed. In this case it would be 1 if the foo group existed and was completed. If foo does not exists then 0 is returned.

There is also an api to complete all groups

To configure this from XML DSL

There is also JMX API on the aggregator which is available under the processors node in the Camel JMX tree.

In the route below we group all the exchanges together using GroupedExchangeAggregationStrategy:

As a result we have one outgoing Exchange being routed to the "mock:result" endpoint. The exchange is a holder containing all the incoming Exchanges.

The output of the aggregator will then contain the exchanges grouped together in a list as shown below:

To use the AggregationStrategy you had to implement the org.apache.camel.AggregationStrategy interface, which means your logic would be tied to the Camel API. You can use a POJO for the logic and let Camel adapt to your POJO. To use a POJO a convention must be followed:

The paired methods is expected to be ordered as follows:

This convention is best explained with some examples.

In the method below, we have only 2 parameters, so the 1st parameter is the body of the oldExchange, and the 2nd is paired to the body of the newExchange:

In the method below, we have only 4 parameters, so the 1st parameter is the body of the oldExchange, and the 2nd is the Map of the oldExchange headers, and the 3rd is paired to the body of the newExchange, and the 4th parameter is the Map of the newExchange headers:

And finally if we have 6 parameters the we also have the properties of the Exchanges:

To use this with the Aggregate EIP we can use a POJO with the aggregate logic as follows:

And then in the Camel route we create an instance of our bean, and then refer to the bean in the route using bean method from org.apache.camel.builder.AggregationStrategies as shown:

We can also provide the bean type directly:

And if the bean has only one method we do not need to specify the name of the method:

And the append method could be static:

If you are using XML DSL then we need to declare a <bean> with the POJO:

And in the Camel route we use strategyRef to refer to the bean by its id, and the strategyMethodName can be used to define the method name to call:

When using XML DSL you must define the POJO as a <bean>.

By default when using POJOs as AggregationStrategy, then the method is only invoked when there is data to be aggregated (by default). You can use the option strategyMethodAllowNull to configure this. Where as without using POJOs then you may have null as oldExchange or newExchange parameters. For example the Aggregate EIP will invoke the AggregationStrategy with oldExchange as null, for the first Exchange incoming to the aggregator. And then for subsequent Exchanges then oldExchange and newExchange parameters are both not null.

Example with Content Enricher EIP and no data

Though with POJOs as AggregationStrategy we made this simpler and only call the method when oldExchange and newExchange is not null, as that would be the most common use-case. If you need to allow oldExchange or newExchange to be null, then you can configure this with the POJO using the AggregationStrategyBeanAdapter as shown below. On the bean adapter we call setAllowNullNewExchange to allow the new exchange to be null.

This can be configured a bit easier using the beanAllowNull method from AggregationStrategies as shown:

Then the append method in the POJO would need to deal with the situation that newExchange can be null:

In the example above we use the Content Enricher EIP using pollEnrich. The newExchange will be null in the situation we could not get any data from the "seda:foo" endpoint, and therefore the timeout was hit after 1 second. So if we need to do some special merge logic we would need to set setAllowNullNewExchange=true, so the append method will be invoked. If we do not do that then when the timeout was hit, then the append method would normally not be invoked, meaning the Content Enricher did not merge/change the message.

In XML DSL you would configure the strategyMethodAllowNull option and set it to true as shown below:

When for example using strategyMethodAllowNull as true, then the parameter types of the message bodies does not have to be the same. For example suppose we want to aggregate from a com.foo.User type to a List<String> that contains the user name. We could code a POJO doing this as follows:

Notice that the return type is a List which we want to contain the user names. The 1st parameter is the list of names, and then notice the 2nd parameter is the incoming com.foo.User type.