ElSQL

Since Camel 2.16

Both producer and consumer are supported

The ELSQL component is an extension to the existing SQL Component that uses ElSql to define the SQL queries.

This component uses spring-jdbc behind the scenes for the actual SQL handling.

This component can be used as a Transactional Client.

Maven users will need to add the following dependency to their pom.xml for this component:

<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-elsql</artifactId>
    <version>x.x.x</version>
    <!-- use the same version as your Camel core version -->
</dependency>

The SQL component uses the following endpoint URI notation:

sql:elSqlName:resourceUri[?options]

You can append query options to the URI in the following format, ?option=value&option=value&…​

The parameters to the SQL queries are named parameters in the elsql mapping files, and maps to corresponding keys from the Camel message, in the given precedence:

  1. from message body if Simple expression.

  2. from message body if its a java.util.Map

  3. from message headers

If a named parameter cannot be resolved, then an exception is thrown.

Configuring Options

Camel components are configured on two separate levels:

  • component level

  • endpoint level

Configuring Component Options

The component level is the highest level which holds general and common configurations that are inherited by the endpoints. For example a component may have security settings, credentials for authentication, urls for network connection and so forth.

Some components only have a few options, and others may have many. Because components typically have pre configured defaults that are commonly used, then you may often only need to configure a few options on a component; or none at all.

Configuring components can be done with the Component DSL, in a configuration file (application.properties|yaml), or directly with Java code.

Configuring Endpoint Options

Where you find yourself configuring the most is on endpoints, as endpoints often have many options, which allows you to configure what you need the endpoint to do. The options are also categorized into whether the endpoint is used as consumer (from) or as a producer (to), or used for both.

Configuring endpoints is most often done directly in the endpoint URI as path and query parameters. You can also use the Endpoint DSL as a type safe way of configuring endpoints.

A good practice when configuring options is to use Property Placeholders, which allows to not hardcode urls, port numbers, sensitive information, and other settings. In other words placeholders allows to externalize the configuration from your code, and gives more flexibility and reuse.

The following two sections lists all the options, firstly for the component followed by the endpoint.

Component Options

The ElSQL component supports 7 options, which are listed below.

Name Description Default Type

databaseVendor (common)

To use a vendor specific com.opengamma.elsql.ElSqlConfig.

Enum values:

  • Default

  • Postgres

  • HSql

  • MySql

  • Oracle

  • SqlServer2008

  • Veritca

ElSqlDatabaseVendor

dataSource (common)

Autowired Sets the DataSource to use to communicate with the database.

DataSource

resourceUri (common)

The resource file which contains the elsql SQL statements to use. You can specify multiple resources separated by comma. The resources are loaded on the classpath by default, you can prefix with file: to load from file system. Notice you can set this option on the component and then you do not have to configure this on the endpoint.

String

bridgeErrorHandler (consumer)

Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.

boolean

lazyStartProducer (producer)

Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing.

boolean

autowiredEnabled (advanced)

Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc.

true

boolean

elSqlConfig (advanced)

To use a specific configured ElSqlConfig. It may be better to use the databaseVendor option instead.

ElSqlConfig

Endpoint Options

The ElSQL endpoint is configured using URI syntax:

elsql:elsqlName:resourceUri

with the following path and query parameters:

Path Parameters (2 parameters)

Name Description Default Type

elsqlName (common)

Required The name of the elsql to use (is NAMED in the elsql file).

String

resourceUri (common)

The resource file which contains the elsql SQL statements to use. You can specify multiple resources separated by comma. The resources are loaded on the classpath by default, you can prefix with file: to load from file system. Notice you can set this option on the component and then you do not have to configure this on the endpoint.

String

Query Parameters (47 parameters)

Name Description Default Type

allowNamedParameters (common)

Whether to allow using named parameters in the queries.

true

boolean

databaseVendor (common)

To use a vendor specific com.opengamma.elsql.ElSqlConfig.

Enum values:

  • Default

  • Postgres

  • HSql

  • MySql

  • Oracle

  • SqlServer2008

  • Veritca

ElSqlDatabaseVendor

dataSource (common)

Autowired Sets the DataSource to use to communicate with the database at endpoint level.

DataSource

outputClass (common)

Specify the full package and class name to use as conversion when outputType=SelectOne.

String

outputHeader (common)

Store the query result in a header instead of the message body. By default, outputHeader == null and the query result is stored in the message body, any existing content in the message body is discarded. If outputHeader is set, the value is used as the name of the header to store the query result and the original message body is preserved.

String

outputType (common)

Make the output of consumer or producer to SelectList as List of Map, or SelectOne as single Java object in the following way: a) If the query has only single column, then that JDBC Column object is returned. (such as SELECT COUNT( ) FROM PROJECT will return a Long object. b) If the query has more than one column, then it will return a Map of that result. c) If the outputClass is set, then it will convert the query result into an Java bean object by calling all the setters that match the column names. It will assume your class has a default constructor to create instance with. d) If the query resulted in more than one rows, it throws an non-unique result exception. StreamList streams the result of the query using an Iterator. This can be used with the Splitter EIP in streaming mode to process the ResultSet in streaming fashion.

Enum values:

  • SelectOne

  • SelectList

  • StreamList

SelectList

SqlOutputType

separator (common)

The separator to use when parameter values is taken from message body (if the body is a String type), to be inserted at # placeholders. Notice if you use named parameters, then a Map type is used instead. The default value is comma.

,

char

breakBatchOnConsumeFail (consumer)

Sets whether to break batch if onConsume failed.

boolean

bridgeErrorHandler (consumer)

Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.

boolean

expectedUpdateCount (consumer)

Sets an expected update count to validate when using onConsume.

-1

int

maxMessagesPerPoll (consumer)

Sets the maximum number of messages to poll.

int

onConsume (consumer)

After processing each row then this query can be executed, if the Exchange was processed successfully, for example to mark the row as processed. The query can have parameter.

String

onConsumeBatchComplete (consumer)

After processing the entire batch, this query can be executed to bulk update rows etc. The query cannot have parameters.

String

onConsumeFailed (consumer)

After processing each row then this query can be executed, if the Exchange failed, for example to mark the row as failed. The query can have parameter.

String

routeEmptyResultSet (consumer)

Sets whether empty resultset should be allowed to be sent to the next hop. Defaults to false. So the empty resultset will be filtered out.

boolean

sendEmptyMessageWhenIdle (consumer)

If the polling consumer did not poll any files, you can enable this option to send an empty message (no body) instead.

boolean

transacted (consumer)

Enables or disables transaction. If enabled then if processing an exchange failed then the consumer breaks out processing any further exchanges to cause a rollback eager.

boolean

useIterator (consumer)

Sets how resultset should be delivered to route. Indicates delivery as either a list or individual object. defaults to true.

true

boolean

exceptionHandler (consumer (advanced))

To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this option is not in use. By default the consumer will deal with exceptions, that will be logged at WARN or ERROR level and ignored.

ExceptionHandler

exchangePattern (consumer (advanced))

Sets the exchange pattern when the consumer creates an exchange.

Enum values:

  • InOnly

  • InOut

  • InOptionalOut

ExchangePattern

pollStrategy (consumer (advanced))

A pluggable org.apache.camel.PollingConsumerPollingStrategy allowing you to provide your custom implementation to control error handling usually occurred during the poll operation before an Exchange have been created and being routed in Camel.

PollingConsumerPollStrategy

processingStrategy (consumer (advanced))

Allows to plugin to use a custom org.apache.camel.component.sql.SqlProcessingStrategy to execute queries when the consumer has processed the rows/batch.

SqlProcessingStrategy

batch (producer)

Enables or disables batch mode.

boolean

lazyStartProducer (producer)

Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing.

boolean

noop (producer)

If set, will ignore the results of the SQL query and use the existing IN message as the OUT message for the continuation of processing.

boolean

useMessageBodyForSql (producer)

Whether to use the message body as the SQL and then headers for parameters. If this option is enabled then the SQL in the uri is not used. Note that query parameters in the message body are represented by a question mark instead of a # symbol.

boolean

alwaysPopulateStatement (advanced)

If enabled then the populateStatement method from org.apache.camel.component.sql.SqlPrepareStatementStrategy is always invoked, also if there is no expected parameters to be prepared. When this is false then the populateStatement is only invoked if there is 1 or more expected parameters to be set; for example this avoids reading the message body/headers for SQL queries with no parameters.

boolean

elSqlConfig (advanced)

To use a specific configured ElSqlConfig. It may be better to use the databaseVendor option instead.

ElSqlConfig

parametersCount (advanced)

If set greater than zero, then Camel will use this count value of parameters to replace instead of querying via JDBC metadata API. This is useful if the JDBC vendor could not return correct parameters count, then user may override instead.

int

placeholder (advanced)

Specifies a character that will be replaced to in SQL query. Notice, that it is simple String.replaceAll() operation and no SQL parsing is involved (quoted strings will also change).

#

String

prepareStatementStrategy (advanced)

Allows to plugin to use a custom org.apache.camel.component.sql.SqlPrepareStatementStrategy to control preparation of the query and prepared statement.

SqlPrepareStatementStrategy

templateOptions (advanced)

Configures the Spring JdbcTemplate with the key/values from the Map.

Map

usePlaceholder (advanced)

Sets whether to use placeholder and replace all placeholder characters with sign in the SQL queries.

true

boolean

backoffErrorThreshold (scheduler)

The number of subsequent error polls (failed due some error) that should happen before the backoffMultipler should kick-in.

int

backoffIdleThreshold (scheduler)

The number of subsequent idle polls that should happen before the backoffMultipler should kick-in.

int

backoffMultiplier (scheduler)

To let the scheduled polling consumer backoff if there has been a number of subsequent idles/errors in a row. The multiplier is then the number of polls that will be skipped before the next actual attempt is happening again. When this option is in use then backoffIdleThreshold and/or backoffErrorThreshold must also be configured.

int

delay (scheduler)

Milliseconds before the next poll.

500

long

greedy (scheduler)

If greedy is enabled, then the ScheduledPollConsumer will run immediately again, if the previous run polled 1 or more messages.

boolean

initialDelay (scheduler)

Milliseconds before the first poll starts.

1000

long

repeatCount (scheduler)

Specifies a maximum limit of number of fires. So if you set it to 1, the scheduler will only fire once. If you set it to 5, it will only fire five times. A value of zero or negative means fire forever.

long

runLoggingLevel (scheduler)

The consumer logs a start/complete log line when it polls. This option allows you to configure the logging level for that.

Enum values:

  • TRACE

  • DEBUG

  • INFO

  • WARN

  • ERROR

  • OFF

TRACE

LoggingLevel

scheduledExecutorService (scheduler)

Allows for configuring a custom/shared thread pool to use for the consumer. By default each consumer has its own single threaded thread pool.

ScheduledExecutorService

scheduler (scheduler)

To use a cron scheduler from either camel-spring or camel-quartz component. Use value spring or quartz for built in scheduler.

none

Object

schedulerProperties (scheduler)

To configure additional properties when using a custom scheduler or any of the Quartz, Spring based scheduler.

Map

startScheduler (scheduler)

Whether the scheduler should be auto started.

true

boolean

timeUnit (scheduler)

Time unit for initialDelay and delay options.

Enum values:

  • NANOSECONDS

  • MICROSECONDS

  • MILLISECONDS

  • SECONDS

  • MINUTES

  • HOURS

  • DAYS

MILLISECONDS

TimeUnit

useFixedDelay (scheduler)

Controls if fixed delay or fixed rate is used. See ScheduledExecutorService in JDK for details.

true

boolean

Result of the query

For select operations, the result is an instance of List<Map<String, Object>> type, as returned by the JdbcTemplate.queryForList() method. For update operations, the result is the number of updated rows, returned as an Integer.

By default, the result is placed in the message body. If the outputHeader parameter is set, the result is placed in the header. This is an alternative to using a full message enrichment pattern to add headers, it provides a concise syntax for querying a sequence or some other small value into a header. It is convenient to use outputHeader and outputType together:

Header values

When performing update operations, the SQL Component stores the update count in the following message headers:

Header Description

CamelSqlUpdateCount

The number of rows updated for update operations, returned as an Integer object.

CamelSqlRowCount

The number of rows returned for select operations, returned as an Integer object.

Sample

In the given route below, we want to get all the projects from the projects table. Notice the SQL query has 2 named parameters, :#lic and :#min.

Camel will then lookup for these parameters from the message body or message headers. Notice in the example above we set two headers with constant value
for the named parameters:

   from("direct:projects")
     .setHeader("lic", constant("ASF"))
     .setHeader("min", constant(123))
     .to("elsql:projects:com/foo/orders.elsql")

And the elsql mapping file

@NAME(projects)
  SELECT *
  FROM projects
  WHERE license = :lic AND id > :min
  ORDER BY id

Though if the message body is a java.util.Map then the named parameters will be taken from the body.

   from("direct:projects")
     .to("elsql:projects:com/foo/orders.elsql")

Using expression parameters in producers

In from Camel 2.16.1 onwards you can use Simple expressions as well, which allows to use an OGNL like notation on the message body, where it assumes to have getLicense and getMinimum methods:

@NAME(projects)
  SELECT *
  FROM projects
  WHERE license = :${body.license} AND id > :${body.minimum}
  ORDER BY id

Using expression parameters in consumers

When using the ElSql component as consumer, you can now also use expression parameters (simple language) to build dynamic query parameters, such as calling a method on a bean to retrieve an id, date or something.

For example in the sample below we call the nextId method on the bean myIdGenerator:

@NAME(projectsByIdBean)
  SELECT *
  FROM projects
  WHERE id = :${bean#myIdGenerator.nextId}
Notice in the bean syntax above, we must use # instead of : in the simple expression. This is because Spring query parameter parser is in-use which will separate parameters on colon. Also pay attention that Spring query parser will invoke the bean twice for each query.

And the bean has the following method:

public static class MyIdGenerator {

    private int id = 1;

    public int nextId() {
        // spring will call this twice, one for initializing query and 2nd for actual value
        id++;
        return id / 2;
    }

Notice that there is no existing Exchange with message body and headers, so the simple expression you can use in the consumer are most useable for calling bean methods as in this example.

Spring Boot Auto-Configuration

When using elsql with Spring Boot make sure to use the following Maven dependency to have support for auto configuration:

<dependency>
  <groupId>org.apache.camel.springboot</groupId>
  <artifactId>camel-elsql-starter</artifactId>
  <version>x.x.x</version>
  <!-- use the same version as your Camel core version -->
</dependency>

The component supports 7 options, which are listed below.

Name Description Default Type

camel.component.elsql.autowired-enabled

Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc.

true

Boolean

camel.component.elsql.bridge-error-handler

Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.

false

Boolean

camel.component.elsql.database-vendor

To use a vendor specific com.opengamma.elsql.ElSqlConfig

ElSqlDatabaseVendor

camel.component.elsql.el-sql-config

To use a specific configured ElSqlConfig. It may be better to use the databaseVendor option instead. The option is a com.opengamma.elsql.ElSqlConfig type.

ElSqlConfig

camel.component.elsql.enabled

Whether to enable auto configuration of the elsql component. This is enabled by default.

Boolean

camel.component.elsql.lazy-start-producer

Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing.

false

Boolean

camel.component.elsql.resource-uri

The resource file which contains the elsql SQL statements to use. You can specify multiple resources separated by comma. The resources are loaded on the classpath by default, you can prefix with file: to load from file system. Notice you can set this option on the component and then you do not have to configure this on the endpoint.

String
Back to top