Azure Storage Blob Service

Since Camel 3.3

Both producer and consumer are supported

The Azure Storage Blob component is used for storing and retrieving blobs from Azure Storage Blob Service using Azure APIs v12. However in case of versions above v12, we will see if this component can adopt these changes depending on how much breaking changes can result.

Prerequisites

You must have a valid Windows Azure Storage account. More information is available at Azure Documentation Portal.

Maven users will need to add the following dependency to their pom.xml for this component:

<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-azure-storage-blob</artifactId>
    <version>x.x.x</version>
  <!-- use the same version as your Camel core version -->
</dependency>

URI Format

azure-storage-blob://accountName[/containerName][?options]

In case of consumer, accountName, containerName are required. In case of producer, it depends on the operation that being requested, for example if operation is on a container level, e.b: createContainer, accountName and containerName are only required, but in case of operation being requested in blob level, e.g: getBlob, accountName, containerName and blobName are required.

The blob will be created if it does not already exist. You can append query options to the URI in the following format, ?options=value&option2=value&…​

For example in order to download a blob content from the block blob hello.txt located on the container1 in the camelazure storage account, use the following snippet:

from("azure-storage-blob:/camelazure/container1?blobName=hello.txt&accessKey=yourAccessKey").
to("file://blobdirectory");

URI Options

The Azure Storage Blob Service component supports 28 options, which are listed below.

Name Description Default Type

autoDiscoverClient (common)

Setting the autoDiscoverClient mechanism, if true, the component will look for a client instance in the registry automatically otherwise it will skip that checking.

true

boolean

blobName (common)

The blob name, to consume specific blob from a container. However on producer, is only required for the operations on the blob level

String

blobOffset (common)

Set the blob offset for the upload or download operations, default is 0

0

long

blobType (common)

The blob type in order to initiate the appropriate settings for each blob type. There are 3 enums and the value can be one of: blockblob, appendblob, pageblob

blockblob

BlobType

closeStreamAfterRead (common)

Close the stream after read or keep it open, default is true

true

boolean

configuration (common)

The component configurations

BlobConfiguration

credentials (common)

StorageSharedKeyCredential can be injected to create the azure client, this holds the important authentication information

StorageSharedKeyCredential

dataCount (common)

How many bytes to include in the range. Must be greater than or equal to 0 if specified.

Long

fileDir (common)

The file directory where the downloaded blobs will be saved to, this can be used in both, producer and consumer

String

maxResultsPerPage (common)

Specifies the maximum number of blobs to return, including all BlobPrefix elements. If the request does not specify maxResultsPerPage or specifies a value greater than 5,000, the server will return up to 5,000 items.

Integer

maxRetryRequests (common)

Specifies the maximum number of additional HTTP Get requests that will be made while reading the data from a response body.

0

int

prefix (common)

Filters the results to return only blobs whose names begin with the specified prefix. May be null to return all blobs.

String

regex (common)

Filters the results to return only blobs whose names match the specified regular expression. May be null to return all if both prefix and regex are set, regex takes the priority and prefix is ignored.

String

serviceClient (common)

Client to a storage account. This client does not hold any state about a particular storage account but is instead a convenient way of sending off appropriate requests to the resource on the service. It may also be used to construct URLs to blobs and containers. This client contains operations on a service account. Operations on a container are available on BlobContainerClient through BlobServiceClient#getBlobContainerClient(String), and operations on a blob are available on BlobClient through BlobContainerClient#getBlobClient(String).

BlobServiceClient

timeout (common)

An optional timeout value beyond which a RuntimeException will be raised.

Duration

bridgeErrorHandler (consumer)

Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.

false

boolean

blobSequenceNumber (producer)

A user-controlled value that you can use to track requests. The value of the sequence number must be between 0 and 263 - 1.The default value is 0.

0

Long

blockListType (producer)

Specifies which type of blocks to return. There are 3 enums and the value can be one of: committed, uncommitted, all

COMMITTED

BlockListType

closeStreamAfterWrite (producer)

Close the stream after write or keep it open, default is true

true

boolean

commitBlockListLater (producer)

When is set to true, the staged blocks will not be committed directly.

true

boolean

createAppendBlob (producer)

When is set to true, the append blocks will be created when committing append blocks.

true

boolean

createPageBlob (producer)

When is set to true, the page blob will be created when uploading page blob.

true

boolean

downloadLinkExpiration (producer)

Override the default expiration (millis) of URL download link.

Long

lazyStartProducer (producer)

Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing.

false

boolean

operation (producer)

The blob operation that can be used with this component on the producer. There are 19 enums and the value can be one of: listBlobContainers, createBlobContainer, deleteBlobContainer, listBlobs, getBlob, deleteBlob, downloadBlobToFile, downloadLink, uploadBlockBlob, stageBlockBlobList, commitBlobBlockList, getBlobBlockList, createAppendBlob, commitAppendBlob, createPageBlob, uploadPageBlob, resizePageBlob, clearPageBlob, getPageBlobRanges

listBlobContainers

BlobOperationsDefinition

pageBlobSize (producer)

Specifies the maximum size for the page blob, up to 8 TB. The page blob size must be aligned to a 512-byte boundary.

512

Long

autowiredEnabled (advanced)

Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc.

true

boolean

accessKey (security)

Access key for the associated azure account name to be used for authentication with azure blob services

String

The Azure Storage Blob Service endpoint is configured using URI syntax:

azure-storage-blob:accountName/containerName

with the following path and query parameters:

Path Parameters (2 parameters):

Name Description Default Type

accountName

Azure account name to be used for authentication with azure blob services

String

containerName

The blob container name

String

Query Parameters (30 parameters):

Name Description Default Type

autoDiscoverClient (common)

Setting the autoDiscoverClient mechanism, if true, the component will look for a client instance in the registry automatically otherwise it will skip that checking.

true

boolean

blobName (common)

The blob name, to consume specific blob from a container. However on producer, is only required for the operations on the blob level

String

blobOffset (common)

Set the blob offset for the upload or download operations, default is 0

0

long

blobServiceClient (common)

Client to a storage account. This client does not hold any state about a particular storage account but is instead a convenient way of sending off appropriate requests to the resource on the service. It may also be used to construct URLs to blobs and containers. This client contains operations on a service account. Operations on a container are available on BlobContainerClient through getBlobContainerClient(String), and operations on a blob are available on BlobClient through getBlobContainerClient(String).getBlobClient(String).

BlobServiceClient

blobType (common)

The blob type in order to initiate the appropriate settings for each blob type. There are 3 enums and the value can be one of: blockblob, appendblob, pageblob

blockblob

BlobType

closeStreamAfterRead (common)

Close the stream after read or keep it open, default is true

true

boolean

credentials (common)

StorageSharedKeyCredential can be injected to create the azure client, this holds the important authentication information

StorageSharedKeyCredential

dataCount (common)

How many bytes to include in the range. Must be greater than or equal to 0 if specified.

Long

fileDir (common)

The file directory where the downloaded blobs will be saved to, this can be used in both, producer and consumer

String

maxResultsPerPage (common)

Specifies the maximum number of blobs to return, including all BlobPrefix elements. If the request does not specify maxResultsPerPage or specifies a value greater than 5,000, the server will return up to 5,000 items.

Integer

maxRetryRequests (common)

Specifies the maximum number of additional HTTP Get requests that will be made while reading the data from a response body.

0

int

prefix (common)

Filters the results to return only blobs whose names begin with the specified prefix. May be null to return all blobs.

String

regex (common)

Filters the results to return only blobs whose names match the specified regular expression. May be null to return all if both prefix and regex are set, regex takes the priority and prefix is ignored.

String

serviceClient (common)

Client to a storage account. This client does not hold any state about a particular storage account but is instead a convenient way of sending off appropriate requests to the resource on the service. It may also be used to construct URLs to blobs and containers. This client contains operations on a service account. Operations on a container are available on BlobContainerClient through BlobServiceClient#getBlobContainerClient(String), and operations on a blob are available on BlobClient through BlobContainerClient#getBlobClient(String).

BlobServiceClient

timeout (common)

An optional timeout value beyond which a RuntimeException will be raised.

Duration

bridgeErrorHandler (consumer)

Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.

false

boolean

exceptionHandler (consumer)

To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this option is not in use. By default the consumer will deal with exceptions, that will be logged at WARN or ERROR level and ignored.

ExceptionHandler

exchangePattern (consumer)

Sets the exchange pattern when the consumer creates an exchange. There are 3 enums and the value can be one of: InOnly, InOut, InOptionalOut

ExchangePattern

blobSequenceNumber (producer)

A user-controlled value that you can use to track requests. The value of the sequence number must be between 0 and 263 - 1.The default value is 0.

0

Long

blockListType (producer)

Specifies which type of blocks to return. There are 3 enums and the value can be one of: committed, uncommitted, all

COMMITTED

BlockListType

closeStreamAfterWrite (producer)

Close the stream after write or keep it open, default is true

true

boolean

commitBlockListLater (producer)

When is set to true, the staged blocks will not be committed directly.

true

boolean

createAppendBlob (producer)

When is set to true, the append blocks will be created when committing append blocks.

true

boolean

createPageBlob (producer)

When is set to true, the page blob will be created when uploading page blob.

true

boolean

downloadLinkExpiration (producer)

Override the default expiration (millis) of URL download link.

Long

lazyStartProducer (producer)

Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing.

false

boolean

operation (producer)

The blob operation that can be used with this component on the producer. There are 19 enums and the value can be one of: listBlobContainers, createBlobContainer, deleteBlobContainer, listBlobs, getBlob, deleteBlob, downloadBlobToFile, downloadLink, uploadBlockBlob, stageBlockBlobList, commitBlobBlockList, getBlobBlockList, createAppendBlob, commitAppendBlob, createPageBlob, uploadPageBlob, resizePageBlob, clearPageBlob, getPageBlobRanges

listBlobContainers

BlobOperationsDefinition

pageBlobSize (producer)

Specifies the maximum size for the page blob, up to 8 TB. The page blob size must be aligned to a 512-byte boundary.

512

Long

synchronous (advanced)

Sets whether synchronous processing should be strictly used, or Camel is allowed to use asynchronous processing (if supported).

false

boolean

accessKey (security)

Access key for the associated azure account name to be used for authentication with azure blob services

String

Required information options:

To use this component, you have 3 options in order to provide the required Azure authentication information:

  • Provide accountName and accessKey for your Azure account, this is the simplest way to get started. The accessKey can be generated through your Azure portal.

  • Provide a StorageSharedKeyCredential instance which can be provided into credentials option.

  • Provide a BlobServiceClient instance which can be provided into blobServiceClient. Note: You don’t need to create a specific client, e.g: BlockBlobClient, the BlobServiceClient represents the upper level which can be used to retrieve lower level clients.

Batch Consumer

This component implements the Batch Consumer.

This allows you for instance to know how many messages exists in this batch and for instance let the Aggregator aggregate this number of messages.

Usage

Message headers evaluated by the component producer

Header Variable Name Type Operations Description

CamelAzureStorageBlobTimeout

BlobConstants.TIMEOUT

Duration

All

An optional timeout value beyond which a {@link RuntimeException} will be raised.

CamelAzureStorageBlobMetadata

BlobConstants.METADATA

Map<String,String>

Operations related to container and blob

Metadata to associate with the container or blob.

CamelAzureStorageBlobPublicAccessType

BlobConstants.PUBLIC_ACCESS_TYPE

PublicAccessType

createContainer

Specifies how the data in this container is available to the public. Pass null for no public access.

CamelAzureStorageBlobRequestCondition

BlobConstants.BLOB_REQUEST_CONDITION

BlobRequestConditions

Operations related to container and blob

This contains values which will restrict the successful operation of a variety of requests to the conditions present. These conditions are entirely optional.

CamelAzureStorageBlobListDetails

BlobConstants.BLOB_LIST_DETAILS

BlobListDetails

listBlobs

The details for listing specific blobs

CamelAzureStorageBlobPrefix

BlobConstants.PREFIX

String

listBlobs,getBlob

Filters the results to return only blobs whose names begin with the specified prefix. May be null to return all blobs.

CamelAzureStorageBlobMaxResultsPerPage

BlobConstants.MAX_RESULTS_PER_PAGE

Integer

listBlobs

Specifies the maximum number of blobs to return, including all BlobPrefix elements. If the request does not specify maxResultsPerPage or specifies a value greater than 5,000, the server will return up to 5,000 items.

CamelAzureStorageBlobListBlobOptions

BlobConstants.LIST_BLOB_OPTIONS

ListBlobsOptions

listBlobs

Defines options available to configure the behavior of a call to listBlobsFlatSegment on a {@link BlobContainerClient} object.

CamelAzureStorageBlobHttpHeaders

BlobConstants.BLOB_HTTP_HEADERS

BlobHttpHeaders

uploadBlockBlob, commitBlobBlockList, createAppendBlob, createPageBlob

Additional parameters for a set of operations.

CamelAzureStorageBlobAccessTier

BlobConstants.ACCESS_TIER

AccessTier

uploadBlockBlob, commitBlobBlockList

Defines values for AccessTier.

CamelAzureStorageBlobContentMD5

BlobConstants.CONTENT_MD5

byte[]

Most operations related to upload blob

An MD5 hash of the block content. This hash is used to verify the integrity of the block during transport. When this header is specified, the storage service compares the hash of the content that has arrived with this header value. Note that this MD5 hash is not stored with the blob. If the two hashes do not match, the operation will fail.

CamelAzureStorageBlobPageBlobRange

BlobConstants.PAGE_BLOB_RANGE

PageRange

Operations related to page blob

A {@link PageRange} object. Given that pages must be aligned with 512-byte boundaries, the start offset must be a modulus of 512 and the end offset must be a modulus of 512 - 1. Examples of valid byte ranges are 0-511, 512-1023, etc.

CamelAzureStorageBlobCommitBlobBlockListLater

BlobConstants.COMMIT_BLOCK_LIST_LATER

boolean

stageBlockBlobList

When is set to true, the staged blocks will not be committed directly.

CamelAzureStorageBlobCreateAppendBlob

BlobConstants.CREATE_APPEND_BLOB

boolean

commitAppendBlob

When is set to true, the append blocks will be created when committing append blocks.

CamelAzureStorageBlobCreatePageBlob

BlobConstants.CREATE_PAGE_BLOB

boolean

uploadPageBlob

When is set to true, the page blob will be created when uploading page blob.

CamelAzureStorageBlobBlockListType

BlobConstants.BLOCK_LIST_TYPE

BlockListType

getBlobBlockList

Specifies which type of blocks to return.

CamelAzureStorageBlobPageBlobSize

BlobConstants.PAGE_BLOB_SIZE

Long

createPageBlob, resizePageBlob

Specifies the maximum size for the page blob, up to 8 TB. The page blob size must be aligned to a 512-byte boundary.

CamelAzureStorageBlobSequenceNumber

BlobConstants.BLOB_SEQUENCE_NUMBER

Long

createPageBlob

A user-controlled value that you can use to track requests. The value of the sequence number must be between 0 and 2^63 - 1.The default value is 0.

CamelAzureStorageBlobDeleteSnapshotsOptionType

BlobConstants.DELETE_SNAPSHOT_OPTION_TYPE

DeleteSnapshotsOptionType

deleteBlob

Specifies the behavior for deleting the snapshots on this blob. {@code Include} will delete the base blob and all snapshots. {@code Only} will delete only the snapshots. If a snapshot is being deleted, you must pass null.

CamelAzureStorageBlobListBlobContainersOptions

BlobConstants.LIST_BLOB_CONTAINERS_OPTIONS

ListBlobContainersOptions

listBlobContainers

A {@link ListBlobContainersOptions} which specifies what data should be returned by the service.

CamelAzureStorageBlobParallelTransferOptions

BlobConstants.PARALLEL_TRANSFER_OPTIONS

ParallelTransferOptions

downloadBlobToFile

{@link ParallelTransferOptions} to use to download to file. Number of parallel transfers parameter is ignored.

CamelAzureStorageBlobFileDir

BlobConstants.FILE_DIR

String

downloadBlobToFile

The file directory where the downloaded blobs will be saved to.

CamelAzureStorageBlobDownloadLinkExpiration

BlobConstants.DOWNLOAD_LINK_EXPIRATION

Long

downloadLink

Override the default expiration (millis) of URL download link.

CamelAzureStorageBlobBlobName

BlobConstants.BLOB_NAME

String

Operations related to blob

Override/set the blob name on the exchange headers.

CamelAzureStorageBlobContainerName

BlobConstants.BLOB_CONTAINER_NAME

String

Operations related to container and blob

Override/set the container name on the exchange headers.

CamelAzureStorageBlobOperation

BlobConstants.BLOB_OPERATION

BlobOperationsDefinition

All

Specify the producer operation to execute, please see the doc on this page related to producer operation.

CamelAzureStorageBlobRegex

BlobConstants.REGEX

String

listBlobs,getBlob

Filters the results to return only blobs whose names match the specified regular expression. May be null to return all. If both prefix and regex are set, regex takes the priority and prefix is ignored.

Message headers set by either component producer or consumer

Header Variable Name Type Description

CamelAzureStorageBlobAccessTier

BlobConstants.ACCESS_TIER

AccessTier

Access tier of the blob.

CamelAzureStorageBlobAccessTierChangeTime

BlobConstants.ACCESS_TIER_CHANGE_TIME

OffsetDateTime

Datetime when the access tier of the blob last changed.

CamelAzureStorageBlobArchiveStatus

BlobConstants.ARCHIVE_STATUS

ArchiveStatus

Archive status of the blob.

CamelAzureStorageBlobCreationTime

BlobConstants.CREATION_TIME

OffsetDateTime

Creation time of the blob.

CamelAzureStorageBlobSequenceNumber

BlobConstants.BLOB_SEQUENCE_NUMBER

Long

The current sequence number for a page blob.

CamelAzureStorageBlobBlobSize

BlobConstants.BLOB_SIZE

long

The size of the blob.

CamelAzureStorageBlobBlobType

BlobConstants.BLOB_TYPE

BlobType

The type of the blob.

CamelAzureStorageBlobCacheControl

BlobConstants.CACHE_CONTROL

String

Cache control specified for the blob.

CamelAzureStorageBlobCommittedBlockCount

BlobConstants.COMMITTED_BLOCK_COUNT

Integer

Number of blocks committed to an append blob

CamelAzureStorageBlobContentDisposition

BlobConstants.CONTENT_DISPOSITION

String

Content disposition specified for the blob.

CamelAzureStorageBlobContentEncoding

BlobConstants.CONTENT_ENCODING

String

Content encoding specified for the blob.

CamelAzureStorageBlobContentLanguage

BlobConstants.CONTENT_LANGUAGE

String

Content language specified for the blob.

CamelAzureStorageBlobContentMd5

BlobConstants.CONTENT_MD5

byte[]

Content MD5 specified for the blob.

CamelAzureStorageBlobContentType

BlobConstants.CONTENT_TYPE

String

Content type specified for the blob.

CamelAzureStorageBlobCopyCompletionTime

BlobConstants.COPY_COMPILATION_TIME

OffsetDateTime

Datetime when the last copy operation on the blob completed.

CamelAzureStorageBlobCopyDestinationSnapshot

BlobConstants.COPY_DESTINATION_SNAPSHOT

String

Snapshot identifier of the last incremental copy snapshot for the blob.

CamelAzureStorageBlobCopyId

BlobConstants.COPY_ID

String

Identifier of the last copy operation performed on the blob.

CamelAzureStorageBlobCopyProgress

BlobConstants.COPY_PROGRESS

String

Progress of the last copy operation performed on the blob.

CamelAzureStorageBlobCopySource

BlobConstants.COPY_SOURCE

String

Source of the last copy operation performed on the blob.

CamelAzureStorageBlobCopyStatus

BlobConstants.COPY_STATUS

CopyStatusType

Status of the last copy operation performed on the blob.

CamelAzureStorageBlobCopyStatusDescription

BlobConstants.COPY_STATUS_DESCRIPTION

String

Description of the last copy operation on the blob.

CamelAzureStorageBlobETag

BlobConstants.E_TAG

String

The E Tag of the blob

CamelAzureStorageBlobIsAccessTierInferred

BlobConstants.IS_ACCESS_TIER_INFRRRED

boolean

Flag indicating if the access tier of the blob was inferred from properties of the blob.

CamelAzureStorageBlobIsIncrementalCopy

BlobConstants.IS_INCREMENTAL_COPY

boolean

Flag indicating if the blob was incrementally copied.

CamelAzureStorageBlobIsServerEncrypted

BlobConstants.IS_SERVER_ENCRYPTED

boolean

Flag indicating if the blob’s content is encrypted on the server.

CamelAzureStorageBlobLastModified

BlobConstants.LAST_MODIFIED

OffsetDateTime

Datetime when the blob was last modified.

CamelAzureStorageBlobLeaseDuration

BlobConstants.LEASE_DURATION

LeaseDurationType

Type of lease on the blob.

CamelAzureStorageBlobLeaseState

BlobConstants.LEASE_STATE

LeaseStateType

State of the lease on the blob.

CamelAzureStorageBlobLeaseStatus

BlobConstants.LEASE_STATUS

LeaseStatusType

Status of the lease on the blob.

CamelAzureStorageBlobMetadata

BlobConstants.METADATA

Map<String, String>

Additional metadata associated with the blob.

CamelAzureStorageBlobAppendOffset

BlobConstants.APPEND_OFFSET

String

The offset at which the block was committed to the block blob.

CamelAzureStorageBlobFileName

BlobConstants.FILE_NAME

String

The downloaded filename from the operation downloadBlobToFile.

CamelAzureStorageBlobDownloadLink

BlobConstants.DOWNLOAD_LINK

String

The download link generated by downloadLink operation.

CamelAzureStorageBlobRawHttpHeaders

BlobConstants.RAW_HTTP_HEADERS

HttpHeaders

Returns non-parsed httpHeaders that can be used by the user.

Advanced Azure Storage Blob configuration

If your Camel Application is running behind a firewall or if you need to have more control over the BlobServiceClient instance configuration, you can create your own instance:

StorageSharedKeyCredential credential = new StorageSharedKeyCredential("yourAccountName", "yourAccessKey");
String uri = String.format("https://%s.blob.core.windows.net", "yourAccountName");

BlobServiceClient client = new BlobServiceClientBuilder()
                          .endpoint(uri)
                          .credential(credential)
                          .buildClient();
// This is camel context
context.getRegistry().bind("client", client);

Then refer to this instance in your Camel azure-storage-blob component configuration:

from("azure-storage-blob://cameldev/container1?blobName=myblob&serviceClient=#client")
.to("mock:result");

Automatic detection of BlobServiceClient client in registry

The component is capable of detecting the presence of an BlobServiceClient bean into the registry. If it’s the only instance of that type it will be used as client and you won’t have to define it as uri parameter, like the example above. This may be really useful for smarter configuration of the endpoint.

Azure Storage Blob Producer operations

Camel Azure Storage Blob component provides wide range of operations on the producer side:

Operations on the service level

For these operations, accountName is required.

Operation Description

listBlobContainers

Get the content of the blob. You can restrict the output of this operation to a blob range.

Operations on the container level

For these operations, accountName and containerName are required.

Operation Description

createBlobContainer

Creates a new container within a storage account. If a container with the same name already exists, the producer will ignore it.

deleteBlobContainer

Deletes the specified container in the storage account. If the container doesn’t exist the operation fails.

listBlobs

Returns a list of blobs in this container, with folder structures flattened.

Operations on the blob level

For these operations, accountName, containerName and blobName are required.

Operation Blob Type Description

getBlob

Common

Get the content of the blob. You can restrict the output of this operation to a blob range.

deleteBlob

Common

Delete a blob.

downloadBlobToFile

Common

Downloads the entire blob into a file specified by the path.The file will be created and must not exist, if the file already exists a {@link FileAlreadyExistsException} will be thrown.

downloadLink

Common

Generates the download link for the specified blob using shared access signatures (SAS). This by default only limit to 1hour of allowed access. However, you can override the default expiration duration through the headers.

uploadBlockBlob

BlockBlob

Creates a new block blob, or updates the content of an existing block blob. Updating an existing block blob overwrites any existing metadata on the blob. Partial updates are not supported with PutBlob; the content of the existing blob is overwritten with the new content.

stageBlockBlobList

BlockBlob

Uploads the specified block to the block blob’s "staging area" to be later committed by a call to commitBlobBlockList. However in case header CamelAzureStorageBlobCommitBlobBlockListLater or config commitBlockListLater is set to false, this will commit the blocks immediately after staging the blocks.

commitBlobBlockList

BlockBlob

Writes a blob by specifying the list of block IDs that are to make up the blob. In order to be written as part of a blob, a block must have been successfully written to the server in a prior stageBlockBlobList operation. You can call commitBlobBlockList to update a blob by uploading only those blocks that have changed, then committing the new and existing blocks together. Any blocks not specified in the block list and permanently deleted.

getBlobBlockList

BlockBlob

Returns the list of blocks that have been uploaded as part of a block blob using the specified block list filter.

createAppendBlob

AppendBlob

Creates a 0-length append blob. Call commitAppendBlo`b operation to append data to an append blob.

commitAppendBlob

AppendBlob

Commits a new block of data to the end of the existing append blob. In case of header CamelAzureStorageBlobCreateAppendBlob or config createAppendBlob is set to true, it will attempt to create the appendBlob through internal call to createAppendBlob operation first before committing.

createPageBlob

PageBlob

Creates a page blob of the specified length. Call uploadPageBlob operation to upload data data to a page blob.

uploadPageBlob

PageBlob

Writes one or more pages to the page blob. The write size must be a multiple of 512. In case of header CamelAzureStorageBlobCreatePageBlob or config createPageBlob is set to true, it will attempt to create the appendBlob through internal call to createPageBlob operation first before uploading.

resizePageBlob

PageBlob

Resizes the page blob to the specified size (which must be a multiple of 512).

clearPageBlob

PageBlob

Frees the specified pages from the page blob. The size of the range must be a multiple of 512.

getPageBlobRanges

PageBlob

Returns the list of valid page ranges for a page blob or snapshot of a page blob.

Refer to the example section in this page to learn how to use these operations into your camel application.

Consumer Examples

To consume a blob into a file using file component, this can be done like this:

from("azure-storage-blob:/camelazure/container1?blobName=hello.txt&accountName=yourAccountName&accessKey=yourAccessKey").
to("file://blobdirectory");

However, you can also write to file directly without using the file component, you will need to specify fileDir folder path in order to save your blob in your machine.

from("azure-storage-blob:/camelazure/container1?blobName=hello.txt&accountName=yourAccountName&accessKey=yourAccessKey&fileDir=/var/to/awesome/dir").
to("mock:results");

Also, the component supports batch consumer, hence you can consume multiple blobs with only specifying the container name, the consumer will return multiple exchanges depending on the number of the blobs in the container. Example:

from("azure-storage-blob:/camelazure/container1?accountName=yourAccountName&accessKey=yourAccessKey&fileDir=/var/to/awesome/dir").
to("mock:results");

Producer Operations Examples

  • listBlobContainers:

from("direct:start")
  .process(exchange -> {
    // set the header you want the producer to evaluate, refer to the previous
    // section to learn about the headers that can be set
    // e.g:
    exchange.getIn().setHeader(BlobConstants.LIST_BLOB_CONTAINERS_OPTIONS, new ListBlobContainersOptions().setMaxResultsPerPage(10));
  })
  .to("azure-storage-blob:/camelazure?operation=listBlobContainers&client&serviceClient=#client")
  .to("mock:result");

  • createBlobContainer:

from("direct:start")
  .process(exchange -> {
    // set the header you want the producer to evaluate, refer to the previous
    // section to learn about the headers that can be set
    // e.g:
    exchange.getIn().setHeader(BlobConstants.BLOB_CONTAINER_NAME, "newContainerName");
  })
  .to("azure-storage-blob:/camelazure/container1?operation=createBlobContainer&serviceClient=#client")
  .to("mock:result");

  • deleteBlobContainer:

from("direct:start")
  .process(exchange -> {
    // set the header you want the producer to evaluate, refer to the previous
    // section to learn about the headers that can be set
    // e.g:
    exchange.getIn().setHeader(BlobConstants.BLOB_CONTAINER_NAME, "overridenName");
  })
  .to("azure-storage-blob:/camelazure/container1?operation=deleteBlobContainer&serviceClient=#client")
  .to("mock:result");

  • listBlobs:

from("direct:start")
  .process(exchange -> {
    // set the header you want the producer to evaluate, refer to the previous
    // section to learn about the headers that can be set
    // e.g:
    exchange.getIn().setHeader(BlobConstants.BLOB_CONTAINER_NAME, "overridenName");
  })
  .to("azure-storage-blob:/camelazure/container1?operation=listBlobs&serviceClient=#client")
  .to("mock:result");

  • getBlob:

We can either set an outputStream in the exchange body and write the data to it. E.g:

from("direct:start")
  .process(exchange -> {
    // set the header you want the producer to evaluate, refer to the previous
    // section to learn about the headers that can be set
    // e.g:
    exchange.getIn().setHeader(BlobConstants.BLOB_CONTAINER_NAME, "overridenName");

    // set our body
    exchange.getIn().setBody(outputStream);
  })
  .to("azure-storage-blob:/camelazure/container1?blobName=blob&operation=getBlob&serviceClient=#client")
  .to("mock:result");

If we don’t set a body, then this operation will give us an InputStream instance which can proceeded further downstream:

from("direct:start")
  .to("azure-storage-blob:/camelazure/container1?blobName=blob&operation=getBlob&serviceClient=#client")
  .process(exchange -> {
      InputStream inputStream = exchange.getMessage().getBody(InputStream.class);
      // We use Apache common IO for simplicity, but you are free to do whatever dealing
      // with inputStream
      System.out.println(IOUtils.toString(inputStream, StandardCharsets.UTF_8.name()));
  })
  .to("mock:result");

  • deleteBlob:

from("direct:start")
  .process(exchange -> {
    // set the header you want the producer to evaluate, refer to the previous
    // section to learn about the headers that can be set
    // e.g:
    exchange.getIn().setHeader(BlobConstants.BLOB_NAME, "overridenName");
  })
  .to("azure-storage-blob:/camelazure/container1?blobName=blob&operation=deleteBlob&serviceClient=#client")
  .to("mock:result");

  • downloadBlobToFile:

from("direct:start")
  .process(exchange -> {
    // set the header you want the producer to evaluate, refer to the previous
    // section to learn about the headers that can be set
    // e.g:
    exchange.getIn().setHeader(BlobConstants.BLOB_NAME, "overridenName");
  })
  .to("azure-storage-blob:/camelazure/container1?blobName=blob&operation=downloadBlobToFile&fileDir=/var/mydir&serviceClient=#client")
  .to("mock:result");

  • downloadLink

from("direct:start")
  .to("azure-storage-blob:/camelazure/container1?blobName=blob&operation=downloadLink&serviceClient=#client")
  .process(exchange -> {
      String link = exchange.getMessage().getHeader(BlobConstants.DOWNLOAD_LINK, String.class);
      System.out.println("My link " + link);
  })
  .to("mock:result");

  • uploadBlockBlob

from("direct:start")
  .process(exchange -> {
    // set the header you want the producer to evaluate, refer to the previous
    // section to learn about the headers that can be set
    // e.g:
    exchange.getIn().setHeader(BlobConstants.BLOB_NAME, "overridenName");
    exchange.getIn().setBody("Block Blob");
  })
  .to("azure-storage-blob:/camelazure/container1?blobName=blob&operation=uploadBlockBlob&serviceClient=#client")
  .to("mock:result");

  • stageBlockBlobList

from("direct:start")
  .process(exchange -> {
      final List<BlobBlock> blocks = new LinkedList<>();
      blocks.add(BlobBlock.createBlobBlock(new ByteArrayInputStream("Hello".getBytes())));
      blocks.add(BlobBlock.createBlobBlock(new ByteArrayInputStream("From".getBytes())));
      blocks.add(BlobBlock.createBlobBlock(new ByteArrayInputStream("Camel".getBytes())));

      exchange.getIn().setBody(blocks);
  })
  .to("azure-storage-blob:/camelazure/container1?blobName=blob&operation=stageBlockBlobList&serviceClient=#client")
  .to("mock:result");

  • commitBlockBlobList

from("direct:start")
  .process(exchange -> {
      // We assume here you have the knowledge of these blocks you want to commit
      final List<Block> blocksIds = new LinkedList<>();
      blocksIds.add(new Block().setName("id-1"));
      blocksIds.add(new Block().setName("id-2"));
      blocksIds.add(new Block().setName("id-3"));

      exchange.getIn().setBody(blocksIds);
  })
  .to("azure-storage-blob:/camelazure/container1?blobName=blob&operation=commitBlockBlobList&serviceClient=#client")
  .to("mock:result");

  • getBlobBlockList

from("direct:start")
  .to("azure-storage-blob:/camelazure/container1?blobName=blob&operation=getBlobBlockList&serviceClient=#client")
  .log("${body}")
  .to("mock:result");

  • createAppendBlob

from("direct:start")
  .to("azure-storage-blob:/camelazure/container1?blobName=blob&operation=createAppendBlob&serviceClient=#client")
  .to("mock:result");

  • commitAppendBlob

from("direct:start")
  .process(exchange -> {
    final String data = "Hello world from my awesome tests!";
    final InputStream dataStream = new ByteArrayInputStream(data.getBytes(StandardCharsets.UTF_8));

    exchange.getIn().setBody(dataStream);

    // of course you can set whatever headers you like, refer to the headers section to learn more
  })
  .to("azure-storage-blob:/camelazure/container1?blobName=blob&operation=commitAppendBlob&serviceClient=#client")
  .to("mock:result");

  • createPageBlob

from("direct:start")
  .to("azure-storage-blob:/camelazure/container1?blobName=blob&operation=createPageBlob&serviceClient=#client")
  .to("mock:result");

  • uploadPageBlob

from("direct:start")
  .process(exchange -> {
    byte[] dataBytes = new byte[512]; // we set range for the page from 0-511
    new Random().nextBytes(dataBytes);
    final InputStream dataStream = new ByteArrayInputStream(dataBytes);
    final PageRange pageRange = new PageRange().setStart(0).setEnd(511);

    exchange.getIn().setHeader(BlobConstants.PAGE_BLOB_RANGE, pageRange);
    exchange.getIn().setBody(dataStream);
  })
  .to("azure-storage-blob:/camelazure/container1?blobName=blob&operation=uploadPageBlob&serviceClient=#client")
  .to("mock:result");

  • resizePageBlob

from("direct:start")
  .process(exchange -> {
    final PageRange pageRange = new PageRange().setStart(0).setEnd(511);

    exchange.getIn().setHeader(BlobConstants.PAGE_BLOB_RANGE, pageRange);
  })
  .to("azure-storage-blob:/camelazure/container1?blobName=blob&operation=resizePageBlob&serviceClient=#client")
  .to("mock:result");

  • clearPageBlob

from("direct:start")
  .process(exchange -> {
    final PageRange pageRange = new PageRange().setStart(0).setEnd(511);

    exchange.getIn().setHeader(BlobConstants.PAGE_BLOB_RANGE, pageRange);
  })
  .to("azure-storage-blob:/camelazure/container1?blobName=blob&operation=clearPageBlob&serviceClient=#client")
  .to("mock:result");

  • getPageBlobRanges

from("direct:start")
  .process(exchange -> {
    final PageRange pageRange = new PageRange().setStart(0).setEnd(511);

    exchange.getIn().setHeader(BlobConstants.PAGE_BLOB_RANGE, pageRange);
  })
  .to("azure-storage-blob:/camelazure/container1?blobName=blob&operation=getPageBlobRanges&serviceClient=#client")
  .log("${body}")
  .to("mock:result");

Development Notes (Important)

All integration tests use [Testcontainers](https://www.testcontainers.org/) and run by default. Obtaining of Azure accessKey and accountName is needed to be able to run all integration tests using Azure services. In addition to the mocked unit tests you will need to run the integration tests with every change you make or even client upgrade as the Azure client can break things even on minor versions upgrade. To run the integration tests, on this component directory, run the following maven command:

mvn verify -PfullTests -DaccountName=myacc -DaccessKey=mykey

Whereby accountName is your Azure account name and accessKey is the access key being generated from Azure portal.

Spring Boot Auto-Configuration

When using azure-storage-blob with Spring Boot make sure to use the following Maven dependency to have support for auto configuration:

<dependency>
  <groupId>org.apache.camel.springboot</groupId>
  <artifactId>camel-azure-storage-blob-starter</artifactId>
  <version>x.x.x</version>
  <!-- use the same version as your Camel core version -->
</dependency>

The component supports 29 options, which are listed below.

Name Description Default Type

camel.component.azure-storage-blob.access-key

Access key for the associated azure account name to be used for authentication with azure blob services

String

camel.component.azure-storage-blob.auto-discover-client

Setting the autoDiscoverClient mechanism, if true, the component will look for a client instance in the registry automatically otherwise it will skip that checking.

true

Boolean

camel.component.azure-storage-blob.autowired-enabled

Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc.

true

Boolean

camel.component.azure-storage-blob.blob-name

The blob name, to consume specific blob from a container. However on producer, is only required for the operations on the blob level

String

camel.component.azure-storage-blob.blob-offset

Set the blob offset for the upload or download operations, default is 0

0

Long

camel.component.azure-storage-blob.blob-sequence-number

A user-controlled value that you can use to track requests. The value of the sequence number must be between 0 and 263 - 1.The default value is 0.

0

Long

camel.component.azure-storage-blob.blob-type

The blob type in order to initiate the appropriate settings for each blob type

BlobType

camel.component.azure-storage-blob.block-list-type

Specifies which type of blocks to return.

BlockListType

camel.component.azure-storage-blob.bridge-error-handler

Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.

false

Boolean

camel.component.azure-storage-blob.close-stream-after-read

Close the stream after read or keep it open, default is true

true

Boolean

camel.component.azure-storage-blob.close-stream-after-write

Close the stream after write or keep it open, default is true

true

Boolean

camel.component.azure-storage-blob.commit-block-list-later

When is set to true, the staged blocks will not be committed directly.

true

Boolean

camel.component.azure-storage-blob.configuration

The component configurations. The option is a org.apache.camel.component.azure.storage.blob.BlobConfiguration type.

BlobConfiguration

camel.component.azure-storage-blob.create-append-blob

When is set to true, the append blocks will be created when committing append blocks.

true

Boolean

camel.component.azure-storage-blob.create-page-blob

When is set to true, the page blob will be created when uploading page blob.

true

Boolean

camel.component.azure-storage-blob.credentials

StorageSharedKeyCredential can be injected to create the azure client, this holds the important authentication information. The option is a com.azure.storage.common.StorageSharedKeyCredential type.

StorageSharedKeyCredential

camel.component.azure-storage-blob.data-count

How many bytes to include in the range. Must be greater than or equal to 0 if specified.

Long

camel.component.azure-storage-blob.download-link-expiration

Override the default expiration (millis) of URL download link.

Long

camel.component.azure-storage-blob.enabled

Whether to enable auto configuration of the azure-storage-blob component. This is enabled by default.

Boolean

camel.component.azure-storage-blob.file-dir

The file directory where the downloaded blobs will be saved to, this can be used in both, producer and consumer

String

camel.component.azure-storage-blob.lazy-start-producer

Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing.

false

Boolean

camel.component.azure-storage-blob.max-results-per-page

Specifies the maximum number of blobs to return, including all BlobPrefix elements. If the request does not specify maxResultsPerPage or specifies a value greater than 5,000, the server will return up to 5,000 items.

Integer

camel.component.azure-storage-blob.max-retry-requests

Specifies the maximum number of additional HTTP Get requests that will be made while reading the data from a response body.

0

Integer

camel.component.azure-storage-blob.operation

The blob operation that can be used with this component on the producer

BlobOperationsDefinition

camel.component.azure-storage-blob.page-blob-size

Specifies the maximum size for the page blob, up to 8 TB. The page blob size must be aligned to a 512-byte boundary.

512

Long

camel.component.azure-storage-blob.prefix

Filters the results to return only blobs whose names begin with the specified prefix. May be null to return all blobs.

String

camel.component.azure-storage-blob.regex

Filters the results to return only blobs whose names match the specified regular expression. May be null to return all if both prefix and regex are set, regex takes the priority and prefix is ignored.

String

camel.component.azure-storage-blob.service-client

Client to a storage account. This client does not hold any state about a particular storage account but is instead a convenient way of sending off appropriate requests to the resource on the service. It may also be used to construct URLs to blobs and containers. This client contains operations on a service account. Operations on a container are available on BlobContainerClient through BlobServiceClient#getBlobContainerClient(String), and operations on a blob are available on BlobClient through BlobContainerClient#getBlobClient(String). The option is a com.azure.storage.blob.BlobServiceClient type.

BlobServiceClient

camel.component.azure-storage-blob.timeout

An optional timeout value beyond which a RuntimeException will be raised. The option is a java.time.Duration type.

Duration
Back to top