azure storage blob cdc source Azure Storage Blob CDC Source

Provided by: "Apache Software Foundation"

Support Level for this Kamelet is: "Preview"

Receive data from Azure Service Bus subscribed to Azure Eventgrid reporting events related to a Azure Storage Blob account.

Connection string is the basic method for authenticating to the Azure Servicebus Queue.

To use this Kamelet you’ll need to set up Events on your Azure Storage Blob account and select as an endpoint an Azure Servicebus Queue.

Configuration Options

The following table summarizes the configuration options available for the azure-storage-blob-cdc-source Kamelet:

Property Name Description Type Default Example

accessKey

Access Key

Required The Azure Storage Blob access key.

string

accountName

Account Name

Required The Azure Storage Blob account name.

string

connectionString

Connection String

Required Connection String for Azure Servicebus instance.

string

containerName

Container Name

Required The Azure Storage Blob container name.

string

topicOrQueueName

Topic Or Queue Name

Required Topic Or Queue Name for the Azure Servicebus instance.

string

credentialType

Credential Type

Determines the credential strategy to adopt. Possible values are SHARED_ACCOUNT_KEY, SHARED_KEY_CREDENTIAL and AZURE_IDENTITY.

string

SHARED_ACCOUNT_KEY

getBlob

Get Object in Container

If getBlob is enabled, then the file created in the container will be get and returned as body, if not only the event will be returned as body.

boolean

false

serviceBusReceiveMode

Servicebus Receive Mode

Sets the receive mode for the receiver, possible values are PEEK_LOCK and RECEIVE_AND_DELETE.

string

RECEIVE_AND_DELETE

subscriptionName

Subscription Name

Sets the name of the subscription in the topic to listen to. This parameter is mandatory in case of topic.

string

Dependencies

At runtime, the azure-storage-blob-cdc-source Kamelet relies upon the presence of the following dependencies:

  • camel:azure-servicebus

  • camel:azure-storage-blob

  • camel:kamelet

  • camel:core

  • camel:jsonpath

  • camel:jackson

Usage

This section describes how you can use the azure-storage-blob-cdc-source.

Knative source

You can use the azure-storage-blob-cdc-source Kamelet as a Knative source by binding it to a Knative object.

azure-storage-blob-cdc-source-binding.yaml
apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
  name: azure-storage-blob-cdc-source-binding
spec:
  source:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1alpha1
      name: azure-storage-blob-cdc-source
    properties:
      accessKey: The Access Key
      accountName: The Account Name
      connectionString: The Connection String
      containerName: The Container Name
      topicOrQueueName: The Topic Or Queue Name
  sink:
    ref:
      kind: Channel
      apiVersion: messaging.knative.dev/v1
      name: mychannel

Prerequisite

You have Camel K installed on the cluster.

Procedure for using the cluster CLI

  1. Save the azure-storage-blob-cdc-source-binding.yaml file to your local drive, and then edit it as needed for your configuration.

  2. Run the source by using the following command:

    kubectl apply -f azure-storage-blob-cdc-source-binding.yaml

Procedure for using the Kamel CLI

Configure and run the source by using the following command:

kamel bind azure-storage-blob-cdc-source -p "source.accessKey=The Access Key" -p "source.accountName=The Account Name" -p "source.connectionString=The Connection String" -p "source.containerName=The Container Name" -p "source.topicOrQueueName=The Topic Or Queue Name" channel:mychannel

This command creates the KameletBinding in the current namespace on the cluster.

Kafka source

You can use the azure-storage-blob-cdc-source Kamelet as a Kafka source by binding it to a Kafka topic.

azure-storage-blob-cdc-source-binding.yaml
apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
  name: azure-storage-blob-cdc-source-binding
spec:
  source:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1alpha1
      name: azure-storage-blob-cdc-source
    properties:
      accessKey: The Access Key
      accountName: The Account Name
      connectionString: The Connection String
      containerName: The Container Name
      topicOrQueueName: The Topic Or Queue Name
  sink:
    ref:
      kind: KafkaTopic
      apiVersion: kafka.strimzi.io/v1beta1
      name: my-topic

Prerequisites

  • You’ve installed Strimzi.

  • You’ve created a topic named my-topic in the current namespace.

  • You have Camel K installed on the cluster.

Procedure for using the cluster CLI

  1. Save the azure-storage-blob-cdc-source-binding.yaml file to your local drive, and then edit it as needed for your configuration.

  2. Run the source by using the following command:

    kubectl apply -f azure-storage-blob-cdc-source-binding.yaml

Procedure for using the Kamel CLI

Configure and run the source by using the following command:

kamel bind azure-storage-blob-cdc-source -p "source.accessKey=The Access Key" -p "source.accountName=The Account Name" -p "source.connectionString=The Connection String" -p "source.containerName=The Container Name" -p "source.topicOrQueueName=The Topic Or Queue Name" kafka.strimzi.io/v1beta1:KafkaTopic:my-topic

This command creates the KameletBinding in the current namespace on the cluster.

Kamelet source file

https://github.com/apache/camel-kamelets/blob/main/kamelets/azure-storage-blob-cdc-source.kamelet.yaml

Back to top