camel-azure-cosmosdb-kafka-connector source configuration

Connector Description: To read and write records to the CosmosDB database on Azure cloud platform.

When using camel-azure-cosmosdb-kafka-connector as source make sure to use the following Maven dependency to have support for the connector:

<dependency>
  <groupId>org.apache.camel.kafkaconnector</groupId>
  <artifactId>camel-azure-cosmosdb-kafka-connector</artifactId>
  <version>x.x.x</version>
  <!-- use the same version as your Camel Kafka connector version -->
</dependency>

To use this source connector in Kafka connect you’ll need to set the following connector.class

connector.class=org.apache.camel.kafkaconnector.azurecosmosdb.CamelAzurecosmosdbSourceConnector

The camel-azure-cosmosdb source connector supports 48 options, which are listed below.

Name Description Default Priority

camel.source.path.databaseName

The name of the Cosmos database that component should connect to. In case you are producing data and have createDatabaseIfNotExists=true, the component will automatically auto create a Cosmos database.

MEDIUM

camel.source.path.containerName

The name of the Cosmos container that component should connect to. In case you are producing data and have createContainerIfNotExists=true, the component will automatically auto create a Cosmos container.

MEDIUM

camel.source.endpoint.clientTelemetryEnabled

Sets the flag to enable client telemetry which will periodically collect database operations aggregation statistics, system information like cpu/memory and send it to cosmos monitoring service, which will be helpful during debugging. DEFAULT value is false indicating this is opt in feature, by default no telemetry collection.

false

MEDIUM

camel.source.endpoint.connectionSharingAcrossClientsEnabled

Enables connections sharing across multiple Cosmos Clients. The default is false. When you have multiple instances of Cosmos Client in the same JVM interacting to multiple Cosmos accounts, enabling this allows connection sharing in Direct mode if possible between instances of Cosmos Client. Please note, when setting this option, the connection configuration (e.g., socket timeout config, idle timeout config) of the first instantiated client will be used for all other client instances.

false

MEDIUM

camel.source.endpoint.consistencyLevel

Sets the consistency levels supported for Azure Cosmos DB client operations in the Azure Cosmos DB service. The requested ConsistencyLevel must match or be weaker than that provisioned for the database account. Consistency levels by order of strength are STRONG, BOUNDED_STALENESS, SESSION and EVENTUAL. Refer to consistency level documentation for additional details: https://docs.microsoft.com/en-us/azure/cosmos-db/consistency-levels One of: [Strong] [BoundedStaleness] [Session] [Eventual] [ConsistentPrefix].

Enum values:

  • Strong

  • BoundedStaleness

  • Session

  • Eventual

  • ConsistentPrefix

"SESSION"

MEDIUM

camel.source.endpoint.containerPartitionKeyPath

Sets the container partition key path.

MEDIUM

camel.source.endpoint.contentResponseOnWriteEnabled

Sets the boolean to only return the headers and status code in Cosmos DB response in case of Create, Update and Delete operations on CosmosItem. In Consumer, it is enabled by default because of the ChangeFeed in the consumer that needs this flag to be enabled and thus is shouldn’t be overridden. In Producer, it advised to disable it since it reduces the network overhead.

true

MEDIUM

camel.source.endpoint.cosmosAsyncClient

Inject an external CosmosAsyncClient into the component which provides a client-side logical representation of the Azure Cosmos DB service. This asynchronous client is used to configure and execute requests against the service.

MEDIUM

camel.source.endpoint.createContainerIfNotExists

Sets if the component should create Cosmos container automatically in case it doesn’t exist in Cosmos database.

false

MEDIUM

camel.source.endpoint.createDatabaseIfNotExists

Sets if the component should create Cosmos database automatically in case it doesn’t exist in Cosmos account.

false

MEDIUM

camel.source.endpoint.databaseEndpoint

Required Sets the Azure Cosmos database endpoint the component will connect to.

HIGH

camel.source.endpoint.multipleWriteRegionsEnabled

Sets the flag to enable writes on any regions for geo-replicated database accounts in the Azure Cosmos DB service. When the value of this property is true, the SDK will direct write operations to available writable regions of geo-replicated database account. Writable regions are ordered by PreferredRegions property. Setting the property value to true has no effect until EnableMultipleWriteRegions in DatabaseAccount is also set to true. DEFAULT value is true indicating that writes are directed to available writable regions of geo-replicated database account.

true

MEDIUM

camel.source.endpoint.preferredRegions

Sets the preferred regions for geo-replicated database accounts. For example, East US as the preferred region. When EnableEndpointDiscovery is true and PreferredRegions is non-empty, the SDK will prefer to use the regions in the container in the order they are specified to perform operations.

MEDIUM

camel.source.endpoint.readRequestsFallbackEnabled

Sets whether to allow for reads to go to multiple regions configured on an account of Azure Cosmos DB service. DEFAULT value is true. If this property is not set, the default is true for all Consistency Levels other than Bounded Staleness, The default is false for Bounded Staleness. 1. endpointDiscoveryEnabled is true 2. the Azure Cosmos DB account has more than one region.

true

MEDIUM

camel.source.endpoint.throughputProperties

Sets throughput of the resources in the Azure Cosmos DB service.

MEDIUM

camel.source.endpoint.bridgeErrorHandler

Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.

false

MEDIUM

camel.source.endpoint.changeFeedProcessorOptions

Sets the ChangeFeedProcessorOptions to be used. Unless specifically set the default values that will be used are: maximum items per page or FeedResponse: 100 lease renew interval: 17 seconds lease acquire interval: 13 seconds lease expiration interval: 60 seconds feed poll delay: 5 seconds maximum scale count: unlimited.

MEDIUM

camel.source.endpoint.createLeaseContainerIfNotExists

Sets if the component should create Cosmos lease container for the consumer automatically in case it doesn’t exist in Cosmos database.

false

MEDIUM

camel.source.endpoint.createLeaseDatabaseIfNotExists

Sets if the component should create Cosmos lease database for the consumer automatically in case it doesn’t exist in Cosmos account.

false

MEDIUM

camel.source.endpoint.hostName

Sets the hostname. The host: a host is an application instance that uses the change feed processor to listen for changes. Multiple instances with the same lease configuration can run in parallel, but each instance should have a different instance name. If not specified, this will be a generated random hostname.

MEDIUM

camel.source.endpoint.leaseContainerName

Sets the lease container which acts as a state storage and coordinates processing the change feed across multiple workers. The lease container can be stored in the same account as the monitored container or in a separate account. It will be auto created if createLeaseContainerIfNotExists is set to true.

"camel-lease"

MEDIUM

camel.source.endpoint.leaseDatabaseName

Sets the lease database where the leaseContainerName will be stored. If it is not specified, this component will store the lease container in the same database that is specified in databaseName. It will be auto created if createLeaseDatabaseIfNotExists is set to true.

MEDIUM

camel.source.endpoint.exceptionHandler

To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this option is not in use. By default the consumer will deal with exceptions, that will be logged at WARN or ERROR level and ignored.

MEDIUM

camel.source.endpoint.exchangePattern

Sets the exchange pattern when the consumer creates an exchange. One of: [InOnly] [InOut] [InOptionalOut].

Enum values:

  • InOnly

  • InOut

  • InOptionalOut

MEDIUM

camel.source.endpoint.accountKey

Required Sets either a master or readonly key used to perform authentication for accessing resource.

HIGH

camel.component.azure-cosmosdb.clientTelemetryEnabled

Sets the flag to enable client telemetry which will periodically collect database operations aggregation statistics, system information like cpu/memory and send it to cosmos monitoring service, which will be helpful during debugging. DEFAULT value is false indicating this is opt in feature, by default no telemetry collection.

false

MEDIUM

camel.component.azure-cosmosdb.configuration

The component configurations.

MEDIUM

camel.component.azure-cosmosdb.connectionSharingAcrossClientsEnabled

Enables connections sharing across multiple Cosmos Clients. The default is false. When you have multiple instances of Cosmos Client in the same JVM interacting to multiple Cosmos accounts, enabling this allows connection sharing in Direct mode if possible between instances of Cosmos Client. Please note, when setting this option, the connection configuration (e.g., socket timeout config, idle timeout config) of the first instantiated client will be used for all other client instances.

false

MEDIUM

camel.component.azure-cosmosdb.consistencyLevel

Sets the consistency levels supported for Azure Cosmos DB client operations in the Azure Cosmos DB service. The requested ConsistencyLevel must match or be weaker than that provisioned for the database account. Consistency levels by order of strength are STRONG, BOUNDED_STALENESS, SESSION and EVENTUAL. Refer to consistency level documentation for additional details: https://docs.microsoft.com/en-us/azure/cosmos-db/consistency-levels One of: [Strong] [BoundedStaleness] [Session] [Eventual] [ConsistentPrefix].

Enum values:

  • Strong

  • BoundedStaleness

  • Session

  • Eventual

  • ConsistentPrefix

"SESSION"

MEDIUM

camel.component.azure-cosmosdb.containerPartitionKeyPath

Sets the container partition key path.

MEDIUM

camel.component.azure-cosmosdb.contentResponseOnWriteEnabled

Sets the boolean to only return the headers and status code in Cosmos DB response in case of Create, Update and Delete operations on CosmosItem. In Consumer, it is enabled by default because of the ChangeFeed in the consumer that needs this flag to be enabled and thus is shouldn’t be overridden. In Producer, it advised to disable it since it reduces the network overhead.

true

MEDIUM

camel.component.azure-cosmosdb.cosmosAsyncClient

Inject an external CosmosAsyncClient into the component which provides a client-side logical representation of the Azure Cosmos DB service. This asynchronous client is used to configure and execute requests against the service.

MEDIUM

camel.component.azure-cosmosdb.createContainerIfNotExists

Sets if the component should create Cosmos container automatically in case it doesn’t exist in Cosmos database.

false

MEDIUM

camel.component.azure-cosmosdb.createDatabaseIfNotExists

Sets if the component should create Cosmos database automatically in case it doesn’t exist in Cosmos account.

false

MEDIUM

camel.component.azure-cosmosdb.databaseEndpoint

Required Sets the Azure Cosmos database endpoint the component will connect to.

HIGH

camel.component.azure-cosmosdb.multipleWriteRegionsEnabled

Sets the flag to enable writes on any regions for geo-replicated database accounts in the Azure Cosmos DB service. When the value of this property is true, the SDK will direct write operations to available writable regions of geo-replicated database account. Writable regions are ordered by PreferredRegions property. Setting the property value to true has no effect until EnableMultipleWriteRegions in DatabaseAccount is also set to true. DEFAULT value is true indicating that writes are directed to available writable regions of geo-replicated database account.

true

MEDIUM

camel.component.azure-cosmosdb.preferredRegions

Sets the preferred regions for geo-replicated database accounts. For example, East US as the preferred region. When EnableEndpointDiscovery is true and PreferredRegions is non-empty, the SDK will prefer to use the regions in the container in the order they are specified to perform operations.

MEDIUM

camel.component.azure-cosmosdb.readRequestsFallbackEnabled

Sets whether to allow for reads to go to multiple regions configured on an account of Azure Cosmos DB service. DEFAULT value is true. If this property is not set, the default is true for all Consistency Levels other than Bounded Staleness, The default is false for Bounded Staleness. 1. endpointDiscoveryEnabled is true 2. the Azure Cosmos DB account has more than one region.

true

MEDIUM

camel.component.azure-cosmosdb.throughputProperties

Sets throughput of the resources in the Azure Cosmos DB service.

MEDIUM

camel.component.azure-cosmosdb.bridgeErrorHandler

Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.

false

MEDIUM

camel.component.azure-cosmosdb.changeFeedProcessorOptions

Sets the ChangeFeedProcessorOptions to be used. Unless specifically set the default values that will be used are: maximum items per page or FeedResponse: 100 lease renew interval: 17 seconds lease acquire interval: 13 seconds lease expiration interval: 60 seconds feed poll delay: 5 seconds maximum scale count: unlimited.

MEDIUM

camel.component.azure-cosmosdb.createLeaseContainerIfNotExists

Sets if the component should create Cosmos lease container for the consumer automatically in case it doesn’t exist in Cosmos database.

false

MEDIUM

camel.component.azure-cosmosdb.createLeaseDatabaseIfNotExists

Sets if the component should create Cosmos lease database for the consumer automatically in case it doesn’t exist in Cosmos account.

false

MEDIUM

camel.component.azure-cosmosdb.hostName

Sets the hostname. The host: a host is an application instance that uses the change feed processor to listen for changes. Multiple instances with the same lease configuration can run in parallel, but each instance should have a different instance name. If not specified, this will be a generated random hostname.

MEDIUM

camel.component.azure-cosmosdb.leaseContainerName

Sets the lease container which acts as a state storage and coordinates processing the change feed across multiple workers. The lease container can be stored in the same account as the monitored container or in a separate account. It will be auto created if createLeaseContainerIfNotExists is set to true.

"camel-lease"

MEDIUM

camel.component.azure-cosmosdb.leaseDatabaseName

Sets the lease database where the leaseContainerName will be stored. If it is not specified, this component will store the lease container in the same database that is specified in databaseName. It will be auto created if createLeaseDatabaseIfNotExists is set to true.

MEDIUM

camel.component.azure-cosmosdb.autowiredEnabled

Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc.

true

MEDIUM

camel.component.azure-cosmosdb.accountKey

Required Sets either a master or readonly key used to perform authentication for accessing resource.

HIGH

The camel-azure-cosmosdb source connector has no converters out of the box.

The camel-azure-cosmosdb source connector has no transforms out of the box.

The camel-azure-cosmosdb source connector has no aggregation strategies out of the box.