camel-azure-cosmosdb-kafka-connector source configuration
Connector Description: To read and write records to the CosmosDB database on Azure cloud platform.
When using camel-azure-cosmosdb-kafka-connector as source make sure to use the following Maven dependency to have support for the connector:
<dependency>
<groupId>org.apache.camel.kafkaconnector</groupId>
<artifactId>camel-azure-cosmosdb-kafka-connector</artifactId>
<version>x.x.x</version>
<!-- use the same version as your Camel Kafka connector version -->
</dependency>
To use this source connector in Kafka connect you’ll need to set the following connector.class
connector.class=org.apache.camel.kafkaconnector.azurecosmosdb.CamelAzurecosmosdbSourceConnector
The camel-azure-cosmosdb source connector supports 48 options, which are listed below.
Name | Description | Default | Priority |
---|---|---|---|
The name of the Cosmos database that component should connect to. In case you are producing data and have createDatabaseIfNotExists=true, the component will automatically auto create a Cosmos database. |
MEDIUM |
||
The name of the Cosmos container that component should connect to. In case you are producing data and have createContainerIfNotExists=true, the component will automatically auto create a Cosmos container. |
MEDIUM |
||
Sets the flag to enable client telemetry which will periodically collect database operations aggregation statistics, system information like cpu/memory and send it to cosmos monitoring service, which will be helpful during debugging. DEFAULT value is false indicating this is opt in feature, by default no telemetry collection. |
false |
MEDIUM |
|
Enables connections sharing across multiple Cosmos Clients. The default is false. When you have multiple instances of Cosmos Client in the same JVM interacting to multiple Cosmos accounts, enabling this allows connection sharing in Direct mode if possible between instances of Cosmos Client. Please note, when setting this option, the connection configuration (e.g., socket timeout config, idle timeout config) of the first instantiated client will be used for all other client instances. |
false |
MEDIUM |
|
Sets the consistency levels supported for Azure Cosmos DB client operations in the Azure Cosmos DB service. The requested ConsistencyLevel must match or be weaker than that provisioned for the database account. Consistency levels by order of strength are STRONG, BOUNDED_STALENESS, SESSION and EVENTUAL. Refer to consistency level documentation for additional details: https://docs.microsoft.com/en-us/azure/cosmos-db/consistency-levels One of: [Strong] [BoundedStaleness] [Session] [Eventual] [ConsistentPrefix]. Enum values:
|
"SESSION" |
MEDIUM |
|
Sets the container partition key path. |
MEDIUM |
||
Sets the boolean to only return the headers and status code in Cosmos DB response in case of Create, Update and Delete operations on CosmosItem. In Consumer, it is enabled by default because of the ChangeFeed in the consumer that needs this flag to be enabled and thus is shouldn’t be overridden. In Producer, it advised to disable it since it reduces the network overhead. |
true |
MEDIUM |
|
Inject an external CosmosAsyncClient into the component which provides a client-side logical representation of the Azure Cosmos DB service. This asynchronous client is used to configure and execute requests against the service. |
MEDIUM |
||
Sets if the component should create Cosmos container automatically in case it doesn’t exist in Cosmos database. |
false |
MEDIUM |
|
Sets if the component should create Cosmos database automatically in case it doesn’t exist in Cosmos account. |
false |
MEDIUM |
|
Required Sets the Azure Cosmos database endpoint the component will connect to. |
HIGH |
||
Sets the flag to enable writes on any regions for geo-replicated database accounts in the Azure Cosmos DB service. When the value of this property is true, the SDK will direct write operations to available writable regions of geo-replicated database account. Writable regions are ordered by PreferredRegions property. Setting the property value to true has no effect until EnableMultipleWriteRegions in DatabaseAccount is also set to true. DEFAULT value is true indicating that writes are directed to available writable regions of geo-replicated database account. |
true |
MEDIUM |
|
Sets the preferred regions for geo-replicated database accounts. For example, East US as the preferred region. When EnableEndpointDiscovery is true and PreferredRegions is non-empty, the SDK will prefer to use the regions in the container in the order they are specified to perform operations. |
MEDIUM |
||
Sets whether to allow for reads to go to multiple regions configured on an account of Azure Cosmos DB service. DEFAULT value is true. If this property is not set, the default is true for all Consistency Levels other than Bounded Staleness, The default is false for Bounded Staleness. 1. endpointDiscoveryEnabled is true 2. the Azure Cosmos DB account has more than one region. |
true |
MEDIUM |
|
Sets throughput of the resources in the Azure Cosmos DB service. |
MEDIUM |
||
Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. |
false |
MEDIUM |
|
Sets the ChangeFeedProcessorOptions to be used. Unless specifically set the default values that will be used are: maximum items per page or FeedResponse: 100 lease renew interval: 17 seconds lease acquire interval: 13 seconds lease expiration interval: 60 seconds feed poll delay: 5 seconds maximum scale count: unlimited. |
MEDIUM |
||
Sets if the component should create Cosmos lease container for the consumer automatically in case it doesn’t exist in Cosmos database. |
false |
MEDIUM |
|
Sets if the component should create Cosmos lease database for the consumer automatically in case it doesn’t exist in Cosmos account. |
false |
MEDIUM |
|
Sets the hostname. The host: a host is an application instance that uses the change feed processor to listen for changes. Multiple instances with the same lease configuration can run in parallel, but each instance should have a different instance name. If not specified, this will be a generated random hostname. |
MEDIUM |
||
Sets the lease container which acts as a state storage and coordinates processing the change feed across multiple workers. The lease container can be stored in the same account as the monitored container or in a separate account. It will be auto created if createLeaseContainerIfNotExists is set to true. |
"camel-lease" |
MEDIUM |
|
Sets the lease database where the leaseContainerName will be stored. If it is not specified, this component will store the lease container in the same database that is specified in databaseName. It will be auto created if createLeaseDatabaseIfNotExists is set to true. |
MEDIUM |
||
To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this option is not in use. By default the consumer will deal with exceptions, that will be logged at WARN or ERROR level and ignored. |
MEDIUM |
||
Sets the exchange pattern when the consumer creates an exchange. One of: [InOnly] [InOut] [InOptionalOut]. Enum values:
|
MEDIUM |
||
Required Sets either a master or readonly key used to perform authentication for accessing resource. |
HIGH |
||
Sets the flag to enable client telemetry which will periodically collect database operations aggregation statistics, system information like cpu/memory and send it to cosmos monitoring service, which will be helpful during debugging. DEFAULT value is false indicating this is opt in feature, by default no telemetry collection. |
false |
MEDIUM |
|
The component configurations. |
MEDIUM |
||
camel.component.azure-cosmosdb.connectionSharingAcrossClientsEnabled |
Enables connections sharing across multiple Cosmos Clients. The default is false. When you have multiple instances of Cosmos Client in the same JVM interacting to multiple Cosmos accounts, enabling this allows connection sharing in Direct mode if possible between instances of Cosmos Client. Please note, when setting this option, the connection configuration (e.g., socket timeout config, idle timeout config) of the first instantiated client will be used for all other client instances. |
false |
MEDIUM |
Sets the consistency levels supported for Azure Cosmos DB client operations in the Azure Cosmos DB service. The requested ConsistencyLevel must match or be weaker than that provisioned for the database account. Consistency levels by order of strength are STRONG, BOUNDED_STALENESS, SESSION and EVENTUAL. Refer to consistency level documentation for additional details: https://docs.microsoft.com/en-us/azure/cosmos-db/consistency-levels One of: [Strong] [BoundedStaleness] [Session] [Eventual] [ConsistentPrefix]. Enum values:
|
"SESSION" |
MEDIUM |
|
Sets the container partition key path. |
MEDIUM |
||
camel.component.azure-cosmosdb.contentResponseOnWriteEnabled |
Sets the boolean to only return the headers and status code in Cosmos DB response in case of Create, Update and Delete operations on CosmosItem. In Consumer, it is enabled by default because of the ChangeFeed in the consumer that needs this flag to be enabled and thus is shouldn’t be overridden. In Producer, it advised to disable it since it reduces the network overhead. |
true |
MEDIUM |
Inject an external CosmosAsyncClient into the component which provides a client-side logical representation of the Azure Cosmos DB service. This asynchronous client is used to configure and execute requests against the service. |
MEDIUM |
||
Sets if the component should create Cosmos container automatically in case it doesn’t exist in Cosmos database. |
false |
MEDIUM |
|
Sets if the component should create Cosmos database automatically in case it doesn’t exist in Cosmos account. |
false |
MEDIUM |
|
Required Sets the Azure Cosmos database endpoint the component will connect to. |
HIGH |
||
Sets the flag to enable writes on any regions for geo-replicated database accounts in the Azure Cosmos DB service. When the value of this property is true, the SDK will direct write operations to available writable regions of geo-replicated database account. Writable regions are ordered by PreferredRegions property. Setting the property value to true has no effect until EnableMultipleWriteRegions in DatabaseAccount is also set to true. DEFAULT value is true indicating that writes are directed to available writable regions of geo-replicated database account. |
true |
MEDIUM |
|
Sets the preferred regions for geo-replicated database accounts. For example, East US as the preferred region. When EnableEndpointDiscovery is true and PreferredRegions is non-empty, the SDK will prefer to use the regions in the container in the order they are specified to perform operations. |
MEDIUM |
||
Sets whether to allow for reads to go to multiple regions configured on an account of Azure Cosmos DB service. DEFAULT value is true. If this property is not set, the default is true for all Consistency Levels other than Bounded Staleness, The default is false for Bounded Staleness. 1. endpointDiscoveryEnabled is true 2. the Azure Cosmos DB account has more than one region. |
true |
MEDIUM |
|
Sets throughput of the resources in the Azure Cosmos DB service. |
MEDIUM |
||
Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. |
false |
MEDIUM |
|
Sets the ChangeFeedProcessorOptions to be used. Unless specifically set the default values that will be used are: maximum items per page or FeedResponse: 100 lease renew interval: 17 seconds lease acquire interval: 13 seconds lease expiration interval: 60 seconds feed poll delay: 5 seconds maximum scale count: unlimited. |
MEDIUM |
||
camel.component.azure-cosmosdb.createLeaseContainerIfNotExists |
Sets if the component should create Cosmos lease container for the consumer automatically in case it doesn’t exist in Cosmos database. |
false |
MEDIUM |
camel.component.azure-cosmosdb.createLeaseDatabaseIfNotExists |
Sets if the component should create Cosmos lease database for the consumer automatically in case it doesn’t exist in Cosmos account. |
false |
MEDIUM |
Sets the hostname. The host: a host is an application instance that uses the change feed processor to listen for changes. Multiple instances with the same lease configuration can run in parallel, but each instance should have a different instance name. If not specified, this will be a generated random hostname. |
MEDIUM |
||
Sets the lease container which acts as a state storage and coordinates processing the change feed across multiple workers. The lease container can be stored in the same account as the monitored container or in a separate account. It will be auto created if createLeaseContainerIfNotExists is set to true. |
"camel-lease" |
MEDIUM |
|
Sets the lease database where the leaseContainerName will be stored. If it is not specified, this component will store the lease container in the same database that is specified in databaseName. It will be auto created if createLeaseDatabaseIfNotExists is set to true. |
MEDIUM |
||
Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc. |
true |
MEDIUM |
|
Required Sets either a master or readonly key used to perform authentication for accessing resource. |
HIGH |
The camel-azure-cosmosdb source connector has no converters out of the box.
The camel-azure-cosmosdb source connector has no transforms out of the box.
The camel-azure-cosmosdb source connector has no aggregation strategies out of the box.