Kafka Source
Provided by: "Apache Software Foundation"
Support Level for this Kamelet is: "Stable"
Receive data from Kafka topics.
Configuration Options
The following table summarizes the configuration options available for the kafka-source
Kamelet:
Property | Name | Description | Type | Default | Example |
---|---|---|---|---|---|
Bootstrap Servers |
Required Comma separated list of Kafka Broker URLs. |
string |
|||
Password |
Required Password to authenticate to kafka. |
string |
|||
Topic Names |
Required Comma separated list of Kafka topic names. |
string |
|||
Username |
Required Username to authenticate to Kafka. |
string |
|||
Allow Manual Commit |
Whether to allow doing manual commits. |
boolean |
false |
||
Auto Commit Enable |
If true, periodically commit to ZooKeeper the offset of messages already fetched by the consumer. |
boolean |
true |
||
Auto Offset Reset |
What to do when there is no initial offset. There are 3 enums and the value can be one of latest, earliest, none. |
string |
latest |
||
Consumer Group |
A string that uniquely identifies the group of consumers to which this source belongs. |
string |
my-group-id |
||
Automatically Deserialize Headers |
When enabled the Kamelet source will deserialize all message headers to String representation. |
boolean |
false |
||
Poll On Error Behavior |
What to do if kafka threw an exception while polling for new messages. There are 5 enums and the value can be one of DISCARD, ERROR_HANDLER, RECONNECT, RETRY, STOP. |
string |
ERROR_HANDLER |
||
SASL Mechanism |
The Simple Authentication and Security Layer (SASL) Mechanism used. |
string |
PLAIN |
||
Security Protocol |
Protocol used to communicate with brokers. SASL_PLAINTEXT, PLAINTEXT, SASL_SSL and SSL are supported. |
string |
SASL_SSL |
Dependencies
At runtime, the kafka-source
Kamelet relies upon the presence of the following dependencies:
-
github:apache.camel-kamelets:camel-kamelets-utils:0.9.4-SNAPSHOT
-
camel:core
-
camel:kafka
-
camel:kamelet
Usage
This section describes how you can use the kafka-source
.
Knative source
You can use the kafka-source
Kamelet as a Knative source by binding it to a Knative object.
apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
name: kafka-source-binding
spec:
source:
ref:
kind: Kamelet
apiVersion: camel.apache.org/v1alpha1
name: kafka-source
properties:
bootstrapServers: The Bootstrap Servers
password: The Password
topic: The Topic Names
user: The Username
sink:
ref:
kind: Channel
apiVersion: messaging.knative.dev/v1
name: mychannel
Prerequisite
You have Camel K installed on the cluster.
Procedure for using the cluster CLI
-
Save the
kafka-source-binding.yaml
file to your local drive, and then edit it as needed for your configuration. -
Run the source by using the following command:
kubectl apply -f kafka-source-binding.yaml
Procedure for using the Kamel CLI
Configure and run the source by using the following command:
kamel bind kafka-source -p "source.bootstrapServers=The Bootstrap Servers" -p "source.password=The Password" -p "source.topic=The Topic Names" -p "source.user=The Username" channel:mychannel
This command creates the KameletBinding in the current namespace on the cluster.
Kafka source
You can use the kafka-source
Kamelet as a Kafka source by binding it to a Kafka topic.
apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
name: kafka-source-binding
spec:
source:
ref:
kind: Kamelet
apiVersion: camel.apache.org/v1alpha1
name: kafka-source
properties:
bootstrapServers: The Bootstrap Servers
password: The Password
topic: The Topic Names
user: The Username
sink:
ref:
kind: KafkaTopic
apiVersion: kafka.strimzi.io/v1beta1
name: my-topic
Prerequisites
-
You’ve installed Strimzi.
-
You’ve created a topic named
my-topic
in the current namespace. -
You have Camel K installed on the cluster.
Procedure for using the cluster CLI
-
Save the
kafka-source-binding.yaml
file to your local drive, and then edit it as needed for your configuration. -
Run the source by using the following command:
kubectl apply -f kafka-source-binding.yaml
Procedure for using the Kamel CLI
Configure and run the source by using the following command:
kamel bind kafka-source -p "source.bootstrapServers=The Bootstrap Servers" -p "source.password=The Password" -p "source.topic=The Topic Names" -p "source.user=The Username" kafka.strimzi.io/v1beta1:KafkaTopic:my-topic
This command creates the KameletBinding in the current namespace on the cluster.