Google Storage CDC Source
Provided by: "Apache Software Foundation"
Support Level for this Kamelet is: "Preview"
Receive data from Google Pubsub reporting events related to a Google Storage bucket.
Service account key is the basic method for authenticating to the Google Pubsub and Google Storage.
To use this Kamelet you’ll need to set up Events notification for your Google Storage Bucket.
You must encode the serviceAccountKey
property before you pass it as a parameter.
Configuration Options
The following table summarizes the configuration options available for the google-storage-cdc-source
Kamelet:
Property | Name | Description | Type | Default | Example |
---|---|---|---|---|---|
Bucket Name Or ARN |
Required The Google Cloud Storage bucket name or Bucket Amazon Resource Name (ARN). |
string |
|||
Project Id |
Required The Google Cloud Pub/Sub Project ID. |
string |
|||
Service Account Key |
Required The service account key to use as credentials for the Pub/Sub publisher/subscriber. You must encode this value in base64. |
binary |
|||
Subscription Name |
Required The subscription name. |
string |
|||
Concurrent Consumers |
The number of parallel streams to consume from the subscription. |
integer |
1 |
||
Get Object in Bucket |
If getObject is enabled, then the file created in the Bucket will be get and returned as body, if not only the event will be returned as body. |
boolean |
false |
||
Max Messages Per Poll |
The maximum number of messages to receive from the server in a single API call. |
integer |
1 |
||
Synchronous Pull |
Specifies to synchronously pull batches of messages. |
boolean |
false |
Dependencies
At runtime, the google-storage-cdc-source
Kamelet relies upon the presence of the following dependencies:
-
camel:kamelet
-
camel:google-pubsub
-
camel:google-storage
-
camel:jackson
-
camel:core
Usage
This section describes how you can use the google-storage-cdc-source
.
Knative source
You can use the google-storage-cdc-source
Kamelet as a Knative source by binding it to a Knative object.
apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
name: google-storage-cdc-source-binding
spec:
source:
ref:
kind: Kamelet
apiVersion: camel.apache.org/v1alpha1
name: google-storage-cdc-source
properties:
bucketNameOrArn: The Bucket Name Or ARN
projectId: The Project Id
serviceAccountKey: The Service Account Key
subscriptionName: The Subscription Name
sink:
ref:
kind: Channel
apiVersion: messaging.knative.dev/v1
name: mychannel
Prerequisite
You have Camel K installed on the cluster.
Procedure for using the cluster CLI
-
Save the
google-storage-cdc-source-binding.yaml
file to your local drive, and then edit it as needed for your configuration. -
Run the source by using the following command:
kubectl apply -f google-storage-cdc-source-binding.yaml
Procedure for using the Kamel CLI
Configure and run the source by using the following command:
kamel bind google-storage-cdc-source -p "source.bucketNameOrArn=The Bucket Name Or ARN" -p "source.projectId=The Project Id" -p "source.serviceAccountKey=The Service Account Key" -p "source.subscriptionName=The Subscription Name" channel:mychannel
This command creates the KameletBinding in the current namespace on the cluster.
Kafka source
You can use the google-storage-cdc-source
Kamelet as a Kafka source by binding it to a Kafka topic.
apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
name: google-storage-cdc-source-binding
spec:
source:
ref:
kind: Kamelet
apiVersion: camel.apache.org/v1alpha1
name: google-storage-cdc-source
properties:
bucketNameOrArn: The Bucket Name Or ARN
projectId: The Project Id
serviceAccountKey: The Service Account Key
subscriptionName: The Subscription Name
sink:
ref:
kind: KafkaTopic
apiVersion: kafka.strimzi.io/v1beta1
name: my-topic
Prerequisites
-
You’ve installed Strimzi.
-
You’ve created a topic named
my-topic
in the current namespace. -
You have Camel K installed on the cluster.
Procedure for using the cluster CLI
-
Save the
google-storage-cdc-source-binding.yaml
file to your local drive, and then edit it as needed for your configuration. -
Run the source by using the following command:
kubectl apply -f google-storage-cdc-source-binding.yaml
Procedure for using the Kamel CLI
Configure and run the source by using the following command:
kamel bind google-storage-cdc-source -p "source.bucketNameOrArn=The Bucket Name Or ARN" -p "source.projectId=The Project Id" -p "source.serviceAccountKey=The Service Account Key" -p "source.subscriptionName=The Subscription Name" kafka.strimzi.io/v1beta1:KafkaTopic:my-topic
This command creates the KameletBinding in the current namespace on the cluster.