camel-aws-s3-streaming-upload-sink-kafka-connector sink configuration

Connector Description: Upload data to AWS S3 in streaming upload mode.

When using camel-aws-s3-streaming-upload-sink-kafka-connector as sink make sure to use the following Maven dependency to have support for the connector:

<dependency>
  <groupId>org.apache.camel.kafkaconnector</groupId>
  <artifactId>camel-aws-s3-streaming-upload-sink-kafka-connector</artifactId>
  <version>x.x.x</version>
  <!-- use the same version as your Camel Kafka connector version -->
</dependency>

To use this sink connector in Kafka connect you’ll need to set the following connector.class

connector.class=org.apache.camel.kafkaconnector.awss3streaminguploadsink.CamelAwss3streaminguploadsinkSinkConnector

The camel-aws-s3-streaming-upload-sink sink connector supports 12 options, which are listed below.

Name Description Default Priority

camel.kamelet.aws-s3-streaming-upload-sink.bucketNameOrArn

Required The S3 Bucket name or ARN.

HIGH

camel.kamelet.aws-s3-streaming-upload-sink.accessKey

Required The access key obtained from AWS.

HIGH

camel.kamelet.aws-s3-streaming-upload-sink.secretKey

Required The secret key obtained from AWS.

HIGH

camel.kamelet.aws-s3-streaming-upload-sink.region

Required The AWS region to connect to. Example: eu-west-1.

HIGH

camel.kamelet.aws-s3-streaming-upload-sink.autoCreateBucket

Setting the autocreation of the S3 bucket bucketName.

false

MEDIUM

camel.kamelet.aws-s3-streaming-upload-sink.streamingUploadMode

Setting the Streaming Upload Mode.

true

MEDIUM

camel.kamelet.aws-s3-streaming-upload-sink.restartingPolicy

The restarting policy to use in streaming upload mode. There are 2 enums and the value can be one of override, lastPart.

"lastPart"

MEDIUM

camel.kamelet.aws-s3-streaming-upload-sink.batchMessageNumber

The number of messages composing a batch in streaming upload mode.

10

MEDIUM

camel.kamelet.aws-s3-streaming-upload-sink.batchSize

The batch size (in bytes) in streaming upload mode.

1000000

MEDIUM

camel.kamelet.aws-s3-streaming-upload-sink.streamingUploadTimeout

While streaming upload mode is true, this option set the timeout to complete upload.

MEDIUM

camel.kamelet.aws-s3-streaming-upload-sink.namingStrategy

The naming strategy to use in streaming upload mode. There are 2 enums and the value can be one of progressive, random.

"progressive"

MEDIUM

camel.kamelet.aws-s3-streaming-upload-sink.keyName

Required Setting the key name for an element in the bucket through endpoint parameter. In Streaming Upload, with the default configuration, this will be the base for the progressive creation of files.

HIGH

The camel-aws-s3-streaming-upload-sink sink connector has no converters out of the box.

The camel-aws-s3-streaming-upload-sink sink connector has no transforms out of the box.

The camel-aws-s3-streaming-upload-sink sink connector has no aggregation strategies out of the box.

Back to top