Kafka binding spec

Detailed documentation on the Kafka binding component

Component format

To setup Kafka binding create a component of type bindings.kafka. See this guide on how to create and apply a binding configuration. For details on using secretKeyRef, see the guide on how to reference secrets in components.

  1. apiVersion: dapr.io/v1alpha1
  2. kind: Component
  3. metadata:
  4.   name: kafka-binding
  5.   namespace: default
  6. spec:
  7.   type: bindings.kafka
  8.   version: v1
  9.   metadata:
  10.   - name: topics # Optional. Used for input bindings.
  11.     value: "topic1,topic2"
  12.   - name: brokers # Required.
  13.     value: "localhost:9092,localhost:9093"
  14.   - name: consumerGroup # Optional. Used for input bindings.
  15.     value: "group1"
  16.   - name: publishTopic # Optional. Used for output bindings.
  17.     value: "topic3"
  18.   - name: authRequired # Required.
  19.     value: "true"
  20.   - name: saslUsername # Required if authRequired is `true`.
  21.     value: "user"
  22.   - name: saslPassword # Required if authRequired is `true`.
  23.     secretKeyRef:
  24.       name: kafka-secrets
  25.       key: saslPasswordSecret
  26.   - name: maxMessageBytes # Optional.
  27.     value: 1024

Spec metadata fields

FieldRequiredBinding supportDetailsExample
topicsNInputA comma-separated string of topics.“mytopic1,topic2”
brokersYInput/OutputA comma-separated string of Kafka brokers.“localhost:9092,dapr-kafka.myapp.svc.cluster.local:9093”
consumerGroupNInputA kafka consumer group to listen on. Each record published to a topic is delivered to one consumer within each consumer group subscribed to the topic.“group1”
publishTopicYOutputThe topic to publish to.“mytopic”
authRequiredYInput/OutputEnable SASL authentication with the Kafka brokers.“true”, “false”
saslUsernameNInput/OutputThe SASL username used for authentication. Only required if authRequired is set to “true”.“adminuser”
saslPasswordNInput/OutputThe SASL password used for authentication. Can be secretKeyRef to use a secret reference. Only required if authRequired is set to “true”.“”, “KeFg23!”
maxMessageBytesNInput/OutputThe maximum size in bytes allowed for a single Kafka message. Defaults to 1024.2048

Binding support

This component supports both input and output binding interfaces.

This component supports output binding with the following operations:

  • create

Specifying a partition key

When invoking the Kafka binding, its possible to provide an optional partition key by using the metadata section in the request body.

The field name is partitionKey.

Example:

  1. curl -X POST http://localhost:3500/v1.0/bindings/myKafka \
  2.   -H "Content-Type: application/json" \
  3.   -d '{
  4.         "data": {
  5.           "message": "Hi"
  6.         },
  7.         "metadata": {
  8.           "partitionKey": "key1"
  9.         },
  10.         "operation": "create"
  11.       }'

Last modified September 17, 2021 : Merge pull request #1757 from georgestevens99/1440SecretKeyRefExplanation (620a5f8)