Kafka binding spec

Detailed documentation on the Kafka binding component

Component format

To setup Kafka binding create a component of type bindings.kafka. See this guide on how to create and apply a binding configuration. For details on using secretKeyRef, see the guide on how to reference secrets in components.

  1. apiVersion: dapr.io/v1alpha1
  2. kind: Component
  3. metadata:
  4.   name: kafka-binding
  5.   namespace: default
  6. spec:
  7.   type: bindings.kafka
  8.   version: v1
  9.   metadata:
  10.   - name: topics # Optional. Used for input bindings.
  11.     value: "topic1,topic2"
  12.   - name: brokers # Required.
  13.     value: "localhost:9092,localhost:9093"
  14.   - name: consumerGroup # Optional. Used for input bindings.
  15.     value: "group1"
  16.   - name: publishTopic # Optional. Used for output bindings.
  17.     value: "topic3"
  18.   - name: authRequired # Required.
  19.     value: "true"
  20.   - name: saslUsername # Required if authRequired is `true`.
  21.     value: "user"
  22.   - name: saslPassword # Required if authRequired is `true`.
  23.     secretKeyRef:
  24.       name: kafka-secrets
  25.       key: saslPasswordSecret
  26.   - name: maxMessageBytes # Optional.
  27.     value: 1024

Spec metadata fields

FieldRequiredBinding supportDetailsExample
topicsNInputA comma-separated string of topics.“mytopic1,topic2”
brokersYInput/OutputA comma-separated string of Kafka brokers.“localhost:9092,dapr-kafka.myapp.svc.cluster.local:9093”
consumerGroupNInputA kafka consumer group to listen on. Each record published to a topic is delivered to one consumer within each consumer group subscribed to the topic.“group1”
publishTopicYOutputThe topic to publish to.“mytopic”
authRequiredYInput/OutputEnable SASL authentication with the Kafka brokers.“true”, “false”
saslUsernameNInput/OutputThe SASL username used for authentication. Only required if authRequired is set to “true”.“adminuser”
saslPasswordNInput/OutputThe SASL password used for authentication. Can be secretKeyRef to use a secret reference. Only required if authRequired is set to “true”.“”, “KeFg23!”
initialOffsetNThe initial offset to use if no offset was previously committed. Should be “newest” or “oldest”. Defaults to “newest”.“oldest”
maxMessageBytesNInput/OutputThe maximum size in bytes allowed for a single Kafka message. Defaults to 1024.2048

Binding support

This component supports both input and output binding interfaces.

This component supports output binding with the following operations:

  • create

Specifying a partition key

When invoking the Kafka binding, its possible to provide an optional partition key by using the metadata section in the request body.

The field name is partitionKey.

Example:

  1. curl -X POST http://localhost:3500/v1.0/bindings/myKafka \
  2.   -H "Content-Type: application/json" \
  3.   -d '{
  4.         "data": {
  5.           "message": "Hi"
  6.         },
  7.         "metadata": {
  8.           "partitionKey": "key1"
  9.         },
  10.         "operation": "create"
  11.       }'

Last modified September 20, 2021 : Merge pull request #1800 from greenie-msft/gRPC_proxying_video (36dff3c)