AWS SNS/SQS

Detailed documentation on the AWS SNS/SQS pubsub component

This article describes configuring Dapr to use AWS SNS/SQS for pub/sub on local and Kubernetes environments.

Setup SNS/SQS

For local development the localstack project is used to integrate AWS SNS/SQS. Follow the instructions here to install the localstack CLI.

In order to use localstack with your pubsub binding, you need to provide the endpoint configuration in the component metadata. The endpoint is unncessary when running against production AWS.

See Authenticating to AWS for information about authentication-related attributes

  1. apiVersion: dapr.io/v1alpha1
  2. kind: Component
  3. metadata:
  4.   name: messagebus
  5. spec:
  6.   type: pubsub.snssqs
  7.   version: v1
  8.   metadata:
  9.     - name: endpoint
  10.       value: http://localhost:4566
  11.     # Use us-east-1 for localstack
  12.     - name: awsRegion
  13.       value: us-east-1

To run localstack on Kubernetes, you can apply the configuration below. Localstack is then reachable at the DNS name http://localstack.default.svc.cluster.local:4566 (assuming this was applied to the default namespace) and this should be used as the endpoint

  1. apiVersion: apps/v1
  2. kind: Deployment
  3. metadata:
  4.   name: localstack
  5. spec:
  6.   # using the selector, we will expose the running deployments
  7.   # this is how Kubernetes knows, that a given service belongs to a deployment
  8.   selector:
  9.     matchLabels:
  10.       app: localstack
  11.   replicas: 1
  12.   template:
  13.     metadata:
  14.       labels:
  15.         app: localstack
  16.     spec:
  17.       containers:
  18.       - name: localstack
  19.         image: localstack/localstack:latest
  20.         ports:
  21.           # Expose the edge endpoint
  22.           - containerPort: 4566
  23. ---
  24. kind: Service
  25. apiVersion: v1
  26. metadata:
  27.   name: localstack
  28.   labels:
  29.     app: localstack
  30. spec:
  31.   selector:
  32.     app: localstack
  33.   ports:
  34.   - protocol: TCP
  35.     port: 4566
  36.     targetPort: 4566
  37.   type: LoadBalancer

In order to run in AWS, you should create an IAM user with permissions to the SNS and SQS services. Use the account ID and account secret and plug them into the awsAccountID and awsAccountSecret in the component metadata using kubernetes secrets.

Create a Dapr component

The next step is to create a Dapr component for SNS/SQS.

Create the following YAML file named snssqs.yaml:

  1. apiVersion: dapr.io/v1alpha1
  2. kind: Component
  3. metadata:
  4.   name: <NAME>
  5.   namespace: <NAMESPACE>
  6. spec:
  7.   type: pubsub.snssqs
  8.   version: v1
  9.   metadata:
  10.     # ID of the AWS account with appropriate permissions to SNS and SQS
  11.     - name: accessKey
  12.       value: **********
  13.     # Secret for the AWS user
  14.     - name: secretKey
  15.       value: **********
  16.     # The AWS region you want to operate in. 
  17.     # See this page for valid regions: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html
  18.     # Make sure that SNS and SQS are available in that region.
  19.     - name: region
  20.       value: us-east-1

Warning

The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described here.

Apply the configuration

Visit this guide for instructions on configuring pub/sub components.

Related links

Last modified February 16, 2021: Merge pull request #1235 from dapr/update-v0.11 (b4e9fbb)