Using a Load Balancer for External Access to NATS

Using a Load Balancer for External Access to NATS

In the example below, you can find how to use an AWS Network Load Balancer to connect externally to a cluster that has TLS setup.

One-line installer creates a secure cluster named ‘nats’

  1. curl -sSL https://nats-io.github.io/k8s/setup.sh | sh

Create AWS Network Load Balancer service

  1. echo '
  2. apiVersion: v1
  3. kind: Service
  4. metadata:
  5. name: nats-nlb
  6. namespace: default
  7. labels:
  8. app: nats
  9. annotations:
  10. service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
  11. spec:
  12. type: LoadBalancer
  13. externalTrafficPolicy: Local
  14. ports:
  15. - name: nats
  16. port: 4222
  17. protocol: TCP
  18. targetPort: 4222
  19. selector:
  20. app.kubernetes.io/name: nats
  21. ' | kubectl apply -f -

Check that it worked

  1. kubectl get svc nats-nlb -o wide

Example output

  1. NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
  2. nats-nlb LoadBalancer 10.100.67.123 a18b60a948fc611eaa7840286c60df32-9e96a2af4b5675ec.elb.us-east-2.amazonaws.com 4222:30297/TCP 151m app=nats

Publish a test message

  1. nats pub -s nats://a18b60a948fc611eaa7840286c60df32-9e96a2af4b5675ec.elb.us-east-2.amazonaws.com:4222 -creds nsc/nkeys/creds/KO/A/test.creds test.foo bar

Also, it would be recommended to set no_advertise to true in order to avoid gossiping internal addresses from pods in Kubernetes to NATS clients.

Setting up a NATS Server with external access on Azure

With the following, you can create a 3-node NATS Server cluster:

  1. kubectl apply -f https://raw.githubusercontent.com/nats-io/k8s/b55687a97a5fd55485e1af302fbdbe43d2d3b968/nats-server/leafnodes/nats-cluster.yaml

The configuration map from the NATS cluster that was created can be found below.

  1. ---
  2. apiVersion: v1
  3. kind: ConfigMap
  4. metadata:
  5. name: nats-config
  6. data:
  7. nats.conf: |
  8. pid_file: "/var/run/nats/nats.pid"
  9. http: 8222
  10. # debug: true
  11. ping_interval: 30s
  12. cluster {
  13. port: 6222
  14. no_advertise: true
  15. routes: [
  16. nats://nats-0.nats.default.svc:6222
  17. nats://nats-1.nats.default.svc:6222
  18. nats://nats-2.nats.default.svc:6222
  19. ]
  20. }
  21. leaf {
  22. port: 7422
  23. authorization {
  24. timeout: 3s
  25. users = [
  26. { user: "foo", pass: "bar" }
  27. ]
  28. }
  29. }

Now let’s expose the NATS Server by creating an L4 load balancer on Azure:

  1. kubectl apply -f https://raw.githubusercontent.com/nats-io/k8s/b55687a97a5fd55485e1af302fbdbe43d2d3b968/nats-server/leafnodes/lb.yaml

Confirm the public IP that was allocated to the nats-lb service that was created, in this case it is 52.155.49.45:

  1. kubectl get svc -o wide

Output

  1. NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
  2. kubernetes ClusterIP 10.0.0.1 <none> 443/TCP 81d <none>
  3. nats ClusterIP None <none> 4222/TCP,6222/TCP,8222/TCP,7777/TCP,7422/TCP,7522/TCP 7h46m app=nats
  4. nats-lb LoadBalancer 10.0.107.18 52.155.49.45 4222:31161/TCP,7422:30960/TCP 7h40m app=nats

Notice that the leafnode configuration requires authorization, so in order to connect to it we will need to configuration as follows:

  1. leaf {
  2. remotes = [
  3. {
  4. url: "nats://foo:bar@52.155.49.45:7422"
  5. }
  6. ]
  7. }

You can also add a NATS Streaming cluster into the cluster connecting to the port 4222:

  1. kubectl apply -f https://raw.githubusercontent.com/nats-io/k8s/b55687a97a5fd55485e1af302fbdbe43d2d3b968/nats-server/leafnodes/stan-server.yaml

Now if you create two NATS Servers that connect to the same leafnode port, they will be able to receive messages to each other:

  1. nats-server -c leafnodes/leaf.conf -p 4222 &
  2. nats-server -c leafnodes/leaf.conf -p 4223 &

Create a subscriber and publish a test message

  1. nats sub -s localhost:4222 foo &
  2. nats pub -s localhost:4223 foo hello

Output

  1. Listening on [foo]
  2. [#1] Received on [foo] : 'hello'