Production deployment

During development, we usually use compressed packages to start services with the form of a single node. However, production operation requires a simpler and more stable way. This page mainly explains how to deploy your EMQ X service from best practices of deployment architecture.

Deployment architecture

EMQ X cluster can be deployed as an IoT access service (IoT Hub). Currently, EMQ provides free software images out of the box on cloud service providers such as QingCloud, Aliyun, and AWS. For special hardware platforms and system versions such as Raspberry Pi and Linux ARM, source code compilation and installation can be used.

Typical deployment architecture:

Production deployment - 图1

Load Balancer (LB)

The Load Balancer (LB) distributes MQTT connections and traffic from devices across the EMQ X clusters. LB enhances the HA of the clusters, balances the loads among the cluster nodes and makes the dynamic expansion possible.

It is recommended that SSL connections are terminated by a LB. The links between devices and the LB are secured by SSL, while the links between the LB and EMQ X cluster nodes are plain TCP connections. By this setup, a single EMQ X cluster can serve a million devices.

LB products of public cloud providers:

Cloud providerSSL TerminationLB Product DOC/URL
QingCloudProduction deployment - 图2 (opens new window)Yeshttps://docs.qingcloud.com/product/network/loadbalancer/Production deployment - 图3 (opens new window)
AWSProduction deployment - 图4 (opens new window)Yeshttps://aws.amazon.com/cn/elasticloadbalancing/Production deployment - 图5 (opens new window)
AliyunProduction deployment - 图6 (opens new window)Nohttps://www.aliyun.com/product/slbProduction deployment - 图7 (opens new window)
UCloudProduction deployment - 图8 (opens new window)Unknownhttps://ucloud.cn/site/product/ulb.htmlProduction deployment - 图9 (opens new window)
QCloudProduction deployment - 图10 (opens new window)Unknownhttps://www.qcloud.com/product/clbProduction deployment - 图11 (opens new window)

LBs for Private Cloud:

Open-Source LBSSL TerminationDOC/URL
HAProxyProduction deployment - 图12 (opens new window)Yeshttps://www.haproxy.com/solutions/load-balancing.htmlProduction deployment - 图13 (opens new window)
NGINXProduction deployment - 图14 (opens new window)Yeshttps://www.nginx.com/solutions/load-balancing/Production deployment - 图15 (opens new window)

TIP

Qingcloud(EMQ X partner) is recommended for domestic public cloud deployments, AWS is recommended for foreign deployments, and HAProxy is recommended for LB for private deployments.

EMQ X Cluster

EMQ X cluster nodes are deployed behind LB. It is suggested that the nodes are deployed on VPCs or on a private network. Cloud provider, such as AWS, Azure or QingCloud, usually provides VPC network.

EMQ X Provides the MQTT service on following TCP ports by default:

PortDescription
1883MQTT
8883MQTT/SSL
8083MQTT/WebSocket
8084MQTT/WebSocket/SSL
8081Management API
18083Dashboard

Firewall should make the relevant ports accessible for public according to the MQTT access method.

TCP ports used by EMQ X node cluster:

PortDescription
4369Node discovery port (EPMD Mode)
4370Node discovery port
5370Cluster PRC

If deployed between nodes, firewalls should be configured that the above ports are inter-accessible between the nodes.

Deploying on QingCloud

  1. Create VPC network.
  2. Create a ‘private network’ for EMQ X cluster inside the VPC network, e.g. 192.168.0.0/24
  3. Create 2 EMQ X hosts inside the private network, like:
NodeIP address
emqx1192.168.0.2
emqx2192.168.0.3
  1. Install and cluster EMQ X on these two hosts. Please refer to the sections of cluster installation for details.
  2. Create LB and assign the public IP address.
  3. Create MQTT TCP listener:

image

Or create SSL listener and terminate the SSL connections on LB:

image

  1. Connect the MQTT clients to the LB using the public IP address and test the deployment.

Deploying on AWS

  1. Create VPC network.
  2. Create a ‘private network’ for EMQ X cluster inside the VPC network, e.g. 192.168.0.0/24
  3. Create 2 hosts inside the private network, like:
NodeIP address
emqx1192.168.0.2
emqx2192.168.0.3
  1. Open the TCP ports for MQTT services (e.g. 1883,8883) on the security group.
  2. Install and cluster EMQ X on these two hosts. Please refer to the sections of cluster installation for details.
  3. Create ELB (Classic Load Balancer), assign the VPC network, and assign the public IP address.
  4. Create MQTT TCP listener on the ELB:

image

Or create SSL listener and terminate the SSL connections on the ELB:

image

  1. Connect the MQTT clients to the ELB using the public IP address and test the deployment.

Deploying on private network

Direct connection of EMQ X cluster

EMQ X cluster should be DNS-resolvable and the clients access the cluster via domain name or IP list:

  1. Deploy EMQ X cluster. Please refer to the sections of ‘Installation’ and ‘EMQ X nodes clustering’ for details.
  2. Enable the access to the MQTT ports on the firewall (e.g. 1883, 8883).
  3. Client devices access the EMQ X cluster via domain name or IP list.

TIP

This kind of deployment is NOT recommended.

HAProxy LB

HAProxy serves as a LB for EMQ X cluster and terminates the SSL connections:

  1. Create EMQ X Cluster nodes like following:
nodeIP
emqx1192.168.0.2
emqx2192.168.0.3
  1. Configure /etc/haproxy/haproxy.cfg:
  1. listen mqtt-ssl
  2. bind *:8883 ssl crt /etc/ssl/emqx/emq.pem no-sslv3
  3. mode tcp
  4. maxconn 50000
  5. timeout client 600s
  6. default_backend emqx_cluster
  7. backend emqx_cluster
  8. mode tcp
  9. balance source
  10. timeout server 50s
  11. timeout check 5000
  12. server emqx1 192.168.0.2:1883 check inter 10000 fall 2 rise 5 weight 1
  13. server emqx2 192.168.0.3:1883 check inter 10000 fall 2 rise 5 weight 1

Nginx LB

NGINX Plus serves as a LB for EMQ X cluster and terminates the SSL connections

  1. Create EMQ X cluster nodes like following:
NodeIP
emqx1192.168.0.2
emqx2192.168.0.3
  1. Configure /etc/nginx/nginx.conf:
  1. stream {
  2. upstream stream_backend {
  3. zone tcp_servers 64k;
  4. hash $remote_addr;
  5. server 192.168.0.2:1883 max_fails=2 fail_timeout=30s;
  6. server 192.168.0.3:1883 max_fails=2 fail_timeout=30s;
  7. }
  8. server {
  9. listen 8883 ssl;
  10. status_zone tcp_server;
  11. proxy_pass stream_backend;
  12. proxy_buffer_size 4k;
  13. ssl_handshake_timeout 15s;
  14. ssl_certificate /etc/emqx/certs/cert.pem;
  15. ssl_certificate_key /etc/emqx/certs/key.pem;
  16. }
  17. }