The Pulsar proxy

Pulsar代理是一个可选的网关,你可以在Pulsar集群中的brokers之前运行它们。 You can run a Pulsar proxy in cases when direction connections between clients and Pulsar brokers are either infeasible, undesirable, or both, for example when you run Pulsar in a cloud environment or on Kubernetes or an analogous platform.

配置Proxy

Proxy 有办法来找到集群中 broker 的地址。 你可以通过配置代理来直接连接到服务发现,或者在配置中指定broker URL。

Option 1: Use service discovery

Pulsar uses ZooKeeper for service discovery. To connect the proxy to ZooKeeper, specify the following in conf/proxy.conf.

  1. zookeeperServers=zk-0,zk-1,zk-2
  2. configurationStoreServers=zk-0:2184,zk-remote:2184

If you use service discovery, the network ACL must allow the proxy to talk to the ZooKeeper nodes on the zookeeper client port, which is usually 2181, and on the configuration store client port, which is 2184 by default. 打开网络ACLs意味着如果有人破坏了代理,那么他们可以完全访问ZooKeeper。 For this reason, using broker URLs to configure the proxy is more secure.

Option 2: Use broker URLs

配置代理的更安全的方法是指定连接到brokers的URL。

Authorization at the proxy requires access to ZooKeeper, so if you use these broker URLs to connect to the brokers, you should disable the Proxy level authorization. Brokers still authorize requests after the proxy forwards them.

您可以在 conf/proxy.conf中配置broker URLs,如下所示。

  1. brokerServiceURL=pulsar://brokers.example.com:6650
  2. brokerWebServiceURL=http://brokers.example.com:8080
  3. functionWorkerWebServiceURL=http://function-workers.example.com:8080

Or if you use TLS:

  1. brokerServiceURLTLS=pulsar+ssl://brokers.example.com:6651
  2. brokerWebServiceURLTLS=https://brokers.example.com:8443
  3. functionWorkerWebServiceURL=https://function-workers.example.com:8443

提供的URLs中的主机名应该是指向多个 broker 的DNS条目,或由多个broker 的IP地址支持的虚拟IP,以便在单个 broker 不可用时 ,代理不会失去与 Pulsar 群集的连接。

The ports to connect to the brokers (6650 and 8080, or in the case of TLS, 6651 and 8443) should be open in the network ACLs.

Note that if you do not use functions, then you do not need to configure functionWorkerWebServiceURL.

启动 Proxy

To start the proxy:

  1. $ cd /path/to/pulsar/directory
  2. $ bin/pulsar proxy

You can run as many instances of the Pulsar proxy in a cluster as you want.

停止 Proxy

The Pulsar proxy runs by default in the foreground. To stop the proxy, simply stop the process in which the proxy is running.

Proxy frontends

You can run the Pulsar proxy behind some kind of load-distributing frontend, such as an HAProxy load balancer.

Pulsar 客户端连接 Proxy

Once your Pulsar proxy is up and running, preferably behind a load-distributing frontend, clients can connect to the proxy via whichever address that the frontend uses. If the address is the DNS address pulsar.cluster.default, for example, then the connection URL for clients is pulsar://pulsar.cluster.default:6650.

Proxy configuration

You can configure the Pulsar proxy using the proxy.conf configuration file. The following parameters are available in that file:

配置项Description默认值
zookeeperServersZooKeeper quorum 连接字符串(以逗号分隔的列表)
configurationStoreServers配置存储连接字符串(以逗号分隔的列表)
zookeeperSessionTimeoutMsZooKeeper会话超时(以毫秒为单位)30000
servicePort用于服务器二进制Protobuf请求的端口6650
servicePortTls用于服务器二进制Protobuf TLS请求的端口6651
statusFilePath在响应服务发现健康检查时,用于确定代理实例的轮换状态的文件的路径
advertisedAddressHostname or IP address the service advertises to the outside world.InetAddress.getLocalHost().getHostname()
authenticationEnabled是否为Pulsar代理启用身份验证false
authenticateMetricsEndpointWhether the ‘/metrics’ endpoint requires authentication. Defaults to true. ‘authenticationEnabled’ must also be set for this to take effect.true
authenticationProviders身份验证提供者名称列表(以逗号分隔的类名列表)
authorizationEnabled是否由Pulsar代理强制执行授权false
authorizationProvider授权提供程序的完全限定类名org.apache.pulsar.broker.authorization.PulsarAuthorizationProvider
brokerClientAuthenticationPluginPulsar代理使用的身份验证插件,用于对Pulsar brokers进行身份验证
brokerClientAuthenticationParametersPulsar代理用于对Pulsar Brokers进行身份验证的参数
brokerClientTrustCertsFilePathPulsar代理用于对Pulsar Brokers进行身份验证的可信证书的路径
superUserRolesRole names that are treated as “super-users,” meaning that they are able to perform all admin
forwardAuthorizationCredentialsWhether client authorization credentials are forwared to the broker for re-authorization. Authentication must be enabled via authenticationEnabled=true for this to take effect.false
maxConcurrentInboundConnectionsMax concurrent inbound connections. The proxy rejects requests beyond that.10000
maxConcurrentLookupRequestsMax concurrent outbound connections. The proxy errors out requests beyond that.50000
tlsEnabledInProxy是否为代理启用TLSfalse
tlsEnabledWithBroker与Pulsar Brokers通信时是否启用TLSfalse
tlsCertificateFilePathTLS证书文件的路径
tlsKeyFilePathTLS私钥文件的路径
tlsTrustCertsFilePath受信任的TLS证书pem文件的路径
tlsHostnameVerificationEnabled当代理与brokers建立TLS连接时是否验证主机名false
tlsRequireTrustedClientCertOnConnectWhether client certificates are required for TLS. Connections are rejected if the client certificate is not trusted.false