Pulsar command-line tools

Pulsar 提供了一些命令行工具，可以用于管理 Pulsar 的安装，测试性能，使用命令行生产者与消费者，等等。

All Pulsar command-line tools can be run from the bin directory of your installed Pulsar package. The following tools are currently documented:

pulsar
pulsar-client
pulsar-daemon
pulsar-perf
bookkeeper
broker-tool

获取帮助
You can get help for any CLI tool, command, or subcommand using the --help flag, or -h for short. 下面是一个示例：
$ bin/pulsar broker --help

`pulsar`

pulsar 工具用于在前台启动 Pulsar 组件，比如 bookies 和 Zookeeper

这些进程也可以在后台启动，使用 nohup，使用 pulsar-daemon 工具（它有着与 pulsar 相同的命令接口）。

用法：

$ pulsar command

Commands:

bookie
broker
compact-topic
discovery
configuration-store
initialize-cluster-metadata
proxy
standalone
websocket
zookeeper
zookeeper-shell

例子：

$ PULSAR_BROKER_CONF=/path/to/broker.conf pulsar broker

下表列出了你可以用来配置 pulsar 工具的环境变量。

变量	说明	默认值
`PULSAR_LOG_CONF`	Log4j 的配置文件	`conf/log4j2.yaml`
`PULSAR_BROKER_CONF`	broker 的配置文件	`conf/broker.conf`
`PULSAR_BOOKKEEPER_CONF`	bookie 的配置文件	`conf/bookkeeper.conf`
`PULSAR_ZK_CONF`	zookeeper 的配置文件	`conf/zookeeper.conf`
`PULSAR_CONFIGURATION_STORE_CONF`	配置存储的配置文件	`conf/global_zookeeper.conf`
`PULSAR_DISCOVERY_CONF`	发现服务的配置文件	`conf/discovery.conf`
`PULSAR_WEBSOCKET_CONF`	WebSocket 代理的配置文件	`conf/websocket.conf`
`PULSAR_STANDALONE_CONF`	standalone 的配置文件	`conf/standalone.conf`
`PULSAR_EXTRA_OPTS`	传递给 jvm 的额外选项
`PULSAR_EXTRA_CLASSPATH`	Pulsar classpath 的额外路径
`PULSAR_PID_DIR`	pulsar 服务器 PID 文件应该被存储的地方
`PULSAR_STOP_TIMEOUT`	当尝试停止 Bookie 服务器失败时，在强制杀死它之前的等待时间。

`bookie`

启动 bookie 服务器

用法：

$ pulsar bookie options

选项

选项	说明	默认值
`-readOnly`	Force start a read-only bookie server	false
`-withAutoRecovery`	启动带自动恢复服务的 bookie 服务器	false

示例

$ PULSAR_BOOKKEEPER_CONF=/path/to/bookkeeper.conf pulsar bookie \
  -readOnly \
  -withAutoRecovery

`broker`

启动一个 Pulsar broker

用法

$ pulsar broker options

选项

选项	说明	默认值
`-bc` , `—bookie-conf`	BookKeeper 的配置文件
`-rb` , `—run-bookie`	在 Pulsar broker 的同一主机上运行 Bookeeper bookie	false
`-ra` , `—run-bookie-autorecovery`	在 Pulsar broker 的同一主机上运行 BooKeeper 自动恢复守护进程。	false

示例

$ PULSAR_BROKER_CONF=/path/to/broker.conf pulsar broker

`compact-topic`

对 Pulsar 主题执行压缩（在新进程里）

用法

$ pulsar compact-topic options

选项

标记	说明	默认值
`-t` , `—topic`	你想压缩的 Pulsar 主题

示例

$ pulsar compact-topic --topic topic-to-compact

`discovery`

运行发现服务器

用法

$ pulsar discovery

示例

$ PULSAR_DISCOVERY_CONF=/path/to/discovery.conf pulsar discovery

`configuration-store`

启动 Pulsar 配置存储

用法

$ pulsar configuration-store

示例

$ PULSAR_CONFIGURATION_STORE_CONF=/path/to/configuration_store.conf pulsar configuration-store

`initialize-cluster-metadata`

一次性集群元数据初始化

用法

$ pulsar initialize-cluster-metadata options

选项

标记	说明	默认值
`-ub` , `—broker-service-url`	新集群的 broker 服务 URL
`-tb` , `—broker-service-url-tls`	新集群的broker 服务 URL（带 TLS 加密）
`-c` , `—cluster`	集群名称
`-cs` , `—configuration-store`	配置存储的 quorum 连接字符串
`—existing-bk-metadata-service-uri`	The metadata service URI of the existing BookKeeper cluster that you want to use
`-h` , `—help`	集群名称	false
`—initial-num-stream-storage-containers`	BookKeeper流存储容器数	16
`—initial-num-transaction-coordinators`	集群中分配的事务协调器的数量	16
`-uw` , `—web-service-url`	新集群的 web 服务 URL
`-tw` , `—web-service-url-tls`	新集群的 web 服务 URL（带 TLS 加密）
`-zk` , `—zookeeper`	本地 ZooKeeper 的 quorum 连接字符串
`—zookeeper-session-timeout-ms`	The local ZooKeeper session timeout. The time unit is in millisecond(ms)	30000

`proxy`

管理 Pulsar 代理

用法

$ pulsar proxy options

选项

标记	说明	默认值
`—configuration-store`	配置存储连接字符串
`-zk` , `—zookeeper-servers`	本地 ZooKeeper 连接字符串

示例

$ PULSAR_PROXY_CONF=/path/to/proxy.conf pulsar proxy \
  --zookeeper-servers zk-0,zk-1,zk2 \
  --configuration-store zk-0,zk-1,zk-2

`standalone`

运行使用本地 bookies 和本地 ZooKeeper 的 broker 服务

用法

$ pulsar standalone options

选项

标记	说明	默认值
`-a` , `—advertised-address`	standalone broker 广告的地址
`—bookkeeper-dir`	本地 bookie 的基本数据目录	data/standalone/bookeeper
`—bookkeeper-port`	本地 bookie 的基本端口目录	3181
`—no-broker`	只启动 ZooKeeper 和 BookKeeper 服务，而不启动 broker	false
`—num-bookies`	本地 bookies 的数量	1
`—only-broker`	只启动 Pulsar broker 服务（而不启动 ZooKeeper 或 BookKeeper）
`—wipe-data`	清除之前的 ZooKeeper/BookKeeper 数据
`—zookeeper-dir`	本地 ZooKeeper 的数据目录	data/standalone/zookeeper
`—zookeeper-port`	本地 ZooKeeper 的端口	2181

示例

$ PULSAR_STANDALONE_CONF=/path/to/standalone.conf pulsar standalone

`websocket`

用法

$ pulsar websocket

示例

$ PULSAR_WEBSOCKET_CONF=/path/to/websocket.conf pulsar websocket

`zookeeper`

启动 ZooKeeper 集群

用法

$ pulsar zookeeper

示例

$ PULSAR_ZK_CONF=/path/to/zookeeper.conf pulsar zookeeper

`zookeeper-shell`

使用 ZooKeeper shell 来连接到一个正在运行的 ZooKeeper 集群

用法

$ pulsar zookeeper-shell options

选项

标记	说明	默认值
`-c`, `—conf`	ZooKeeper 的配置文件

`pulsar-client`

The pulsar-client tool

用法

$ pulsar-client command

Commands

produce
consume

选项

标记	说明	默认值
`—auth-params`	认证参数，其格式取决于认证插件类中 `configure` 方法的实现，例如 “key1:val1,key2:val2” 或 “{“key1”:”val1”,”key2”:”val2”}”	{“saslJaasClientSectionName”:”PulsarClient”, “serverType”:”broker”}
`—auth-plugin`	认证插件类名称	org.apache.pulsar.client.impl.auth.AuthenticationSasl
`—listener-name`	broker 监听器名称
`—url`	Broker URL to which to connect	pulsar://localhost:6650/ ws://localhost:8080
`-v`, `—version`	获取Pulsar客户端版本

`produce`

发送单条或多条消息给指定的 broker 和主题

用法

$ pulsar-client produce topic options

选项

标记	说明	默认值
`-f`, `—files`	将要发送的以逗号分隔的文件路径，必须指定 -m 或 -f	[]
`-m`, `—messages`	将要发送的以逗号分隔的消息字符串，必须指定 -m 或 -f	[]
`-n`, `—num-produce`	发送单条或多条消息的次数；消息/文件 * 生产次数应小于 1000	1
`-r`, `—rate`	生产消息的频率（单位是消息/秒），值为 0 表示尽可能快地生产消息	0.0
`-c`, `—chunking`	Split the message and publish in chunks if the message size is larger than the allowed max size	false
`-s`, `—separator`	Character to split messages string with.	“,”
`-k`, `—key`	要添加的消息密钥	key=value string, like k1=v1,k2=v2.
`-p`, `—properties`	Properties to add. If you want to add multiple properties, use the comma as the separator, e.g. `k1=v1,k2=v2`.

`consume`

从指定的 broker 和主题消费消息

用法

$ pulsar-client consume topic options

选项

标记	说明	默认值
`—hex`	以十六进制格式显示二进制消息。	false
`-n`, `—num-messages`	要消费的消息数量，0 表示一直消费。	1
`-r`, `—rate`	消费消息的频率（单位是消息/秒），值为 0 表示尽可能快地消费消息	0.0
`—regex`	表明主题名称为正则表达式	false
`-s`, `—subscription-name`	Subscription name
`-t`, `—subscription-type`	The type of the subscription. Possible values: Exclusive, Shared, Failover, Key_Shared.	Exclusive
`-p`, `—subscription-position`	The position of the subscription. Possible values: Latest, Earliest.	Latest
`-m`, `—subscription-mode`	订阅模式。	Durable
`-q`, `—queue-size`	The size of consumer’s receiver queue.	0
`-mc`, `—max_chunked_msg`	最大的待处理块消息。	0
`-ac`, `—auto_ack_chunk_q_full`	如果队列已满，则自动确认为消费者的接收队列中最早的消息。	false
`—hide-content`	不要将消息打印到控制台。	false
`-st`, `—schema-type`	Set the schema type. Use `auto_consume` to dump AVRO and other structured data types. Possible values: bytes, auto_consume.	bytes

`pulsar-daemon`

围绕着 pulsar 工具的包装，用于在后台用 nohup 启动和停止进程，比如 ZooKeeper，bookies 和 Pulsar brokers。

pulsar-daemon 有一个与 pulsar 命令相似的接口，但是为各种服务添加了启动和停止命令。如果需要这些服务的列表，运行 pulsar-daemon 来查看帮助文档的输出或查看 pulsar 命令的文档。

用法

$ pulsar-daemon command

Commands

start
stop

`start`

使用 nohup 在后台启动服务

用法

$ pulsar-daemon start service

`stop`

停止一个已经使用 start 命令启动了的服务

用法

$ pulsar-daemon stop service options

选项

标记	说明	默认值
-force	如果无法正常停止则强制停止服务	false

`pulsar-perf`

用于性能测试 Pulsar broker 的工具。

用法

$ pulsar-perf command

Commands

consume
produce
read
websocket-producer
managed-ledger
monitor-brokers
simulation-client
simulation-controller
help

环境变量

下表列出了你可以用来配置 pulsar-perf 工具的环境变量。

变量	说明	默认值
`PULSAR_LOG_CONF`	Log4j 的配置文件	conf/log4j2.yaml
`PULSAR_CLIENT_CONF`	客户端的配置文件	conf/client.conf
`PULSAR_EXTRA_OPTS`	传递给 JVM 的额外选项
`PULSAR_EXTRA_CLASSPATH`	Pulsar classpath 的额外路径

`consume`

Run a consumer

用法

$ pulsar-perf consume options

选项

标记	说明	默认值
`—auth_params`	认证参数，其格式取决于认证插件类中 `configure` 方法的实现，例如 “key1:val1,key2:val2” 或 “{“key1”:”val1”,”key2”:”val2”}.
`—auth_plugin`	认证插件类名称
`—listener-name`	broker 监听器名称
`—acks-delay-millis`	Acknowledgements grouping delay in millis	100
`-k`, `—encryption-key-name`	The private key name to decrypt payload
`-v`, `—encryption-key-value-file`	The file which contains the private key to decrypt payload
`-h`, `—help`	帮助信息	false
`—conf-file`	Configuration file
`-c`, `—max-connections`	Max number of TCP connections to a single broker	100
`-n`, `—num-consumers`	Number of consumers (per topic)	1
`-t`, `—num-topics`	The number of topics	1
`-r`, `—rate`	Simulate a slow message consumer (rate in msg/s)	0
`-q`, `—receiver-queue-size`	Size of the receiver queue	1000
`-u`, `—service-url`	Pulsar service URL
`-i`, `—stats-interval-seconds`	Statistics interval seconds. If 0, statistics will be disabled	0
`-s`, `—subscriber-name`	Subscriber name prefix	sub
`-ss`, `—subscriptions`	A list of subscriptions to consume on (e.g. sub1,sub2)	sub
`-st`, `—subscription-type`	Subscriber type. Possible values are Exclusive, Shared, Failover, Key_Shared.	Exclusive
`-sp`, `—subscription-position`	Subscriber position. Possible values are Latest, Earliest.	Latest
`—trust-cert-file`	Path for the trusted TLS certificate file
`—tls-allow-insecure`	允许不安全的 TLS 连接。

`produce`

Run a producer

用法

$ pulsar-perf produce options

选项

标记	说明	默认值
`—auth_params`	认证参数，其格式取决于认证插件类中 `configure` 方法的实现，例如 “key1:val1,key2:val2” 或 “{“key1”:”val1”,”key2”:”val2”}.
`—auth_plugin`	认证插件类名称
`—listener-name`	broker 监听器名称
`-b`, `—batch-time-window`	Batch messages in a window of the specified number of milliseconds	1
`-z`, `—compression`	Compress messages’ payload. Possible values are NONE, LZ4, ZLIB, ZSTD or SNAPPY.
`—conf-file`	Configuration file
`-k`, `—encryption-key-name`	The public key name to encrypt payload
`-v`, `—encryption-key-value-file`	The file which contains the public key to encrypt payload
`-h`, `—help`	帮助信息	false
`-c`, `—max-connections`	Max number of TCP connections to a single broker	100
`-o`, `—max-outstanding`	Max number of outstanding messages	1000
`-p`, `—max-outstanding-across-partitions`	Max number of outstanding messages across partitions	50000
`-m`, `—num-messages`	Number of messages to publish in total. If set to 0, it will keep publishing.	0
`-n`, `—num-producers`	The number of producers (per topic)	1
`-t`, `—num-topic`	The number of topics	1
`-f`, `—payload-file`	Use payload from an UTF-8 encoded text file and a payload will be randomly selected when publishing messages
`-e`, `—payload-delimiter`	The delimiter used to split lines when using payload from a file	\n
`-r`, `—rate`	Publish rate msg/s across topics	100
`-u`, `—service-url`	Pulsar service URL
`-s`, `—size`	Message size (in bytes)	1024
`-i`, `—stats-interval-seconds`	Statistics interval seconds. If 0, statistics will be disabled.	0
`-time`, `—test-duration`	Test duration in secs. If set to 0, it will keep publishing.	0
`—trust-cert-file`	Path for the trusted TLS certificate file
`—warmup-time`	Warm-up time in seconds	1
`—tls-allow-insecure`	允许不安全的 TLS 连接。

`read`

运行主题读者

用法

$ pulsar-perf read options

选项

标记	说明	默认值
`—auth_params`	认证参数，其格式取决于认证插件类中 `configure` 方法的实现，例如 “key1:val1,key2:val2” 或 “{“key1”:”val1”,”key2”:”val2”}.
`—auth_plugin`	认证插件类名称
`—listener-name`	broker 监听器名称
`—conf-file`	Configuration file
`-h`, `—help`	帮助信息	false
`-c`, `—max-connections`	Max number of TCP connections to a single broker	100
`-t`, `—num-topics`	The number of topics	1
`-r`, `—rate`	Simulate a slow message reader (rate in msg/s)	0
`-q`, `—receiver-queue-size`	Size of the receiver queue	1000
`-u`, `—service-url`	Pulsar service URL
`-m`, `—start-message-id`	Start message id. This can be either ‘earliest’, ‘latest’ or a specific message id by using ‘lid:eid’	earliest
`-i`, `—stats-interval-seconds`	Statistics interval seconds. If 0, statistics will be disabled.	0
`—trust-cert-file`	Path for the trusted TLS certificate file
`—use-tls`	Use TLS encryption on the connection	false
`—tls-allow-insecure`	允许不安全的 TLS 连接。

`websocket-producer`

启动 websocket 生产者

用法

$ pulsar-perf websocket-producer options

选项

标记	说明	默认值
`—auth_params`	认证参数，其格式取决于认证插件类中 `configure` 方法的实现，例如 “key1:val1,key2:val2” 或 “{“key1”:”val1”,”key2”:”val2”}.
`—auth_plugin`	认证插件类名称
`—conf-file`	Configuration file
`-h`, `—help`	帮助信息	false
`-m`, `—num-messages`	Number of messages to publish in total. If 0, it will keep publishing	0
`-t`, `—num-topic`	The number of topics	1
`-f`, `—payload-file`	使用文件中的 payload 而不是空缓存中的
`-u`, `—proxy-url`	Pulsar 代理 URL，比如 “ws://localhost:8080/“
`-r`, `—rate`	Publish rate msg/s across topics	100
`-s`, `—size`	消息大小（字节）	1024
`-time`, `—test-duration`	Test duration in secs. If 0, it will keep publishing	0

`managed-ledger`

直接写入 managed-ledgers

用法

$ pulsar-perf managed-ledger options

选项

标记	说明	默认值
`-a`, `—ack-quorum`	Ledger ack quorum	1
`-dt`, `—digest-type`	BookKeeper digest type. Possible Values: [CRC32, MAC, CRC32C, DUMMY]	CRC32C
`-e`, `—ensemble-size`	Ledger 整体大小	1
`-h`, `—help`	帮助信息	false
`-c`, `—max-connections`	单个 bookie 的最大 TCP 连接数	1
`-o`, `—max-outstanding`	未处理请求的最大数量	1000
`-m`, `—num-messages`	Number of messages to publish in total. If 0, it will keep publishing	0
`-t`, `—num-topic`	managed ledger 的数量	1
`-r`, `—rate`	在managed ledger中的写入速率（消息/秒）	100
`-s`, `—size`	消息大小（字节）	1024
`-time`, `—test-duration`	Test duration in secs. If 0, it will keep publishing	0
`—threads`	正在写入的线程数	1
`-w`, `—write-quorum`	Ledger write quorum	1
`-zk`, `—zookeeperServers`	ZooKeeper 连接字符串

`monitor-brokers`

Continuously receive broker data and/or load reports

用法

$ pulsar-perf monitor-brokers options

选项

标记	说明	默认值
`—connect-string`	A connection string for one or more ZooKeeper servers
`-h`, `—help`	帮助信息	false

`simulation-client`

Run a simulation server acting as a Pulsar client. Uses the client configuration specified in conf/client.conf.

用法

$ pulsar-perf simulation-client options

选项

标记	说明	默认值
`—port`	用于监听控制器的端口	0
`—service-url`	Pulsar 服务 URL
`-h`, `—help`	帮助信息	false

`simulation-controller`

Run a simulation controller to give commands to servers

用法

$ pulsar-perf simulation-controller options

选项

标记	说明	默认值
`—client-port`	The port that the clients are listening on	0
`—clients`	Comma-separated list of client hostnames
`—cluster`	The cluster to test on
`-h`, `—help`	帮助信息	false

`help`

帮助消息

用法

$ pulsar-perf help

`bookkeeper`

A tool for managing BookKeeper.

用法

$ bookkeeper command

Commands

auto-recovery
bookie
localbookie
upgrade
shell

环境变量

The table below lists the environment variables that you can use to configure the bookkeeper tool.

变量	说明	默认值
BOOKIE_LOG_CONF	Log4j 的配置文件	conf/log4j2.yaml
BOOKIE_CONF	BookKeeper configuration file	conf/bk_server.conf
BOOKIE_EXTRA_OPTS	传递给 JVM 的额外选项
BOOKIE_EXTRA_CLASSPATH	Extra paths for BookKeeper’s classpath
ENTRY_FORMATTER_CLASS	The Java class used to format entries
BOOKIE_PID_DIR	Folder where the BookKeeper server PID file should be stored
BOOKIE_STOP_TIMEOUT	当尝试停止 Bookie 服务器失败时，在强制杀死它之前的等待时间。

`auto-recovery`

Runs an auto-recovery service daemon

用法

$ bookkeeper auto-recovery options

选项

标记	说明	默认值
`-c`, `—conf`	Configuration for the auto-recovery daemon

`bookie`

Starts up a BookKeeper server (aka bookie)

用法

$ bookkeeper bookie options

选项

标记	说明	默认值
`-c`, `—conf`	Configuration for the auto-recovery daemon
-readOnly	Force start a read-only bookie server	false
-withAutoRecovery	Start auto-recovery service bookie server	false

`localbookie`

Runs a test ensemble of N bookies locally

用法

$ bookkeeper localbookie N

`upgrade`

Upgrade the bookie’s filesystem

用法

$ bookkeeper upgrade options

选项

标记	说明	默认值
`-c`, `—conf`	Configuration for the auto-recovery daemon
`-u`, `—upgrade`	Upgrade the bookie’s directories

`shell`

Run shell for admin commands. To see a full listing of those commands, run bookkeeper shell without an argument.

用法

$ bookkeeper shell

示例

$ bookkeeper shell bookiesanity

`broker-tool`

broker- tool 用于在指定的 broker 上进行操作

用法

$ broker-tool command

Commands

load-report
help

示例获取关于某条命令的更多信息的两种方式：

$ broker-tool help command
$ broker-tool command --help

`load-report`

Collect the load report of a specific broker. The command is run on a broker, and used for troubleshooting why broker can’t collect right load report.

选项

标记	说明	默认值
`-i`, `—interval`	收集负载报告的间隔（毫秒）
`-h`, `—help`	显示帮助信息

Pulsar CLI tools

Pulsar command-line tools

获取帮助

pulsar

bookie

broker

compact-topic

discovery

configuration-store

initialize-cluster-metadata

proxy

standalone

websocket

zookeeper

zookeeper-shell

pulsar-client

produce

consume

pulsar-daemon

start

stop

pulsar-perf

consume

produce

read

websocket-producer

managed-ledger

monitor-brokers

simulation-client

simulation-controller

help

bookkeeper

auto-recovery

bookie

localbookie

upgrade

shell

broker-tool

load-report

`pulsar`

`bookie`

`broker`

`compact-topic`

`discovery`

`configuration-store`

`initialize-cluster-metadata`

`proxy`

`standalone`

`websocket`

`zookeeper`

`zookeeper-shell`

`pulsar-client`

`produce`

`consume`

`pulsar-daemon`

`start`

`stop`

`pulsar-perf`

`consume`

`produce`

`read`

`websocket-producer`

`managed-ledger`

`monitor-brokers`

`simulation-client`

`simulation-controller`

`help`

`bookkeeper`

`auto-recovery`

`bookie`

`localbookie`

`upgrade`

`shell`

`broker-tool`

`load-report`