Managing partitioned topics

您可以使用Pulsar的 admin API来创建和管理分区主题。

本教程中，topic 名称的结构为：

persistent://tenant/namespace/topic

分区主题资源

创建

Partitioned topics in Pulsar must be explicitly created. When creating a new partitioned topic you need to provide a name for the topic as well as the desired number of partitions.

Note

By default, after 60 seconds of creation, topics are considered inactive and deleted automatically to prevent from generating trash data.

To disable this feature, set brokerDeleteInactiveTopicsEnabled to false.

To change the frequency of checking inactive topics, set brokerDeleteInactiveTopicsFrequencySeconds to your desired value.

For more information about these two parameters, see here.

pulsar-admin

你可以使用create-partitioned-topic命令创建partitioned topic，并指定topic的名字；使用-p 或 --partitions标志指定分区数。

下面是一个示例：

$ bin/pulsar-admin topics create-partitioned-topic \
  persistent://my-tenant/my-namespace/my-topic \
  --partitions 4

Note

If there already exists a non partitioned topic with suffix ‘-partition-‘ followed by numeric value like ‘xyz-topic-partition-10’, then you can not create partitioned topic with name ‘xyz-topic’ as the partitions of the partitioned topic could override the existing non partitioned topic. You have to delete that non partitioned topic first then create the partitioned topic.

REST API

PUT /admin/v2/persistent/:tenant/:namespace/:topic/partitions

Java

String topicName = "persistent://my-tenant/my-namespace/my-topic";
int numPartitions = 4;
admin.persistentTopics().createPartitionedTopic(topicName, numPartitions);

Create missed partitions

Try to create partitions for partitioned topic. The partitions of partition topic has to be created, can be used by repair partitions when topic auto creation is disabled

pulsar-admin

You can create missed partitions using the create-missed-partitions command and specifying the topic name as an argument.

下面是一个示例：

$ bin/pulsar-admin topics create-missed-partitions \
  persistent://my-tenant/my-namespace/my-topic \

REST API

POST /admin/v2/persistent/:tenant/:namespace/:topic

Java

String topicName = "persistent://my-tenant/my-namespace/my-topic";
admin.persistentTopics().createMissedPartitions(topicName);

获取元数据

Partitioned topics have metadata associated with them that you can fetch as a JSON object. The following metadata fields are currently available:

pulsar-admin

You can see the number of partitions in a partitioned topic using the get-partitioned-topic-metadata subcommand. 下面是一个示例：

$ pulsar-admin topics get-partitioned-topic-metadata \
  persistent://my-tenant/my-namespace/my-topic
{
  "partitions": 4
}

REST API

GET /admin/v2/persistent/:tenant/:namespace/:topic/partitions

Java

String topicName = "persistent://my-tenant/my-namespace/my-topic";
admin.persistentTopics().getPartitionedTopicMetadata(topicName);

更新

You can update the number of partitions on an existing partitioned topic if the topic is non-global. To update, the new number of partitions must be greater than the existing number.

Decrementing the number of partitions would deleting the topic, which is not supported in Pulsar.

已创建的分区生产者和消费者，将自动查找新创建的分区。

pulsar-admin

Partitioned topics can be updated using the update-partitioned-topic command.

$ pulsar-admin topics update-partitioned-topic \
  persistent://my-tenant/my-namespace/my-topic \
  --partitions 8

REST API

POST /admin/v2/persistent/:tenant/:cluster/:namespace/:destination/partitions

Java

admin.persistentTopics().updatePartitionedTopic(persistentTopic, numPartitions);

删除

pulsar-admin

Partitioned topics can be deleted using the delete-partitioned-topic command, specifying the topic by name:

$ bin/pulsar-admin topics delete-partitioned-topic \
  persistent://my-tenant/my-namespace/my-topic

REST API

DELETE /admin/v2/persistent/:topic/:namespace/:destination/partitions

Java

admin.persistentTopics().delete(persistentTopic);

获取资源列表

It provides a list of persistent topics existing under a given namespace.

pulsar-admin

$ pulsar-admin topics list tenant/namespace
persistent://tenant/namespace/topic1
persistent://tenant/namespace/topic2

REST API

GET /admin/v2/persistent/:tenant/:namespace

Java

admin.persistentTopics().getList(namespace);

统计信息

It shows current statistics of a given partitioned topic. Here’s an example payload:

{
  "msgRateIn": 4641.528542257553,
  "msgThroughputIn": 44663039.74947473,
  "msgRateOut": 0,
  "msgThroughputOut": 0,
  "averageMsgSize": 1232439.816728665,
  "storageSize": 135532389160,
  "publishers": [
    {
      "msgRateIn": 57.855383881403576,
      "msgThroughputIn": 558994.7078932219,
      "averageMsgSize": 613135,
      "producerId": 0,
      "producerName": null,
      "address": null,
      "connectedSince": null
    }
  ],
  "subscriptions": {
    "my-topic_subscription": {
      "msgRateOut": 0,
      "msgThroughputOut": 0,
      "msgBacklog": 116632,
      "type": null,
      "msgRateExpired": 36.98245516804671,
      "consumers": []
    }
  },
  "replication": {}
}

The following stats are available:

pulsar-admin

The stats for the partitioned topic and its connected producers and consumers can be fetched by using the partitioned-stats command, specifying the topic by name:

$ pulsar-admin topics partitioned-stats \
  persistent://test-tenant/namespace/topic \
  --per-partition

REST API

GET /admin/v2/persistent/:tenant/:namespace/:topic/partitioned-stats

Java

admin.topics().getPartitionedStats(persistentTopic, true /* per partition */, false /* is precise backlog */);

Internal stats

获取 topic 的详细统计信息。

{
  "entriesAddedCounter": 20449518,
  "numberOfEntries": 3233,
  "totalSize": 331482,
  "currentLedgerEntries": 3233,
  "currentLedgerSize": 331482,
  "lastLedgerCreatedTimestamp": "2016-06-29 03:00:23.825",
  "lastLedgerCreationFailureTimestamp": null,
  "waitingCursorsCount": 1,
  "pendingAddEntriesCount": 0,
  "lastConfirmedEntry": "324711539:3232",
  "state": "LedgerOpened",
  "ledgers": [
    {
      "ledgerId": 324711539,
      "entries": 0,
      "size": 0
    }
  ],
  "cursors": {
    "my-subscription": {
      "markDeletePosition": "324711539:3133",
      "readPosition": "324711539:3233",
      "waitingReadOp": true,
      "pendingReadOps": 0,
      "messagesConsumedCounter": 20449501,
      "cursorLedger": 324702104,
      "cursorLedgerLastEntry": 21,
      "individuallyDeletedMessages": "[(324711539:3134‥324711539:3136], (324711539:3137‥324711539:3140], ]",
      "lastLedgerSwitchTimestamp": "2016-06-29 01:30:19.313",
      "state": "Open"
    }
  }
}

pulsar-admin

The internal stats for the partitioned topic can be fetched by using the stats-internal command, specifying the topic by name:

$ pulsar-admin topics stats-internal \
  persistent://test-tenant/namespace/topic

REST API

GET /admin/v2/persistent/:tenant/:namespace/:topic/internalStats

Java

admin.persistentTopics().getInternalStats(persistentTopic);