Creating a configuration for deploying a cluster

To deploy a YDB cluster, you need to create a cluster configuration.
This section describes how to create a YDB cluster configuration in YAML format.

Description of storage host configurations

Create and describe a host configuration. For each host configuration, specify the sequence number and a list of paths to disks and their types.
The following disk types are available: SSD, NVME, and ROT (in this case, ROT disks are HDD).

For example:

  1. host_configs:
  2. - drive:
  3.   - path: /dev/disk/by-partlabel/ydb_disk_ssd_01
  4.     type: SSD
  5.   host_config_id: 1

Cluster configuration - 图1

In this example, we can find exactly one type of host whose sequence number is 1. In this host configuration, exactly one disk is specified, its type is SSD and path is /dev/disk/by-partlabel/ydb_disk_ssd_01.

Below is another configuration example. Let’s assume that we have two types of host configurations, one of them with 2 disks available on the host and the other one with 3 disks.
This configuration can be specified as follows:

  1. host_configs:
  2. - drive:
  3.   - path: /dev/disk/by-partlabel/ydb_ssd_01
  4.     type: SSD
  5.   - path: /dev/disk/by-partlabel/ydb_ssd_02
  6.     type: SSD
  7.   host_config_id: 1
  8. - drive:
  9.   - path: /dev/disk/by-partlabel/ydb_ssd_01
  10.     type: SSD
  11.   - path: /dev/disk/by-partlabel/ydb_ssd_02
  12.     type: SSD
  13.   - path: /dev/disk/by-partlabel/ydb_ssd_03
  14.     type: SSD
  15.   host_config_id: 2

Cluster configuration - 图2

Description of cluster hosts

List the hosts to run a cluster on. For each host, specify the sequence number and the port where Interconnect will be run on this host.
You also need to specify the physical location of the host and the unique ID of the host configuration.

For example,

  1. hosts:
  2. - host: hostname1
  3.   host_config_id: 1
  4.   port: 19001
  5.   location:
  6.     unit: '1'
  7.     data_center: '1'
  8.     rack: '1'
  9. - host: hostname2
  10.   host_config_id: 1
  11.   node_id: 2
  12.   port: 19001
  13.   location:
  14.     unit: '1'
  15.     data_center: '1'
  16.     rack: '1'

Cluster configuration - 图3

Description of a cluster domain configuration

Describe a cluster domain configuration. Specify the domain name, storage types, the numbers of the hosts that will be included in StateStorage, and the nto_select parameter.
In the storage configuration, specify the type of storage and the type of storage fault tolerance (erasure), which will be used to initialize the database storage.
You should also specify what types of disks this storage type will correspond to. The following models of storage fault tolerance are available:

  • The block-4-2 configuration assumes deployment in 8 fail domains (by default, fail domains are racks) and can withstand a failure of no more than 2 fail domains.
  • The mirror-3-dc configuration assumes deployment in 3 data centers, each of which has 3 fault tolerance domains and can withstand a failure of one data center and one more fault tolerance domain (rack).
  • The none configuration doesn’t provide fault tolerance but is convenient for functional testing.

The StateStorage fault tolerance is defined by the nto_select parameter. The StateStorage configuration is fault-tolerant if any subset of nto_select servers that are part of StateStorage is fault-tolerant.
StateStorage remains available if most hosts are available for any subset of nto_select servers that are part of StateStorage.

For example, if a cluster uses the block-4-2 fault tolerance model for its storage, to make StateStorage fault-tolerant, set the nto_select parameter to 5 and place the hosts in 5 different availability domains.
If the cluster uses the mirror-3-dc fault tolerance model, to make StateStorage fault-tolerant, set the nto_select parameter to 9 and place the hosts in 3 data centers with 3 availability domains in each of them.

Warning

Make sure the NToSelect value is odd. For the proper operation of StateStorage, make sure that most of the NToSelect replicas are available for an arbitrary set of NToSelect hosts in StateStorage.

For example, you can use the following configuration for functional testing on a single server:

  1. domains_config:
  2.   domain:
  3.   - name: Root
  4.     storage_pool_types:
  5.     - kind: ssd
  6.       pool_config:
  7.         box_id: 1
  8.         erasure_species: none
  9.         kind: ssd
  10.         pdisk_filter:
  11.         - property:
  12.           - type: SSD
  13.         vdisk_kind: Default
  14.   state_storage:
  15.   - ring:
  16.       node:
  17.       - 1
  18.       nto_select: 1
  19.     ssid: 1

Cluster configuration - 图4

In this case, a domain is named Root and storage of the SSD type is created in it. This type of storage corresponds to disks with the type parameter set to SSD.
The erasure_species: none line in the parameter indicates that storage is created without fault tolerance.

If a cluster is located in three availability zones with 3 servers available in each of them, it may have the following configuration:

  1. domains_config:
  2.   domain:
  3.   - name: global
  4.     storage_pool_types:
  5.     - kind: ssd
  6.       pool_config:
  7.         box_id: 1
  8.         erasure_species: mirror-3-dc
  9.         kind: ssd
  10.         pdisk_filter:
  11.         - property:
  12.           - type: SSD
  13.         vdisk_kind: Default
  14.   state_storage:
  15.   - ring:
  16.       node: [1, 2, 3, 4, 5, 6, 7, 8, 9]
  17.       nto_select: 9
  18.     ssid: 1

Cluster configuration - 图5

In this case, a domain is named global and storage of the SSD type is also created in it. The erasure_species: mirror-3-dc line indicates that storage is created with the mirror-3-dc fault tolerance model. StateStorage will include 9 servers with the nto_select parameter set to 9.

Description of an actor system configuration

Create an actor system configuration. Specify how processor cores will be distributed across the pools of cores available in the system.

  1. actor_system_config:
  2.   executor:
  3.   - name: System
  4.     spin_threshold: 0
  5.     threads: 2
  6.     type: BASIC
  7.   - name: User
  8.     spin_threshold: 0
  9.     threads: 3
  10.     type: BASIC
  11.   - name: Batch
  12.     spin_threshold: 0
  13.     threads: 2
  14.     type: BASIC
  15.   - name: IO
  16.     threads: 1
  17.     time_per_mailbox_micro_secs: 100
  18.     type: IO
  19.   - name: IC
  20.     spin_threshold: 10
  21.     threads: 1
  22.     time_per_mailbox_micro_secs: 100
  23.     type: BASIC
  24.   scheduler:
  25.     progress_threshold: 10000
  26.     resolution: 256
  27.     spin_threshold: 0

Cluster configuration - 图6

Warning

Make sure the total number of cores assigned to the IC, Batch, System, and User pools does not exceed the number of available system cores.

Description of a static cluster group

Specify a static cluster group’s configuration. A static group is necessary for the operation of the basic cluster tablets, including Hive, SchemeShard, and BlobstorageContoller.
As a rule, these tablets do not store a lot of data, so we don’t recommend creating more than one static group.

For a static group, specify the disks and nodes that the static group will be placed on. For example, a configuration for the erasure: none model can be as follows:

  1. blob_storage_config:
  2.   service_set:
  3.     groups:
  4.     - erasure_species: none
  5.       rings:
  6.       - fail_domains:
  7.         - vdisk_locations:
  8.           - node_id: 1
  9.             path: /dev/disk/by-partlabel/ydb_ssd_02
  10.             pdisk_category: SSD
  11. ....

Cluster configuration - 图7

For a configuration located in 3 availability zones, specify 3 rings. For a configuration within a single availability zone, specify exactly one ring.

Sample cluster configurations

The repository provides model examples of cluster configurations for self-deployment. Check them out before deploying a cluster.