ArangoDB cluster

There are several ways to start an ArangoDB cluster. In this section we will focuson our recommended way to start ArangoDB: the ArangoDB Starter.

Datacenter to datacenter replication requires the rocksdb storage engine. Theexample setup described in this section will have rocksdb enabled. If you chooseto deploy with a different strategy keep in mind to set the storage engine.

For other possibilities to deploy an ArangoDB cluster seeCluster Deployment.

The Starter simplifies things for the operator and will coordinate a distributedcluster startup across several machines and assign cluster roles automatically.

When started on several machines and enough machines have joined, the Starters_will start _Agents, Coordinators and DBservers on these machines.

When running the Starter will supervise its child tasks (namely Coordinators,DBservers and Agents) and restart them in case of failure.

To start the cluster using a systemd unit file use the following:

  1. [Unit]
  2. Description=Run the ArangoDB Starter
  3. After=network.target
  5. [Service]
  6. Restart=on-failure
  7. EnvironmentFile=/etc/arangodb.env
  8. EnvironmentFile=/etc/arangodb.env.local
  9. Environment=DATADIR=/var/lib/arangodb/cluster
  10. ExecStartPre=/usr/bin/sh -c "mkdir -p ${DATADIR}"
  11. ExecStart=/usr/bin/arangodb \
  12.     --starter.address=${PRIVATEIP} \
  13.     --starter.data-dir=${DATADIR} \
  14.     --starter.join=${STARTERENDPOINTS} \
  15.     --server.storage-engine=rocksdb \
  16.     --auth.jwt-secret=${CLUSTERSECRETPATH}
  17. TimeoutStopSec=60
  18. LimitNOFILE=1048576
  20. [Install]
  21. WantedBy=multi-user.target

Note that we set rocksdb in the unit service file.

Cluster authentication

The communication between the cluster nodes use a token (JWT) to authenticate.This must be shared between cluster nodes.

Sharing secrets is obviously a very delicate topic. The above workflow assumesthat the operator will put a secret in a file named ${CLUSTERSECRETPATH}.

We recommend to use a dedicated system for managing secrets like HashiCorp’s Vault.

Required ports

As soon as enough machines have joined, the Starter will begin starting Agents,Coordinators and DBservers.

Each of these tasks needs a port to communicate. Please make sure that the followingports are available on all machines:

  • 8529 for Coordinators
  • 8530 for DB-Servers
  • 8531 for Agents

The Starter itself will use port 8528.

Since the Agents are so critical to the availability of both the ArangoDB and the ArangoSync cluster,it is recommended to run Agents on dedicated machines. Consider these machines “pets”.

Coordinators and DBServers can be deployed on other machines that should be considered “cattle”.