Foxx in a cluster setup

When running ArangoDB in a cluster the Foxx services will run on eachcoordinator. Installing, upgrading and uninstalling services on any coordinatorwill automatically distribute the changes to the service to the other coordinators,making deployments as easy as in single-server mode.

The same considerations that apply to writing Foxx services for astandalone server also apply to writing services for a cluster:

You should avoid any kind of file system state beyond the deployed servicebundle itself. Don’t write data to the file system or encodeany expectations of the file system state other than the files in theservice folder that were installed as part of the service(e.g. don’t use file uploads or custom log files).

Additionally, special precautions need to be taken when using thedevelopment mode in a cluster.

How ArangoDB distributes services

When you install, replace, upgrade or remove a service, these actions firsttake place on a single coordinator and are then distributed to the othercoordinators. If a coordinator for some reason fails to be informed,its periodic self-healing process will pick up the changes eventuallyand apply them anyway.

• When installing, upgrading or replacing a service, the new service isextracted to a temporary directory where Foxx validates the manifest fileand parses the referenced scripts and main file.

• When replacing, upgrading or removing a service, the old service’s teardownscript is executed in a single thread of the coordinator as desired.

• When replacing, upgrading or installing a service, the new service’s setupscript is executed in a single thread of the coordinator as desired.

• The validated service bundle is copied to the coordinator’s service bundlesdirectory, extracted to the coordinator’s service directory and committedto an internal collection along with a signature.

• The service metadata stored in another internal collection is updated,replaced or created with the new service bundle’s signature. An upgraderetains existing metadata like configuration and dependencies whereasa replace completely discards any existing metadata.

• The existing service is unloaded from the coordinator’s worker threadsand the new service is reloaded. If the new service runs into an errorat this point, the service will be marked as broken andneeds to be replaced manually.

• The coordinator triggers a local self-heal followed by triggeringa self-heal on all other coordinators.

• During the self-heal the coordinator compares the signature of thelocal bundle of each service against the signature stored in thatservice’s metadata in the database. If necessary, the correspondingnew bundle is downloaded from the database and extracted and the serviceis reloaded as in step 6 before.

Note that this means that any service that passes the initial validation stepwill complete the install, upgrade or replace process, even if any of theconsecutive steps fail (e.g. due to a runtime error encountered while executingthe service’s main file or a syntax error in a required file not referencedfrom the manifest directly).