3.3. Basic features

This section will enumerate a number of features that HAProxy implements, some
of which are generally expected from any modern load balancer, and some of
which are a direct benefit of HAProxy's architecture. More advanced features
will be detailed in the next section.

3.3.1. Basic features : Proxying

Proxying is the action of transferring data between a client and a server over
two independent connections. The following basic features are supported by
HAProxy regarding proxying and connection management :
 
  - Provide the server with a clean connection to protect them against any
    client-side defect or attack;
 
  - Listen to multiple IP addresses and/or ports, even port ranges;
 
  - Transparent accept : intercept traffic targeting any arbitrary IP address
    that doesn't even belong to the local system;
 
  - Server port doesn't need to be related to listening port, and may even be
    translated by a fixed offset (useful with ranges);
 
  - Transparent connect : spoof the client's (or any) IP address if needed
    when connecting to the server;
 
  - Provide a reliable return IP address to the servers in multi-site LBs;
 
  - Offload the server thanks to buffers and possibly short-lived connections
    to reduce their concurrent connection count and their memory footprint;
 
  - Optimize TCP stacks (e.g. SACK), congestion control, and reduce RTT impacts;
 
  - Support different protocol families on both sides (e.g. IPv4/IPv6/Unix);
 
  - Timeout enforcement : HAProxy supports multiple levels of timeouts depending
    on the stage the connection is, so that a dead client or server, or an
    attacker cannot be granted resources for too long;
 
  - Protocol validation: HTTP, SSL, or payload are inspected and invalid
    protocol elements are rejected, unless instructed to accept them anyway;
 
  - Policy enforcement : ensure that only what is allowed may be forwarded;
 
  - Both incoming and outgoing connections may be limited to certain network
    namespaces (Linux only), making it easy to build a cross-container,
    multi-tenant load balancer;
 
  - PROXY protocol presents the client's IP address to the server even for
    non-HTTP traffic. This is an HAProxy extension that was adopted by a number
    of third-party products by now, at least these ones at the time of writing :
      - client : haproxy, stud, stunnel, exaproxy, ELB, squid
      - server : haproxy, stud, postfix, exim, nginx, squid, node.js, varnish

3.3.2. Basic features : SSL

HAProxy's SSL stack is recognized as one of the most featureful according to
Google's engineers (http://istlsfastyet.com/). The most commonly used features
making it quite complete are :
 
  - SNI-based multi-hosting with no limit on sites count and focus on
    performance. At least one deployment is known for running 50000 domains
    with their respective certificates;
 
  - support for wildcard certificates reduces the need for many certificates ;
 
  - certificate-based client authentication with configurable policies on
    failure to present a valid certificate. This allows to present a different
    server farm to regenerate the client certificate for example;
 
  - authentication of the backend server ensures the backend server is the real
    one and not a man in the middle;
 
  - authentication with the backend server lets the backend server know it's
    really the expected haproxy node that is connecting to it;
 
  - TLS NPN and ALPN extensions make it possible to reliably offload SPDY/HTTP2
    connections and pass them in clear text to backend servers;
 
  - OCSP stapling further reduces first page load time by delivering inline an
    OCSP response when the client requests a Certificate Status Request;
 
  - Dynamic record sizing provides both high performance and low latency, and
    significantly reduces page load time by letting the browser start to fetch
    new objects while packets are still in flight;
 
  - permanent access to all relevant SSL/TLS layer information for logging,
    access control, reporting etc. These elements can be embedded into HTTP
    header or even as a PROXY protocol extension so that the offloaded server
    gets all the information it would have had if it performed the SSL
    termination itself.
 
  - Detect, log and block certain known attacks even on vulnerable SSL libs,
    such as the Heartbleed attack affecting certain versions of OpenSSL.
 
  - support for stateless session resumption (RFC 5077 TLS Ticket extension).
    TLS tickets can be updated from CLI which provides them means to implement
    Perfect Forward Secrecy by frequently rotating the tickets.

3.3.3. Basic features : Monitoring

HAProxy focuses a lot on availability. As such it cares about servers state,
and about reporting its own state to other network components :
 
  - Servers' state is continuously monitored using per-server parameters. This
    ensures the path to the server is operational for regular traffic;
 
  - Health checks support two hysteresis for up and down transitions in order
    to protect against state flapping;
 
  - Checks can be sent to a different address/port/protocol : this makes it
    easy to check a single service that is considered representative of multiple
    ones, for example the HTTPS port for an HTTP+HTTPS server.
 
  - Servers can track other servers and go down simultaneously : this ensures
    that servers hosting multiple services can fail atomically and that no one
    will be sent to a partially failed server;
 
  - Agents may be deployed on the server to monitor load and health : a server
    may be interested in reporting its load, operational status, administrative
    status independently from what health checks can see. By running a simple
    agent on the server, it's possible to consider the server's view of its own
    health in addition to the health checks validating the whole path;
 
  - Various check methods are available : TCP connect, HTTP request, SMTP hello,
    SSL hello, LDAP, SQL, Redis, send/expect scripts, all with/without SSL;
 
  - State change is notified in the logs and stats page with the failure reason
    (e.g. the HTTP response received at the moment the failure was detected). An
    e-mail can also be sent to a configurable address upon such a change ;
 
  - Server state is also reported on the stats interface and can be used to take
    routing decisions so that traffic may be sent to different farms depending
    on their sizes and/or health (e.g. loss of an inter-DC link);
 
  - HAProxy can use health check requests to pass information to the servers,
    such as their names, weight, the number of other servers in the farm etc.
    so that servers can adjust their response and decisions based on this
    knowledge (e.g. postpone backups to keep more CPU available);
 
  - Servers can use health checks to report more detailed state than just on/off
    (e.g. I would like to stop, please stop sending new visitors);
 
  - HAProxy itself can report its state to external components such as routers
    or other load balancers, allowing to build very complete multi-path and
    multi-layer infrastructures.

3.3.4. Basic features : High availability

Just like any serious load balancer, HAProxy cares a lot about availability to
ensure the best global service continuity :
 
  - Only valid servers are used ; the other ones are automatically evicted from
    load balancing farms ; under certain conditions it is still possible to
    force to use them though;
 
  - Support for a graceful shutdown so that it is possible to take servers out
    of a farm without affecting any connection;
 
  - Backup servers are automatically used when active servers are down and
    replace them so that sessions are not lost when possible. This also allows
    to build multiple paths to reach the same server (e.g. multiple interfaces);
 
  - Ability to return a global failed status for a farm when too many servers
    are down. This, combined with the monitoring capabilities makes it possible
    for an upstream component to choose a different LB node for a given service;
 
  - Stateless design makes it easy to build clusters : by design, HAProxy does
    its best to ensure the highest service continuity without having to store
    information that could be lost in the event of a failure. This ensures that
    a takeover is the most seamless possible;
 
  - Integrates well with standard VRRP daemon keepalived : HAProxy easily tells
    keepalived about its state and copes very well with floating virtual IP
    addresses. Note: only use IP redundancy protocols (VRRP/CARP) over cluster-
    based solutions (Heartbeat, ...) as they're the ones offering the fastest,
    most seamless, and most reliable switchover.

3.3.5. Basic features : Load balancing

HAProxy offers a fairly complete set of load balancing features, most of which
are unfortunately not available in a number of other load balancing products :
 
  - no less than 9 load balancing algorithms are supported, some of which apply
    to input data to offer an infinite list of possibilities. The most common
    ones are round-robin (for short connections, pick each server in turn),
    leastconn (for long connections, pick the least recently used of the servers
    with the lowest connection count), source (for SSL farms or terminal server
    farms, the server directly depends on the client's source address), URI (for
    HTTP caches, the server directly depends on the HTTP URI), hdr (the server
    directly depends on the contents of a specific HTTP header field), first
    (for short-lived virtual machines, all connections are packed on the
    smallest possible subset of servers so that unused ones can be powered
    down);
 
  - all algorithms above support per-server weights so that it is possible to
    accommodate from different server generations in a farm, or direct a small
    fraction of the traffic to specific servers (debug mode, running the next
    version of the software, etc);
 
  - dynamic weights are supported for round-robin, leastconn and consistent
    hashing ; this allows server weights to be modified on the fly from the CLI
    or even by an agent running on the server;
 
  - slow-start is supported whenever a dynamic weight is supported; this allows
    a server to progressively take the traffic. This is an important feature
    for fragile application servers which require to compile classes at runtime
    as well as cold caches which need to fill up before being run at full
    throttle;
 
  - hashing can apply to various elements such as client's source address, URL
    components, query string element, header field values, POST parameter, RDP
    cookie;
 
  - consistent hashing protects server farms against massive redistribution when
    adding or removing servers in a farm. That's very important in large cache
    farms and it allows slow-start to be used to refill cold caches;
 
  - a number of internal metrics such as the number of connections per server,
    per backend, the amount of available connection slots in a backend etc makes
    it possible to build very advanced load balancing strategies.

3.3.6. Basic features : Stickiness

Application load balancing would be useless without stickiness. HAProxy provides
a fairly comprehensive set of possibilities to maintain a visitor on the same
server even across various events such as server addition/removal, down/up
cycles, and some methods are designed to be resistant to the distance between
multiple load balancing nodes in that they don't require any replication :
 
  - stickiness information can be individually matched and learned from
    different places if desired. For example a JSESSIONID cookie may be matched
    both in a cookie and in the URL. Up to 8 parallel sources can be learned at
    the same time and each of them may point to a different stick-table;
 
  - stickiness information can come from anything that can be seen within a
    request or response, including source address, TCP payload offset and
    length, HTTP query string elements, header field values, cookies, and so
    on.
 
  - stick-tables are replicated between all nodes in a multi-master fashion;
 
  - commonly used elements such as SSL-ID or RDP cookies (for TSE farms) are
    directly accessible to ease manipulation;
 
  - all sticking rules may be dynamically conditioned by ACLs;
 
  - it is possible to decide not to stick to certain servers, such as backup
    servers, so that when the nominal server comes back, it automatically takes
    the load back. This is often used in multi-path environments;
 
  - in HTTP it is often preferred not to learn anything and instead manipulate
    a cookie dedicated to stickiness. For this, it's possible to detect,
    rewrite, insert or prefix such a cookie to let the client remember what
    server was assigned;
 
  - the server may decide to change or clean the stickiness cookie on logout,
    so that leaving visitors are automatically unbound from the server;
 
  - using ACL-based rules it is also possible to selectively ignore or enforce
    stickiness regardless of the server's state; combined with advanced health
    checks, that helps admins verify that the server they're installing is up
    and running before presenting it to the whole world;
 
  - an innovative mechanism to set a maximum idle time and duration on cookies
    ensures that stickiness can be smoothly stopped on devices which are never
    closed (smartphones, TVs, home appliances) without having to store them on
    persistent storage;
 
  - multiple server entries may share the same stickiness keys so that
    stickiness is not lost in multi-path environments when one path goes down;
 
  - soft-stop ensures that only users with stickiness information will continue
    to reach the server they've been assigned to but no new users will go there.

3.3.7. Basic features : Sampling and converting information

HAProxy supports information sampling using a wide set of "sample fetch
functions". The principle is to extract pieces of information known as samples,
for immediate use. This is used for stickiness, to build conditions, to produce
information in logs or to enrich HTTP headers.
 
Samples can be fetched from various sources :
 
  - constants : integers, strings, IP addresses, binary blocks;
 
  - the process : date, environment variables, server/frontend/backend/process
    state, byte/connection counts/rates, queue length, random generator, ...
 
  - variables : per-session, per-request, per-response variables;
 
  - the client connection : source and destination addresses and ports, and all
    related statistics counters;
 
  - the SSL client session : protocol, version, algorithm, cipher, key size,
    session ID, all client and server certificate fields, certificate serial,
    SNI, ALPN, NPN, client support for certain extensions;
 
  - request and response buffers contents : arbitrary payload at offset/length,
    data length, RDP cookie, decoding of SSL hello type, decoding of TLS SNI;
 
  - HTTP (request and response) : method, URI, path, query string arguments,
    status code, headers values, positional header value, cookies, captures,
    authentication, body elements;
 
A sample may then pass through a number of operators known as "converters" to
experience some transformation. A converter consumes a sample and produces a
new one, possibly of a completely different type. For example, a converter may
be used to return only the integer length of the input string, or could turn a
string to upper case. Any arbitrary number of converters may be applied in
series to a sample before final use. Among all available sample converters, the
following ones are the most commonly used :
 
  - arithmetic and logic operators : they make it possible to perform advanced
    computation on input data, such as computing ratios, percentages or simply
    converting from one unit to another one;
 
  - IP address masks are useful when some addresses need to be grouped by larger
    networks;
 
  - data representation : URL-decode, base64, hex, JSON strings, hashing;
 
  - string conversion : extract substrings at fixed positions, fixed length,
    extract specific fields around certain delimiters, extract certain words,
    change case, apply regex-based substitution;
 
  - date conversion : convert to HTTP date format, convert local to UTC and
    conversely, add or remove offset;
 
  - lookup an entry in a stick table to find statistics or assigned server;
 
  - map-based key-to-value conversion from a file (mostly used for geolocation).

3.3.8. Basic features : Maps

Maps are a powerful type of converter consisting in loading a two-columns file
into memory at boot time, then looking up each input sample from the first
column and either returning the corresponding pattern on the second column if
the entry was found, or returning a default value. The output information also
being a sample, it can in turn experience other transformations including other
map lookups. Maps are most commonly used to translate the client's IP address
to an AS number or country code since they support a longest match for network
addresses but they can be used for various other purposes.
 
Part of their strength comes from being updatable on the fly either from the CLI
or from certain actions using other samples, making them capable of storing and
retrieving information between subsequent accesses. Another strength comes from
the binary tree based indexation which makes them extremely fast even when they
contain hundreds of thousands of entries, making geolocation very cheap and easy
to set up.

3.3.9. Basic features : ACLs and conditions

Most operations in HAProxy can be made conditional. Conditions are built by
combining multiple ACLs using logic operators (AND, OR, NOT). Each ACL is a
series of tests based on the following elements :
 
  - a sample fetch method to retrieve the element to test ;
 
  - an optional series of converters to transform the element ;
 
  - a list of patterns to match against ;
 
  - a matching method to indicate how to compare the patterns with the sample
 
For example, the sample may be taken from the HTTP "Host" header, it could then
be converted to lower case, then matched against a number of regex patterns
using the regex matching method.
 
Technically, ACLs are built on the same core as the maps, they share the exact
same internal structure, pattern matching methods and performance. The only real
difference is that instead of returning a sample, they only return "found" or
or "not found". In terms of usage, ACL patterns may be declared inline in the
configuration file and do not require their own file. ACLs may be named for ease
of use or to make configurations understandable. A named ACL may be declared
multiple times and it will evaluate all definitions in turn until one matches.
 
About 13 different pattern matching methods are provided, among which IP address
mask, integer ranges, substrings, regex. They work like functions, and just like
with any programming language, only what is needed is evaluated, so when a
condition involving an OR is already true, next ones are not evaluated, and
similarly when a condition involving an AND is already false, the rest of the
condition is not evaluated.
 
There is no practical limit to the number of declared ACLs, and a handful of
commonly used ones are provided. However experience has shown that setups using
a lot of named ACLs are quite hard to troubleshoot and that sometimes using
anonymous ACLs inline is easier as it requires less references out of the scope
being analyzed.

3.3.10. Basic features : Content switching

HAProxy implements a mechanism known as content-based switching. The principle
is that a connection or request arrives on a frontend, then the information
carried with this request or connection are processed, and at this point it is
possible to write ACLs-based conditions making use of these information to
decide what backend will process the request. Thus the traffic is directed to
one backend or another based on the request's contents. The most common example
consists in using the Host header and/or elements from the path (sub-directories
or file-name extensions) to decide whether an HTTP request targets a static
object or the application, and to route static objects traffic to a backend made
of fast and light servers, and all the remaining traffic to a more complex
application server, thus constituting a fine-grained virtual hosting solution.
This is quite convenient to make multiple technologies coexist as a more global
solution.
 
Another use case of content-switching consists in using different load balancing
algorithms depending on various criteria. A cache may use a URI hash while an
application would use round-robin.
 
Last but not least, it allows multiple customers to use a small share of a
common resource by enforcing per-backend (thus per-customer connection limits).
 
Content switching rules scale very well, though their performance may depend on
the number and complexity of the ACLs in use. But it is also possible to write
dynamic content switching rules where a sample value directly turns into a
backend name and without making use of ACLs at all. Such configurations have
been reported to work fine at least with 300000 backends in production.

3.3.11. Basic features : Stick-tables

Stick-tables are commonly used to store stickiness information, that is, to keep
a reference to the server a certain visitor was directed to. The key is then the
identifier associated with the visitor (its source address, the SSL ID of the
connection, an HTTP or RDP cookie, the customer number extracted from the URL or
from the payload, ...) and the stored value is then the server's identifier.
 
Stick tables may use 3 different types of samples for their keys : integers,
strings and addresses. Only one stick-table may be referenced in a proxy, and it
is designated everywhere with the proxy name. Up to 8 keys may be tracked in
parallel. The server identifier is committed during request or response
processing once both the key and the server are known.
 
Stick-table contents may be replicated in active-active mode with other HAProxy
nodes known as "peers" as well as with the new process during a reload operation
so that all load balancing nodes share the same information and take the same
routing decision if client's requests are spread over multiple nodes.
 
Since stick-tables are indexed on what allows to recognize a client, they are
often also used to store extra information such as per-client statistics. The
extra statistics take some extra space and need to be explicitly declared. The
type of statistics that may be stored includes the input and output bandwidth,
the number of concurrent connections, the connection rate and count over a
period, the amount and frequency of errors, some specific tags and counters,
etc. In order to support keeping such information without being forced to
stick to a given server, a special "tracking" feature is implemented and allows
to track up to 3 simultaneous keys from different tables at the same time
regardless of stickiness rules. Each stored statistics may be searched, dumped
and cleared from the CLI and adds to the live troubleshooting capabilities.
 
While this mechanism can be used to surclass a returning visitor or to adjust
the delivered quality of service depending on good or bad behavior, it is
mostly used to fight against service abuse and more generally DDoS as it allows
to build complex models to detect certain bad behaviors at a high processing
speed.

3.3.12. Basic features : Formatted strings

There are many places where HAProxy needs to manipulate character strings, such
as logs, redirects, header additions, and so on. In order to provide the
greatest flexibility, the notion of Formatted strings was introduced, initially
for logging purposes, which explains why it's still called "log-format". These
strings contain escape characters allowing to introduce various dynamic data
including variables and sample fetch expressions into strings, and even to
adjust the encoding while the result is being turned into a string (for example,
adding quotes). This provides a powerful way to build header contents or to
customize log lines. Additionally, in order to remain simple to build most
common strings, about 50 special tags are provided as shortcuts for information
commonly used in logs.

3.3.13. Basic features : HTTP rewriting and redirection

Installing a load balancer in front of an application that was never designed
for this can be a challenging task without the proper tools. One of the most
commonly requested operation in this case is to adjust requests and response
headers to make the load balancer appear as the origin server and to fix hard
coded information. This comes with changing the path in requests (which is
strongly advised against), modifying Host header field, modifying the Location
response header field for redirects, modifying the path and domain attribute
for cookies, and so on. It also happens that a number of servers are somewhat
verbose and tend to leak too much information in the response, making them more
vulnerable to targeted attacks. While it's theoretically not the role of a load
balancer to clean this up, in practice it's located at the best place in the
infrastructure to guarantee that everything is cleaned up.
 
Similarly, sometimes the load balancer will have to intercept some requests and
respond with a redirect to a new target URL. While some people tend to confuse
redirects and rewriting, these are two completely different concepts, since the
rewriting makes the client and the server see different things (and disagree on
the location of the page being visited) while redirects ask the client to visit
the new URL so that it sees the same location as the server.
 
In order to do this, HAProxy supports various possibilities for rewriting and
redirects, among which :
 
  - regex-based URL and header rewriting in requests and responses. Regex are
    the most commonly used tool to modify header values since they're easy to
    manipulate and well understood;
 
  - headers may also be appended, deleted or replaced based on formatted strings
    so that it is possible to pass information there (e.g. client side TLS
    algorithm and cipher);
 
  - HTTP redirects can use any 3xx code to a relative, absolute, or completely
    dynamic (formatted string) URI;
 
  - HTTP redirects also support some extra options such as setting or clearing
    a specific cookie, dropping the query string, appending a slash if missing,
    and so on;
 
  - all operations support ACL-based conditions;

3.3.14. Basic features : Server protection

HAProxy does a lot to maximize service availability, and for this it takes
large efforts to protect servers against overloading and attacks. The first
and most important point is that only complete and valid requests are forwarded
to the servers. The initial reason is that HAProxy needs to find the protocol
elements it needs to stay synchronized with the byte stream, and the second
reason is that until the request is complete, there is no way to know if some
elements will change its semantics. The direct benefit from this is that servers
are not exposed to invalid or incomplete requests. This is a very effective
protection against slowloris attacks, which have almost no impact on HAProxy.
 
Another important point is that HAProxy contains buffers to store requests and
responses, and that by only sending a request to a server when it's complete and
by reading the whole response very quickly from the local network, the server
side connection is used for a very short time and this preserves server
resources as much as possible.
 
A direct extension to this is that HAProxy can artificially limit the number of
concurrent connections or outstanding requests to a server, which guarantees
that the server will never be overloaded even if it continuously runs at 100% of
its capacity during traffic spikes. All excess requests will simply be queued to
be processed when one slot is released. In the end, this huge resource savings
most often ensures so much better server response times that it ends up actually
being faster than by overloading the server. Queued requests may be redispatched
to other servers, or even aborted in queue when the client aborts, which also
protects the servers against the "reload effect", where each click on "reload"
by a visitor on a slow-loading page usually induces a new request and maintains
the server in an overloaded state.
 
The slow-start mechanism also protects restarting servers against high traffic
levels while they're still finalizing their startup or compiling some classes.
 
Regarding the protocol-level protection, it is possible to relax the HTTP parser
to accept non standard-compliant but harmless requests or responses and even to
fix them. This allows bogus applications to be accessible while a fix is being
developed. In parallel, offending messages are completely captured with a
detailed report that help developers spot the issue in the application. The most
dangerous protocol violations are properly detected and dealt with and fixed.
For example malformed requests or responses with two Content-length headers are
either fixed if the values are exactly the same, or rejected if they differ,
since it becomes a security problem. Protocol inspection is not limited to HTTP,
it is also available for other protocols like TLS or RDP.
 
When a protocol violation or attack is detected, there are various options to
respond to the user, such as returning the common "HTTP 400 bad request",
closing the connection with a TCP reset, or faking an error after a long delay
("tarpit") to confuse the attacker. All of these contribute to protecting the
servers by discouraging the offending client from pursuing an attack that
becomes very expensive to maintain.
 
HAProxy also proposes some more advanced options to protect against accidental
data leaks and session crossing. Not only it can log suspicious server responses
but it will also log and optionally block a response which might affect a given
visitors' confidentiality. One such example is a cacheable cookie appearing in a
cacheable response and which may result in an intermediary cache to deliver it
to another visitor, causing an accidental session sharing.

3.3.15. Basic features : Logging

Logging is an extremely important feature for a load balancer, first because a
load balancer is often wrongly accused of causing the problems it reveals, and
second because it is placed at a critical point in an infrastructure where all
normal and abnormal activity needs to be analyzed and correlated with other
components.
 
HAProxy provides very detailed logs, with millisecond accuracy and the exact
connection accept time that can be searched in firewalls logs (e.g. for NAT
correlation). By default, TCP and HTTP logs are quite detailed an contain
everything needed for troubleshooting, such as source IP address and port,
frontend, backend, server, timers (request receipt duration, queue duration,
connection setup time, response headers time, data transfer time), global
process state, connection counts, queue status, retries count, detailed
stickiness actions and disconnect reasons, header captures with a safe output
encoding. It is then possible to extend or replace this format to include any
sampled data, variables, captures, resulting in very detailed information. For
example it is possible to log the number of cumulative requests or number of
different URLs visited by a client.
 
The log level may be adjusted per request using standard ACLs, so it is possible
to automatically silent some logs considered as pollution and instead raise
warnings when some abnormal behavior happen for a small part of the traffic
(e.g. too many URLs or HTTP errors for a source address). Administrative logs
are also emitted with their own levels to inform about the loss or recovery of a
server for example.
 
Each frontend and backend may use multiple independent log outputs, which eases
multi-tenancy. Logs are preferably sent over UDP, maybe JSON-encoded, and are
truncated after a configurable line length in order to guarantee delivery.

3.3.16. Basic features : Statistics

HAProxy provides a web-based statistics reporting interface with authentication,
security levels and scopes. It is thus possible to provide each hosted customer
with his own page showing only his own instances. This page can be located in a
hidden URL part of the regular web site so that no new port needs to be opened.
This page may also report the availability of other HAProxy nodes so that it is
easy to spot if everything works as expected at a glance. The view is synthetic
with a lot of details accessible (such as error causes, last access and last
change duration, etc), which are also accessible as a CSV table that other tools
may import to draw graphs. The page may self-refresh to be used as a monitoring
page on a large display. In administration mode, the page also allows to change
server state to ease maintenance operations.