Service Naming

Understanding Mesos-DNS service naming conventions

Mesos-DNS defines the DNS top-level domain .mesos for Mesos tasks that are running on DC/OS. Tasks and services are discovered by looking up A and, optionally, SRV records within this Mesos domain.

A Records

An A record associates a hostname to an IP address. When a task is launched by a DC/OS service, Mesos-DNS generates an A record for a hostname in the format <task>.<service>.mesos that provides one of the following:

  • The IP address of the agent node that is running the task
  • The IP address of the task’s network container (provided by a Mesos containerizer)

For example, other DC/OS tasks can discover the IP address for a task named search launched by the marathon service with a lookup for search.marathon.mesos:

  1. dig search.marathon.mesos
  3. ; <<>> DiG 9.8.4-rpz2+rl005.12-P1 <<>> search.marathon.mesos
  4. ;; global options: +cmd
  5. ;; Got answer:
  6. ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24471
  7. ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 0
  9. ;; QUESTION SECTION:
  10. ;search.marathon.mesos.         IN  A
  12. ;; ANSWER SECTION:
  13. search.marathon.mesos.      60  IN  A   10.9.87.94

If the Mesos containerizer that launches the task provides a container IP 10.0.4.1 for the task search.marathon.mesos, then the lookup result is:

  1. dig search.marathon.mesos
  3. ; <<>> DiG 9.8.4-rpz2+rl005.12-P1 <<>> search.marathon.mesos
  4. ;; global options: +cmd
  5. ;; Got answer:
  6. ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24471
  7. ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 0
  9. ;; QUESTION SECTION:
  10. ;search.marathon.mesos.         IN  A
  12. ;; ANSWER SECTION:
  13. search.marathon.mesos.      60  IN  A   10.0.4.1

In addition to the <task>.<service>.mesos syntax shown above, Mesos-DNS also generates A records that contain the IP addresses of the agent nodes that are running the task: <task>.<service>.slave.mesos.

For example, a query of the A records for search.marathon.slave.mesos shows the IP address of each agent node running one or more instances of the search application on the marathon service.

SRV Records

An SRV record specifies the hostname and port of a service.

For a task named mytask launched by a service named myservice, Mesos-DNS generates an SRV record _mytask._protocol.myservice.mesos, where protocol is udp or tcp. For example, other Mesos tasks can discover a task named search launched by the marathon service with a query for _search._tcp.marathon.mesos:

  1. dig _search._tcp.marathon.mesos SRV
  3. ;  DiG 9.8.4-rpz2+rl005.12-P1 &lt;&lt;&gt;&gt; _search._tcp.marathon.mesos SRV
  4. ;; global options: +cmd
  5. ;; Got answer:
  6. ;; -&gt;&gt;HEADER&lt;&lt;- opcode: QUERY, status: NOERROR, id: 33793
  7. ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
  9. ;; QUESTION SECTION:
  10. ;_search._tcp.marathon.mesos.   IN SRV
  12. ;; ANSWER SECTION:
  13. _search._tcp.marathon.mesos.    60 IN SRV 0 0 31302 10.254.132.41.

Mesos-DNS supports the use of a task’s DiscoveryInfo for SRV record generation. On a DC/OS cluster, ports are offered by agent nodes in the same way as other resources such as CPU and memory. If DiscoveryInfo is not available, Mesos-DNS uses the ports that were allocated for the task.

The following table shows the rules that govern SRV generation:

ServiceContainer IP KnownDiscoveryInfo ProvidedTarget HostTarget PortA Record Target IP
_mytask._protocol.myservice.mesosNoNomytask.myservice.slave.mesosHost PortAgent IP
_mytask._protocol.myservice.mesosYesNomytask.myservice.slave.mesosHost PortAgent IP
_mytask._protocol.myservice.mesosNoYesmytask.myservice.mesosDiscoveryInfo PortAgent IP
_mytask._protocol.myservice.mesosYesYesmytask.myservice.mesosDiscoveryInfo PortContainer IP
mytask.protocol.myservice.slave.mesosN/AN/Amytask.myservice.slave.mesosHost PortAgent IP

Table 1. SRV generation rules

Other Records

Mesos-DNS generates a few special records:

  • For the leading master: A record (leader.mesos) and SRV records (_leader._tcp.mesos and _leader._udp.mesos)
  • For all service schedulers: A records (myservice.mesos) and SRV records (_myservice._tcp.myservice.mesos)
  • For every known DC/OS master: A records (master.mesos)
  • For every known DC/OS agent: A records (slave.mesos) and SRV records (_slave._tcp.mesos)

IMPORTANT: To query the leading master node, always query “leader.mesos”, not “master.mesos”. See this FAQ entry for more information.

There is a delay between the election of a new master and the update of leader/master records in Mesos-DNS. Mesos-DNS also supports requests for SOA and NS records for the Mesos domain. DNS requests for records of other types in the Mesos domain will return NXDOMAIN. Mesos-DNS does not support PTR records needed for reverse lookups. Mesos-DNS also generates A records for itself that list all the IP addresses that Mesos-DNS will answer lookup requests on. The hostname for these A records is ns1.mesos.

Task and Service Naming Conventions

Mesos-DNS follows RFC 1123 for name formatting. All fields used to construct hostnames for A records and service names for SRV records must be 63 characters or shorter and can include letters of the alphabet (A-Z), numbers (0-9), and a dash (-). Names are not case sensitive. If the task name does not comply with these constraints, Mesos-DNS will shorten the name to 24 characters, remove all invalid characters, and replace periods (.) with a dash (-). For Mesos DNS names, enforcement of RFC 952 is optional.

Note that there is a difference in the rules for service names and task names. For service names, periods (.) are allowed, but all other rules apply. For example, a task named apiserver.myservice launched by service marathon.prod will have A records associated with the name apiserver-myservice.marathon.prod.mesos and SRV records associated with the name _apiserver-myservice._tcp.marathon.prod.mesos.

Some services register with default names that are difficult to understand. For example, older versions of Marathon may register with names such as marathon-0.7.5, which will lead to a Mesos-DNS hostname such as search.marathon-0.7.5.mesos. You can avoid this problem by launching services with customized names. For example, launch Marathon with --framework_name marathon to register the service as marathon.

If you are using Marathon groups, the Mesos-DNS hostname is created from the app ID. For example, if you have an app named nginx-router and it is within the mesosphere-tutorial group with an app ID of /mesosphere-tutorial/nginx-router, then the DNS name will be nginx-router-mesosphere-.marathon.mesos. Note that Mesos-DNS truncated the hostname to 24 characters and substituted a dash for the slash between mesosphere-tutorial and nginx-router.

If a service launches multiple tasks with the same name, the DNS lookup will return multiple records, one per task. Mesos-DNS randomly shuffles the order of records to provide rudimentary load balancing between these tasks.

WARNING: It is possible to have a name collision if different services launch tasks that have the same hostname. If different services launch tasks with identical Mesos-DNS hostnames, or if Mesos-DNS truncates app IDs to create identical Mesos-DNS hostnames, applications will communicate with the wrong agent nodes and fail unpredictably.

Discovering the DNS names for a service

You can get a comprehensive list of the apps running on your DC/OS cluster nodes.

Prerequisites: DC/OS and DC/OS CLI are installed.

  1. SSH into your node. For example, use this CLI command to SSH to your master:

    1. dcos node ssh --leader --master-proxy

    For more information, see the SSH documentation.

  2. Run this command from your master node to view the node details:

    1. curl -H "Authorization: token=<auth-token>" http://<master-ip>/mesos_dns/v1/enumerate

    In this example, Kafka and Chronos are installed:

    1.    curl -H "Authorization: token=<auth-token>" http://<master-ip>/mesos_dns/v1/enumerate
    2.      {
    3.        "frameworks": [
    4.         {
    5.          "tasks": null,
    6.          "name": "chronos"
    7.         },
    8.         {
    9.          "tasks": null,
    10.          "name": "kafka"
    11.         },
    12.         {
    13.          "tasks": [
    14.           {
    15.            "name": "kafka",
    16.            "id": "kafka.443d5d63-f527-11e5-81a5-2a8c0aaf83b5",
    17.            "records": [
    18.             {
    19.              "name": "kafka.marathon.mesos.",
    20.              "host": "10.0.2.162",
    21.              "rtype": "A"
    22.             },
    23.             {
    24.              "name": "kafka-7fdws-s0.marathon.mesos.",
    25.              "host": "10.0.2.162",
    26.              "rtype": "A"
    27.             },
    28.             {
    29.              "name": "kafka.marathon.slave.mesos.",
    30.              "host": "10.0.2.162",
    31.              "rtype": "A"
    32.             },
    33.             {
    34.              "name": "kafka-7fdws-s0.marathon.slave.mesos.",
    35.              "host": "10.0.2.162",
    36.              "rtype": "A"
    37.             },
    38.             {
    39.              "name": "_kafka._tcp.marathon.slave.mesos.",
    40.              "host": "kafka-7fdws-s0.marathon.slave.mesos.:14799",
    41.              "rtype": "SRV"
    42.             },
    43.             {
    44.              "name": "_kafka._udp.marathon.slave.mesos.",
    45.              "host": "kafka-7fdws-s0.marathon.slave.mesos.:14799",
    46.              "rtype": "SRV"
    47.             },
    48.             {
    49.              "name": "_kafka._tcp.marathon.mesos.",
    50.              "host": "kafka-7fdws-s0.marathon.mesos.:14799",
    51.              "rtype": "SRV"
    52.             }
    53.            ]
    54.           },
    55.           {
    56.            "name": "chronos",
    57.            "id": "chronos.40a4f462-f527-11e5-81a5-2a8c0aaf83b5",
    58.            "records": [
    59.             {
    60.              "name": "chronos.marathon.mesos.",
    61.              "host": "10.0.2.162",
    62.              "rtype": "A"
    63.             },
    64.             {
    65.              "name": "chronos-4dj75-s0.marathon.mesos.",
    66.              "host": "10.0.2.162",
    67.              "rtype": "A"
    68.             },
    69.             {
    70.              "name": "chronos.marathon.slave.mesos.",
    71.              "host": "10.0.2.162",
    72.              "rtype": "A"
    73.             },
    74.             {
    75.              "name": "chronos-4dj75-s0.marathon.slave.mesos.",
    76.              "host": "10.0.2.162",
    77.              "rtype": "A"
    78.             },
    79.             {
    80.              "name": "_chronos._tcp.marathon.slave.mesos.",
    81.              "host": "chronos-4dj75-s0.marathon.slave.mesos.:9106",
    82.              "rtype": "SRV"
    83.             },
    84.             {
    85.              "name": "_chronos._udp.marathon.slave.mesos.",
    86.              "host": "chronos-4dj75-s0.marathon.slave.mesos.:9106",
    87.              "rtype": "SRV"
    88.             },
    89.             {
    90.              "name": "_chronos._tcp.marathon.mesos.",
    91.              "host": "chronos-4dj75-s0.marathon.mesos.:9106",
    92.              "rtype": "SRV"
    93.             }
    94.            ]
    95.           }
    96.          ],
    97.          "name": "marathon"
    98.         }
    99.        ]