Dump file format

This description uses the same conventions as the protocol description.

The dump file format is not final and is subject to change before EdgeDB 1.0.

General Structure

Dump file is structure as follows:

  1. Dump file format marker \xFF\xD8\x00\x00\xD8EDGEDB\x00DUMP\x00 (17 bytes)

  2. Format version number \x00\x00\x00\x00\x00\x00\x00\x01 (8 bytes)

  3. Header block

  4. Any number of data blocks

General Dump Block

Both header and data blocks are formatted as follows:

  1. struct DumpHeader {
  2.     int8            mtype;
  4.     // SHA1 hash sum of block data
  5.     byte            sha1sum[20];
  7.     // Length of message contents in bytes,
  8.     // including self.
  9.     int32           message_length;
  11.     // Block data. Should be treated in opaque way by a client.
  12.     byte            data[message_length];
  13. }

Upon receiving a protocol dump data message, the dump client should:

  • Replace packet type:

    • @ (0x40) → H (0x48)

    • = (0x3d) → D (0x44)

  • Prepend SHA1 checksum to the block

  • Append the entire dump protocol message disregarding the first byte (the message type).

Header Block

Format:

  1. struct DumpHeader {
  2.     // Message type ('H')
  3.     int8            mtype = 0x48;
  5.     // SHA1 hash sum of block data
  6.     byte            sha1sum[20];
  8.     // Length of message contents in bytes,
  9.     // including self.
  10.     int32           message_length;
  12.     // A set of message headers.
  13.     Headers         headers;
  15.     // Protocol version of the dump
  16.     int16           major_ver;
  17.     int16           minor_ver;
  19.     // Schema data
  20.     string          schema_ddl;
  22.     // Type identifiers
  23.     int32           num_types;
  24.     TypeInfo        types[num_types];
  26.     // Object descriptors
  27.     int32           num_descriptors;
  28.     ObjectDesc      descriptors[num_descriptors]
  29. };
  31. struct TypeInfo {
  32.     string          type_name;
  33.     string          type_class;
  34.     byte            type_id[16];
  35. }
  37. struct ObjectDesc {
  38.     byte            object_id[16];
  39.     bytes           description;
  41.     int16           num_dependencies;
  42.     byte            dependency_id[num_dependencies][16];
  43. }

Known headers:

  • 101 BLOCK_TYPE – block type, always “I”

  • 102 SERVER_TIME – server time when dump is started as a floating point unix timestamp stringified

  • 103 SERVER_VERSION – full version of server as string

Data Block

Format:

  1. struct DumpBlock {
  2.     // Message type ('=')
  3.     int8            mtype = 0x3d;
  5.     // Length of message contents in bytes,
  6.     // including self.
  7.     int32           message_length;
  9.     // A set of message headers.
  10.     Headers         headers;
  11. }

Known headers:

  • 101 BLOCK_TYPE – block type, always “D”

  • 110 BLOCK_ID – block identifier (16 bytes of UUID)

  • 111 BLOCK_NUM – integer block index stringified

  • 112 BLOCK_DATA – the actual block data