Serialization (encode/decode)

When a structure is sent over the network or written to disk, it isencoded into a string of bytes. Serializable structures haveencode and decode methods that write and read from bufferlistobjects representing byte strings.

Adding a field to a structure

You can see examples of this all over the Ceph code, but here’s anexample:

  1. class AcmeClass
  2. {
  3.     int member1;
  4.     std::string member2;
  5.  
  6.     void encode(bufferlist &bl)
  7.     {
  8.         ENCODE_START(1, 1, bl);
  9.         ::encode(member1, bl);
  10.         ::encode(member2, bl);
  11.         ENCODE_FINISH(bl);
  12.     }
  13.  
  14.     void decode(bufferlist::iterator &bl)
  15.     {
  16.         DECODE_START(1, bl);
  17.         ::decode(member1, bl);
  18.         ::decode(member2, bl);
  19.         DECODE_FINISH(bl);
  20.     }
  21. };

The ENCODESTART macro writes a header that specifies a _version anda compat_version (both initially 1). The message version is incrementedwhenever a change is made to the encoding. The compat_version is incrementedonly if the change will break existing decoders – decoders are tolerantof trailing bytes, so changes that add fields at the end of the structuredo not require incrementing compat_version.

The DECODE_START macro takes an argument specifying the most recentmessage version that the code can handle. This is compared with thecompat_version encoded in the message, and if the message is too new thenan exception will be thrown. Because changes to compat_verison are rare,this isn’t usually something to worry about when adding fields.

In practice, changes to encoding usually involve simply adding the desired fieldsat the end of the encode and decode functions, and incrementingthe versions in ENCODE_START and DECODE_START. For example, here’s howto add a third field to AcmeClass:

  1. class AcmeClass
  2. {
  3.     int member1;
  4.     std::string member2;
  5.     std::vector<std::string> member3;
  6.  
  7.     void encode(bufferlist &bl)
  8.     {
  9.         ENCODE_START(2, 1, bl);
  10.         ::encode(member1, bl);
  11.         ::encode(member2, bl);
  12.         ::encode(member3, bl);
  13.         ENCODE_FINISH(bl);
  14.     }
  15.  
  16.     void decode(bufferlist::iterator &bl)
  17.     {
  18.         DECODE_START(2, bl);
  19.         ::decode(member1, bl);
  20.         ::decode(member2, bl);
  21.         if (struct_v >= 2) {
  22.             ::decode(member3, bl);
  23.         }
  24.         DECODE_FINISH(bl);
  25.     }
  26. };

Note that the compat_version did not change because the encoded messagewill still be decodable by versions of the code that only understandversion 1 – they will just ignore the trailing bytes where we encode member3.

In the decode function, decoding the new field is conditional: this isbecause we might still be passed older-versioned messages that do nothave the field. The struct_v variable is a local set by the DECODE_STARTmacro.