... h2. Definitions
This part of the specification can be read in the context of any high-level language for which a Fudge implementation exists. Terminology can vary between languages so we will use terms as defined below.
|| Our term || Definition ||
| field | A named attribute, member of property within an object. A field holds a value. Depending on the language it may hold a reference, pointer, or content of another object |
| object | A data construct containing a group of fields. Objects may contain no fields. Objects may contain other objects. |
| reference | Any field is a reference if it is a way of directly or indirectly accessing the containing object, or another object's fields. Depending on the language it may be a pointer, a handle, or the full content of the referenced language. |
| type | A set of fields that objects of that type will hold. A class in C# or Java languages. |
h2. Basic object representation
Every object is represented by a Fudge message. The message field with ordinal 0, or labelled with the empty string, is used to indicate the object type. Object fields are represented by further message fields.
|| Field || Representation \\ ||
|
...
A reference to another object is written as a sub-message. The sub-message contains either the content of that object, or identifies an object previously serialized on the same stream. Identifying a previously serialized object is necessary to process cyclic object reference graphs, and avoid data duplication at the receiver when the source graph contains objects that are referenced multiple times.
When serialising a set of objects, the first encounter of an object will require it be written as a message or sub-message containing all of the object's fields. If that object is subsequently encountered, the message or sub-message contains the identifier of the message used to encode it previously. The encoding and decoding applications must maintain a list of previously encoded and decoded objects for the stream. These are identified by a zero based index from the object currently being processed. I.e. an identifier of 0 is a reference to the current object and 1 is the previously processed object. After being processed, the referenced object is moved from its location in the list to the head of the list. For example, in a fully connected graph of 3 objects - A, B, C where A = (B, C), B = (A, C), C = (B, A):
Object A could be serialised as:
message A - message containing all of object A's fields. Buffer contains \{ A \}.
message B - sub-message containing all of object B's fields. Buffer now contains \{ B, A \}.
message A - index of A in the buffer which is 1 as buffer contains \{ B, A \}. Buffer becomes \{ A, B \}.
message C - sub-message containing all of object C's fields. Buffer contains \{ C, A, B \}.
message B - index of B in the buffer which is 2. Buffer becomes \{ B, C, A \}.
message A - index of A in the buffer which is 2. Buffer becomes \{ A, B, C \}.
message C - index of C in the buffer which is 2. Buffer becomes \{ C, A, B \}.
Moving a referenced object to the head of the buffer after sending it is to keep the integral values low for objects that are heavily referenced as these will be reduced to the most compact encoding.
When encoding a field as an identifier of a previously sent (or started) object, the type (field labelled with the empty string or ordinal 0) is used to encode the identifier by specifying it as an integer.
h3. Types and polymorphism
An object's type is a textual string coming from an application defined namespace that is case sensitive. The type must contain only alphanumeric, '_' and '.' characters, starting with a letter. These characters are either valid within most high level languages or a simple mapping can convert to and from valid type names for the language.
The first instance of the type field is the preferred type for the object. Subsequent values for the type field within the message are alternative, ancestral, types from the inheritance tree. An implementation must decode the message to an object of the first type that it recognises. The object should retain the original type details and any additional fields so that data is preserved if it is reserialised.
To produce a more efficient stream encoding, the type of a previously encoded object can be used instead of the full strings. Using the same buffer used to process object references, the identifier of an earlier object with the same type can be given. This can be done by specifying the type as an integer field with a negative value. A value of -1 means the type of whichever object is at index 1 from the head of the buffer. In the event of there being multiple objects in the buffer with a matching type, the one closest to the head must be used so that the integral encoding is smallest.
h2. Layout of serialisation stream
A serialisation stream is itself just a Fudge message, with the following fields:
|| Name || Type\\ || Purpose\\ ||
| version\\ | integer\\ | Version of the Fudge serialisation framework\\ |
| object\\ | message\\ | Repeated field for one or more serialised objects.\\ |
The stream may contain a single object (with the graph of any objects it references), or a sequence of objects.
h3. Example
As an example. assuming we have a class Person, which contains a name, a list of siblings, and an optional Address. If we serialise a brother-sister combo (starting with "Bob") we get a serialisation stream that looks like:
{code}
version : byte = 0
object : FudgeMsg =
0 : string = "Person"
name : string = "Bob"
siblings : FudgeMsg = // Begin inline encoding of the "Shirly" object
0 : byte = -1 // Same type as the "Bob" object
name : string = "Shirly"
siblings : FudgeMsg =
0 : byte = 1 // Reference to the "Bob" object
address : FudgeMsg =
0 : string = "Address"
line1 : string = "Our house"
line2 : string = "In the middle of our street"
{code}
h2. Evolvability of data
One of the annoying (read costly) things about distributed systems is when you have to upgrade all parts simultaneously because a message structure changes. Even worse if you don't actually own all the pieces. Evolvability goes some way to alleviate this pain.
We do this:
* Evolvable messages have a version (integer). This is passed in the serialisation stream.
* Messages can be evolved by adding new fields, and increasing the version number. Types of existing fields should not be modified unless there is an automatic conversion between the types.
* Objects into which we are deserialising data are coded against a specific version, and know what version this is.
* If an object receives a message with an _older_ version, it is responsible for upgrading the data with sensible defaults if possible. This allows an older client to send data to a newer server (or vice-versa), decoupling the version dependency.
* If an object receives a message with a _newer_ version, it stores the additional fields and adds them back when later reserialising. This allows a service in the middle to pass through future data without needing modification.
In the last scenario, we need to decide which of two strategies to use for the version of the data:
* The data keeps the future version number. This has the down-side that each message will have to carry the version number, increasing data bloat.
* The data reverts back to the service's version number even though it carries the future fields. This means that the version for all instances of a type will be the same, and also allows the (future) recipient to be aware that it has been processed by an older intermediary (so relationships between the old and new fields may need to be reenforced).
My vote at the moment is for the latter.
|
