

Messages

At their lowest, messages on Kafka are represented by bytes. However, the current version of Dafda only supports string representations.

In order to send message on Kafka the following is "required":

Data/info	Comment
Topic	The name of the Kafka topic
Partition Key	A unique key that Kafka uses to guarantee ordering, amongst other
Message	The actual message to be transported

Dafda adds a layer on top using a Message Envelope.

Partition Keys

Like the message body format, the partition keys are also represented as string.

Message Envelope

Messages are by default wrapped in a message envelope, which makes the consuming and producing of makes straight forward.

Given the following C# snippet:

var someMessage = new SomeMessage
{
    SomeId = Guid.CreateNew(),
    SomeData = "This is very important data"
}

producer.Produce(someMessage);

Dafda's default (out-of-the-box) implementation comes with a message envelope that is serialized/deserialized from JSON, like:

{
    "messageId": "100bee5a-82af-4003-82a2-fa6e543de24f",
    "type": "some-message",
    "data": {
        "someId": "538b7db6-54c0-4115-ab6d-583d9714a289",
        "someData": "This is very important data"
    }
}

Message Id

The message id is by default a GUID generated by Dafda, but can be overriden in the Producer Configuration.

Message Headers

Currently message headers are embedded in the message envelope, and not using the available Kafka message header, which is more in tune with the CloudEvents specification. Any message header added will, using the default serializer, be at the same level as the messageId, type and data properties as in the listing above.

Thus the following C# snippet:

var headers = new Dictionary<string, object>
{
    ["some-header"] = "some-value",
    ["another-header"] = "another-value"
};

var someMessage = new SomeMessage
{
    SomeId = Guid.CreateNew(),
    SomeData = "This is very important data"
}

producer.Produce(someMessage, headers);

Will yield the following JSON:

{
    "messageId": "100bee5a-82af-4003-82a2-fa6e543de24f",
    "type": "some-message",
    "some-header": "some-value",
    "another-header": "another-value",
    "data": {
        "someId": "538b7db6-54c0-4115-ab6d-583d9714a289",
        "someData": "This is very important data"
    }
}