6. Message Formats

See also

GitHub: RHInception/re-common
The re-common repository holds JSON Schema files which can be used for message validation.

6.1. re-rest ↔ re-core

re-rest produces two formats: Deployment Message Format and Notification Message Format. It also receives on message format in return: Deployment Response Message Format.

6.1.1. Deployment Message Format

The Deployment Message format comes in two flavors, Simple and Dynamic.

6.1.1.1. Simple Deployment Message Format

The simple format has two required keys: group and playbook_id.

Required Keys:

  • group: the group’s name as a string
  • playbook_id: the specific playbook in the group to utilize

Example:

{
   "group": "My Group",
   "playbook_id": "1234567890"
}

6.1.1.2. Dynamic Deployment Message Format

The dynamic format adds a key to the simple format: dynamic.

Note

For more information on dynamic variables see Dynamic Variables.

Required Keys:

  • group: the group’s name as a string
  • playbook_id: the specific playbook in the group to utilize
  • dynamic: a JSON object holding key/values for specific workers

Example:

{
   "group": "My Group",
   "playbook_id": "1234567890",
   "dynamic": {
       "cart": "my cart",
       "environment": "QA"
   }
}

6.1.1.3. Deployment Response Message Format

This format is sent from re-core back to re-rest which tells re-rest if the deployment was accepted or not. The message has one key: id. If the deployment was able to start the id will have a deployment id in it. If there was an issue starting a deployment, for example, because the group or playbook doesn’t exist, the id will be null.

Required Keys:

  • id: the deployment id or null

Example of a Deployment Successfully Starting:

{
   "id": "9999999999999999"
}

Example of a Deployment Failing to Start:

{
   "id": null
}

6.2. re-core ↔ re-workers

While executing releases, the re-core component emits messages in one of three formats. The format selected is determined by the type of execution step being ran.

Simple
For execution steps which require no arguments
Argument
For execution steps which require defined
Dynamic
For execution steps which require dynamic arguments

In the rest of this section we will find that these three formats are very similar in structure. Each section will highlight the elements which make the respective format unique from the others.

6.2.1. Deployment Message Formats

This section describes the message formats emitted by re-core to workers.

6.2.1.1. Simple Message Format

On line 8 in the following playbook we can see there is only one step to execute, bigip:OutOfRotation. From how the step is written, as a simple string, we know that this step requires no arguments. That is to say, we do not need to define any parameters in the playbook or provide any dynamic data when starting the release.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
{
    "name": "docs test",
    "group": "test",
    "execution": [
        {
            "description": "sequence 0",
            "hosts": [ "host01.example.com" ],
            "steps": [ "bigip:OutOfRotation" ]
        }
    ]
}

For an execution step like this (line 8), re-core selects the simple message format. The following example shows the format of the message which it would emit to the bus.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
{
    "parameters": {
        "hosts": [ "host01.example.com" ],
        "command": "bigip",
        "subcommand": "OutOfRotation"
    },
    "group": "test",
    "dynamic": {},
    "notify": {}
}

This is the simplest message format used by re-core when interacting with simple workers. As such, we can see that it’s quite terse. The Argument and Dynamic message formats use this same structure, however they fill in different items.

On line 2 we have the parameters item. This holds the basic information required for this worker to complete its job:

hosts
An array of hosts to apply the step to
command
The name of the worker being utilized
subcommand
The specific action the worker should take

Additionally the parameters item has two sibblings: group and dynamic. These items are always sent to the worker, even if (as in this example), there is no dynamic data to send.

6.2.1.2. Argument Message Format

On line 10 in the following playbook we can see there is only one step to execute, service:Restart. From how the step is written, as a dictionary, we know that this step requires one argument, service which is defined as megafrobber (line 11).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
{
    "name": "docs test",
    "group": "test",
    "execution": [
        {
            "description": "sequence 0",
            "hosts": [ "host01.example.com" ],
            "steps": [
                {
                    "service:Restart": {
                        "service": "megafrobber"
                    }
                }
            ]
        }
    ]
}

For an execution step like this (line 10), re-core selects the argument message format. The following example shows the format of the message which it would emit to the bus.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
{
    "parameters": {
        "service": "megafrobber",
        "hosts": [
            "host01.example.com"
        ],
        "subcommand": "Restart",
        "command": "service"
    },
    "group": "test",
    "dynamic": {},
    "notify": {}
}

What makes this message format different from the previous format is the presence of an additional key in the parameters item. That key is service (line 3). This comes directly from line 11 in the example playbook.

6.2.1.3. Dynamic Message Format

Still referencing the previous playbook (Argument Message Format), let’s add an execution step which requires dynamic arguments (this example only shows the additional step).

1
2
3
4
5
6
7
8
{
    "juicer:Promote": {
        "dynamic": [
            "cart",
            "environment"
        ]
    }
}

See also

RE-WORKER-JUICER
Documentation for the re-worker-juicer worker

On line 2 we see that the execution step is called juicer:Promote. On the following line we see the dictionary key dynamic, and that it’s value is a list type. The items in the list (lines 45) indicate the required dynamic variables to run the step. This step requires two such variables, cart and environment. The user would supply the values for these variables when starting the release.

Note

For more information on dynamic variables see Dynamic Variables.

The following example shows the format of the message which re-core would emit to the bus.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
{
    "parameters": {
        "command": "juicer",
        "dynamic": [
            "cart",
            "environment"
        ],
        "subcommand": "Promote",
        "hosts": [
            "host01.example.com"
        ]
    },
    "group": "test",
    "dynamic": {
        "cart": "bitmath",
        "environment": "re"
    },
    "notify": {}
}

Here we see a familiar key appearing in the parameters item, dynamic.

Warning

In future releases, the dynamic key will not be copied to workers in the parameters item. It will only appear as a sibbling of the parameters item.

Now, different from the previous format (argument), we see the dynamic item (sibbling to parameters) contains actual key-values (lines 1416).

dynamic
A dictionary with the required dynamic variables for a worker to run. The type of each argument is dictated by the respective worker.

6.2.1.4. Per-Step Notifications

Auxiliary to the formats we’ve just discussed, are per-step notifications. Per-step notifications are an optional item which may be added to any given step (simple, argument, and dynamic). Using the previous example playbook for reference, we would see notifications defined as in lines 68, below:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
{
    "steps": [
        {
            "service:Restart": {
                "service": "megafrobber",
                "notify": {
                    "started": {
                        "irc": [ "PHB", "#teamchannel" ]
                    }
                }
            }
        }
    ]
}

The corresponding message sent to workers, with the additional notify item would look like lines 1214 in the following example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
{
    "parameters": {
        "service": "megafrobber",
        "hosts": [
            "host01.example.com"
        ],
        "subcommand": "Restart",
        "command": "service"
    },
    "group": "test",
    "dynamic": {},
    "notify": {
        "started": {
            "irc": [ "PHB", "#teamchannel" ]
        }
    }
}

See also

Playbooks - Notify
The documentation for notify elements in playbooks. That section defines the allowed syntax for the notify item.

6.2.2. Response Message Formats

Complimenting the Deployment Message Formats are the Response Message Formats. There are three status messages which workers may reply to re-core with. This section describes the messages which workers send to the re-core component.

6.2.2.1. General Syntax

Status messages are defined as:

6.2.2.2. Job Started

After a worker has received a message from re-core, the message payload is inspected for correctness. If the message payload is successfully verified then the worker will reply to re-core with a status update message indicating the job has been started:

{
    "status": "started"
}

6.2.2.3. Job Completed

Once a worker has completed the job it was given(without errors), the worker will reply to re-core with a status update message indicating success:

{
    "status": "completed"
}

Optionally a worker may reply to re-core with an additional item, data. The value of the data key may be of any type.

Example

1
2
3
4
5
6
7
{
    "status": "completed",
    "data": {
        "items_frobbed": 1337,
        "avg_time_to_frob_ms": 100
    }
}

On line 3 we see the data key being defined in the response message. On lines 4 and 5 we see two additional items being reported: items_frobbed (the number of items which were frobnicated) and avg_time_to_frob_ms, the average amount of time (in miliseconds) it took to frob each item.

Important

The data item is not currently used by any Release Engine component

Remember that in the previous example, items_frobbed and avg_time_to_frob_ms are just made-up examples. In reality, workers should use the notification system for communicating such information.

6.2.2.4. Job Failed

If for some reason a worker cannot start a job (for example, due to insufficient or incorrect parameters), or if there is an error while executing the job, then the worker will reply to re-core with a status update message:

{
    "status": "failed"
}

6.3. Notification Message Format

Notifications are sent out by components of the Release Engine and follow a standard message format. This format is then consumed by notification workers who turn the standard format into an external notification of some kind (like email).

The Notification Message Format has 4 required keys: slug, message, phase and target.

Required Keys:

  • slug: A “short” message (up to 80 characters).
  • message: The message of the notification.
  • phase: The phase that the notification occured within: “started”, “completed”, “failed”
  • target: A list denoting where the notification should go. This may be irc nicknames, email address, etc.. and is different for different workers.

Note

Even though slug and message are required it does not mean the notification worker will use them both. Some notification workers will only use one or the other due to space constraints. However, if either key is missing the notification will be rejected as malformed and/or cause problems!

Example:

{
    "slug": "RPM's have been promoted",
    "message": "The rpms in deployment 12345667890 have been promoted from DEV to QA and are ready for installation.",
    "phase": "completed",
    "target": ["someone@example.com"]
}

6.4. Output Message Format

Notifications are sent out by workers connected to the Release Engine and follow a standard and very simple message format. This format is then consumed by the output worker who turns the standard format into messages in a file.

The Output Message Format has 1 key: message. The only other bit of information needed by an output worker is the correlation id which happens to be stored in the AMQP properties.

Required Keys:

  • message: The message which should be written to a file.

Example:

{
    "message": "Something happened and you should know about it"
}