7. Playbooks

Playbooks are documents which describe the exact set of steps required to successfully start and finish a given software release. When the Release Engine begins a deployment, the actions it takes come directly from playbooks.

A playbook might be ran automatically each time a code builder finishes so that it may deploy the latest snapshots. Alternatively, if more control is desired over the release process, playbooks may be ran by hand. Playbooks may be written in YAML syntax, or optionally in JSON Syntax.

In this section we’ll learn:

  • what a playbook looks like (by reviewing a simple example)
  • the major items of playbooks
  • the basics of how to describe execution steps in your playbooks, including:
  • describing a release step
  • identifying which worker to use
  • passing data to the worker

7.1. Example Playbook

Here is an example of what a super simple playbook looks like. The playbook is owned by the group called inception.

When ran, all this would do is restart the httpd service on foo.bar.example.com

1
2
3
4
5
6
7
8
9
---
group: inception
name: Simple playbook
execution:
  - description: restart httpd
    hosts:
      - foo.bar.example.com
    steps:
      - service:Restart: {service: httpd}

7.2. Playbook Components

A Release Engine playbook is made up of the following required items:

group

A short string (acronyms are best) defining the ownership of this playbook. Think of it like the unix group a team might all be members of.

The group in our example is inception

execution

A list of playbook execution sequences. These execution sequences are composed of release steps and are accompanied by supporting meta-data. These sequences are explained fully in Execution Sequences.

In our example, we have one execution sequence with one release step (service:Restart).

Additionally, a playbook may define the optional item:

name

A short description of what this playbook accomplishes overall.

In our example the name is Simple playbook.

7.3. Execution Sequences

Recall that execution items hold a list type. Each item in the list is an execution sequence. This section describes exactly what execution sequences do, and how to write our own.

In our example, simple playbook, the execution sequence is defined in lines 5 → 9. Let’s review those lines again:

1
2
3
4
5
6
7
8
9
---
group: inception
name: Simple playbook
execution:
  - description: restart httpd
    hosts:
      - foo.bar.example.com
    steps:
      - service:Restart: {service: httpd}

Like playbooks themselves, each execution sequence is comprised of several required and optional elements. In Sample playbook we can see several items are already being used: description, hosts, and steps. The following sections will describe these, and all other items which are allowed in execution steps.

7.3.1. Required Items

This section describes the required items in execution sequences.

7.3.1.1. Hosts

  • Required: Yes
  • Argument type: list
  • Default: None

The hosts element is used to describe the target hosts for the script to act on.

---
- hosts:
   - www01.web.ext.example.com
   - www02.web.ext.example.com
   - www03.web.ext.example.com

Or in YAML shorthand:

---
- hosts: [www01.web.ext.example.com, www02.web.ext.example.come, www03.web.ext.example.com]

7.3.1.2. Steps

  • Required: Yes
  • Argument type: list
  • Default: None

The steps element defines the steps that will be performed on each host in hosts. The syntax of each possible step varies, however they will assume one of three legal forms, respective to the information (if any) which the execution step requires to run. In brief, these forms are described below:

  • Some steps require no information at all, as such they are given as strings
  • Some require explicit parameters given in the form of keyword arguments
  • Similar to the previous form, some steps require an enumeration of their dynamic parameters
7.3.1.2.1. Steps - Strings

An execution step may be a simple string. To us, this means that this given step requires no additional information. The actual workers implementing each step may include possible several sub-commands. Because of this it is common to see step names given in colon delimited notation. In this form the string leading up to but not including the colon (”:”) character refers to the worker or module. The string after the colon character refers to the specific sub-command to run.

OK. Enough of the boring stuff. Let’s see some examples.

Assume there is a module called logrotate, this module provides one sub-command: Rotate.

The documentation for the logrotate module tell us that the Rotate sub-command requires no keyword parameters. That is to say, we can define it in our execution steps as a simple string. Recall that steps are denoted using colon notation, where the module name comes first, followed by the sub-command. In this case our module is logrotate and our sub-command is Rotate. We can see this on line 9 in the following example:

1
2
3
4
5
6
7
8
9
---
group: inception
name: Simple playbook
execution:
  - description: Rotate all the megafrobber logs
    hosts:
      - foo.bar.example.com
    steps:
      - logrotate:Rotate

But what is happening exactly in this example? On line 9 we see the entry: - logrotate:Rotate. Recall that each step is given as an item to the steps list. This is why line 9 in the previous code example begins with a hyphen character. In YAML this indicates a list item.

Note closely exactly how we gave logrotate:Rotate, because in the next example this will change very slightly.

7.3.1.2.2. Steps - Keyword Arguments

Now let us assume there is a module called service which can control system services. The documentation for this module tells us that there are three sub-commands provided: Start, Stop, and Restart. Additionally, the documentation tells us that each sub-command requires one keyword parameter: service. On lines 9 and 10 in the following example, we see how to provide keyword arguments to steps:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
---
group: inception
name: Simple playbook
execution:
  - description: Restart the megafrobber service
    hosts:
      - foo.bar.example.com
    steps:
      - service:Restart:
          service: megafrobber

Or in YAML shorthand (only focusing on the step definition)

1
2
3
---
# ...
      - service:Restart: {service: megafrobber}

Let’s look closer at this and see exactly what is happening.

Recall that playbooks are YAML Documents. As such, there are defined ways to describe different datastructures. Review the dictionary section in YAML Scripts if you need a refresher.

The service:Restart sub-command requires one parameter, service. You describe parameters in execution steps by using a hash, or dictionary. For our example, a dictionary describing a keyword service with value megafrobber would look like the following example in YAML:

1
2
---
service: megafrobber

Additionally, recall that you can nest datastructure in YAML. If we wanted to represent a list of dictionaries, we could do that in the following way. Here’s an example of a list of nested dictionaries:

1
2
3
4
5
---
- thingies:
    service: megafrobber
- stuffs:
    penguins: cute

Or in alternative representation:

1
2
---
[{thingies: {service: megafrobber}}, {stuffs: {penguins: cute}}]

Now that we know all of this, to give the required parameters to our step we will define the step as a dictionary key with a nested-dictionary describing our parameters. This is shown on lines 8 and 9 in the following example:

1
2
3
4
5
6
7
8
9
---
# ...
execution:
  - description: Restart the megafrobber service
    hosts:
      - foo.bar.example.com
    steps:
      - service:Restart:
          service: megafrobber

Important

Note the syntax change
In the previous example we only gave the string: logrotate:Rotate. Now, instead of a string we’re describing a dictionary key.

Therefore, the text for this step begins with a hyphen character (to indicate a list item) and ends with a colon character.

Finally, on line 4 you see the provided parameters.

If there were a module which required more than one parameter, the syntax is very similar. Lines 46 show this in the following example:

1
2
3
4
5
6
---
# ...
      - service:Restart:
          service: megafrobber
          foo: bar
          noop: true
7.3.1.2.3. Steps - Dynamic Arguments

This section is about dynamic arguments. Dynamic arguments differ from normal arguments in that their values are not stored in playbooks. Rather, within a playbook, we assert that their values will be provided by the calling client when starting a deployment. The syntax for defining dynamic arguments differs only slightly from how keyword arguments are defined.

The scope of this section is limited to the role of dynamic arguments in playbooks only. That is to say, discussion of dynamic arguments in worker development will not be covered here. Instead, see the re-worker documentation for that information.

Caution

The ability for clients to provide a broad spectrum of dynamic data is both a pro and a con.

If you’re writing a new worker, think very carefully before making arguments dynamic. Consider if the there is a non-interactive way for the information to be obtained instead.
7.3.1.2.3.1. Use Cases

Situations where dynamic arguments may be required are generally limited to actions which require data that changes every, or nearly every, release. Examples might include:

For a more complete example, imagine our workplace has strict policies around software releases. These policies state that any software release must have an associated change record with it. Additionally the policy states that every time a release happens for a change, an update to the change record document must be recorded. This update must indicate the date of the release.

In this situation, the pragmatic approach to automating this task would be to develop a worker which can interface with the change management system and add updates to the change record over an API. Let’s pretend such a worker already exists.

It is clear that we cannot hard-code change record numbers into the worker. And storing this information in the playbook would require a manual update to the playbook every time a new change is created. Furthermore, this limitation effectively nullifies the ability of a playbook to be used for two changes happening in an overlapping time span.

This is an excellent opportunity to use dynamic variables.


The following examples will be using a fictional worker called change. This worker has one usable function: Update. The change worker documentation provides the following API signature for the change:Update function:

  • Function: change:Update

    • Arguments:

      • None
    • Dynamic Arguments:

      • Name: id

        • Required: True
        • Type: string or int
        • Description: The ID of the change record to update

For the rest of this section, let’s pretend our id is CHG1337.

7.3.1.2.3.2. Dynamic Argument Syntax

Let’s begin by considering our example simple playbook again.

1
2
3
4
5
6
7
8
9
---
group: inception
name: Simple playbook
execution:
  - description: restart httpd
    hosts:
      - foo.bar.example.com
    steps:
      - service:Restart: {service: httpd}

However, instead of just restarting the httpd service, we have to have an additional step: updating the change record (CHG1337). In this section we will learn how add that step.

The general syntax for defining a step with dynamic arguments is shown on lines 49 in the following example:

1
2
3
4
5
6
7
8
---
# ...
    steps:
      - worker:Function:
         dynamic:
            - arg_name_0
            - arg_name_1
            - arg_name_2
Line 4
As before, we begin by providing the worker.Function name, ending with a : character
Line 5
We define a dictionary key called dynamic. Again note, this must end with a : character.
Lines 6 → 9

We define the value of the dynamic key. This value must be a list.

The items in the list are arg_name_0, arg_name_1, and arg_name_1. Each of these is the name of a dynamic variable required by worker:Function.


Applying this example to our fictional situation will yield the following playbook:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
---
group: inception
name: Simple playbook
execution:
  - description: restart httpd
    hosts:
      - foo.bar.example.com
    steps:
      - change:Update:
          dynamic:
            - id
      - service:Restart: {service: httpd}
Line 9
Insert the new sequence step, change:Update
Line 10
Begin the dynamic argument dictionary
Line 11
Define the dynamic argument list with one item: id
7.3.1.2.3.3. Providing Values

Now that we have learned how to add a sequence step that requires dynamic arguments to a playbook, it might be helpful to quickly review how clients can provide the information.

A commonly used command line tool for interacting with REST endpoints (such as RE-REST) is curl. Put simply, curl allows you to make a request to anything http. This is exactly like following a hyperlink in a web page.

The following is an example of how to use the curl command to provide the value of the dynamic argument, id, to the release engine and start a deployment.

1
2
3
4
$ curl -u "user:passwd" -H "Content-Type: application/json" \
-d '{"id": "CHG1337"}' \
-X PUT \
http://rerest.example.com/api/v0/test/playbook/12345/deployment/

Line 2 utilizes the -d (or --data) option to provide the value of the dynamic argument. When curl is ran in this manner, dynamic arguments are provided by describing a dictionary including key-value pairs where the key is the dynamic argument name, and the value is the unique-value of that argument for this particular deployment. In this example the dictionary is {"id": "CHG1337"}.

See also

RE-REST - Dynamic Variables
See the RE-REST → dynamic variables documentation for a complete review of this topic.

7.3.2. Optional Items

This section describes the optional items which are allowed in execution sequences.

7.3.2.1. Description

Finally, an execution step may define a optional description item.

  • Required: No
  • Argument Type: string
  • Default: None

The description element allows you to provide a useful human-readable description of what the step is exactly supposed to do. Use of description items are encouraged.

Recall our previous example of using service.Restart to restart the megafrobber service. Below, line 5, shows an example of how to use the description item.

1
2
3
4
5
6
7
8
9
---
group: inception
name: Simple playbook
execution:
  - description: Restart the megafrobber service
    hosts:
      - foo.bar.example.com
    steps:
      - service.Restart: {service: megafrobber}

Because this item is optional, we could just as well have omitted it entirely:

1
2
3
4
5
6
7
8
---
group: inception
name: Simple playbook
execution:
  - hosts:
      - foo.bar.example.com
    steps:
      - service.Restart: {service: megafrobber}

7.3.2.2. Notify

  • Required: No
  • Argument type: dict
  • Default: None

The notify element allows you to set custom notification hooks which trigger at different phases of each sequence step. For example, you may desire to receive an email every time an RPM promotion step completes, or fails.

Here’s an example of a notify step that updates IRC when the step has started:

1
2
3
4
5
6
7
8
9
---
# ...
    steps:
      - service:Restart:
          service: httpd
          notify:
            started:
              irc: [ "PHB", "#teamchannel" ]
              email: [ "someone@example.com", "anothersomeone@example.com" ]
Line 6
Shows the beginning of the notify syntax.
Line 7

This indicates which phase of worker execution this notification is for.

Recognized phases include: started, completed, and failed

Line 8

Define the IRC notification parameters.

For the irc item we define a list of users/channels for messages to be sent to.

In this example, the user PHB would be notified directly with status updates. Additionally, a notification would be sent to the channel #teamchannel.

See also

Components
For a list of available notification workers, see the Components section.

7.4. Putting it all together

To finish up, let’s put together everything we’ve seen up to now. That will include hosts, and some example items for steps.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
---
# Playbook owned by group inception
group: inception

# This playbook is clearly awesome:
name: Simple playbook

# This playbook executes **two** sequences of steps for this
# release:
execution:

  ################################################################
  # Sequence 1
  ################################################################
  # Including a description is optional. This must be a string.
  - description: frobnicate these lil guys
    hosts:
      - foo.dev.example.com
      - bar.ops.example.com

    steps:
      # Some steps don't require parameters:
      - bigip:OutOfRotation

      # Whereas, some require parameters:
      - misc:Echo:
          input: "This is a test message"

      # And some times you just want to tell the world what you're doing
      - frob:Nicate:
          things: "all the things"
          notify:
              started:
                  irc: [ "PHB", "#myteam" ]
                  email: [ "someone@example.com", "someoneelse@example.com" ]

  ################################################################
  # Sequence 2
  ################################################################
  - description: then frobnicate the other half
    hosts:
      - dev.foo.example.com
      - ops.bar.example.com

    steps:
      - bigip:OutOfRotation

      # Some may even accept lists as the value of their parameters
      - misc:ListFrob:
          frob_list:
            - item1
            - item2
            - item3

Todo

Describe interesting parts of the previous example