3.3. RE-REST

Simple REST Api for the Release Engine. By design RE-REST is the only way to interact with the Release Engine.

3.3.1. RE-REST Configuration

Configuration of the server is done in JSON and is by default kept in the current directories settings.json file.

You can override the location by setting REREST_CONFIG environment variable.

Name Type Parent Value
LOGFILE str None File name for the application level log
LOGLEVEL str None DEBUG, INFO (default), WARN, FATAL
MQ dict None Where all of the MQ connection settings are
PASSWORD str MQ Password to authenticate with
PORT int MQ AMQP connection port (default 5672 for non-ssl, 5671 for ssl)
SERVER str MQ Hostname or IP of the server
USER str MQ Username to connect with
SSL bool MQ If the AMQP connection should be SSL/TLS secured (default false)
VHOST str MQ vhost on the server to utilize
MONGODB_SETTINGS dict None Where all of the MongoDB settings live
DB str MONGODB_Settings Name of the database to use
USERNAME str MONGODB_Settings Username to auth with
Password str MONGODB_Settings Password to auth with
HOST str MONGODB_Settings Host to connect to
PORT int MONGODB_Settings Port to connect to on the host
PLAYBOOK_UI bool None Turn’s on/off the experimental playbook ui. It’s off by default.
AUTHORIZATION_CALLABLE str None module.location:callable. Eg: rerest.authorization:no_authorization
AUTHORIZATION_ENVIRONMENT_CALLABLE str None module.location:callable that decides is a user can modify an environment
AUTHORIZATION_CONFIG dict None Authorization callable specific configuration items
GROUP_ENVIRONMENT_MAPPING dict None Mapping of group: [“allowed”, “environments”].

Further configuration items can be found in the Flask Documentation or look at specific AUTHORIZATION_CALLABLE/AUTHORIZATION_ENVIRONMENT_CALLABLE documentation.

For a full example see example-settings.json

3.3.1.1. MQ Configuration Notes

There are two optional parameters in the MQ configuration section: PORT, and SSL. Their defaults are shown below:

  • PORT - 5672 (rabbitmq no-ssl)
  • SSL - false

If SSL is not set (or is false) then re-rest uses the default rabbit MQ port (5672), unless a port has been specified in the config file. If SSL is set to true then re-rest uses the RabbitMQ SSL port (5671), unless a port has been specified in the configuration file.

Simply put, you don’t need to set SSL or PORT unless:

  • You want to enable SSL (in which case, set SSL to true in the config file)
  • You are running RabbitMQ on non-standard ports.

Here’s a bare-minimum MQ configuration section:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
{
    "MQ": {
        "EXCHANGE": "my_exchange",
        "NAME": "username",
        "PASSWORD": "password",
        "QUEUE": "re",
        "SERVER": "amqp.example.com",
        "VHOST": "/"
    }
}

Note that PORT and SSL are not set. Therefore this will open an unencrypted connection to Rabbit MQ using the default port (5672).

Here’s a bare-minimum MQ configuration file for an encrypted connection:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
{
    "MQ": {
        "EXCHANGE": "my_exchange",
        "NAME": "username",
        "PASSWORD": "password",
        "QUEUE": "re",
        "SERVER": "amqp.example.com",
        "SSL": true,
        "VHOST": "/"
    }
}

Note on line 8 that we set SSL to true (remember, it’s lower-case “true” in JSON files) and we are not setting the port. In this case the port is automatically set to 5671.

And now a non-standard configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
{
    "MQ": {
        "EXCHANGE": "my_exchange",
        "NAME": "username",
        "PASSWORD": "password",
        "PORT": 5672,
        "QUEUE": "re",
        "SERVER": "amqp.example.com",
        "SSL": true,
        "VHOST": "/"
    }
}

In this confusing and non-standard configuration we are connecting to an SSL enabled RabbitMQ server which is listening for SSL connections on port 5672, a port which is normally reserved for non-SSL connections.

3.3.2. Authentication

re-rest uses a simple decorator which enforces a REMOTE_USER be set.

3.3.2.1. rerest.decorators:remote_user_required

This decorator assumes that re-rest is running behind another web server which is taking care of authentication. If REMOTE_USER is passed to re-rest from the web server re-rest assumes authentication has succeeded. If it is not passed through re-rest treats the users as unauthenticated.

Warning

When using this decorator it is very important that re-rest not be reachable by any means other than through the front end webserver!!

3.3.3. Authorization: Callable

re-rest uses two decorators. The first keys off the AUTHORIZATION_CALLABLE configuration parameter. This callable is responsible for deciding if a user has access to the URL in question.

3.3.3.1. rerest.authorization.no_authorization

Warning

This should not be used in a production environment

To use this callable set AUTHORIZATION_CALLABLE to rerest.authorization:no_authorization.

3.3.4. AUTHORIZATION: ENVIRONMENT_CALLABLE

The second authorization callable keys off AUTHORIZATION_ENVIRONMENT_CALLABLE configuration parameter. This callable is responsible for deciding if a user has access to the environment(s).

3.3.4.1. rerest.authorization.envrestrictions:environment_allow_all

Warning

This should not be used in a production environment

To use this callable set AUTHORIZATION_ENVIRONMENT_CALLABLE to rerest.authorization.envrestrictions:environment_allow_all.

3.3.4.2. rerest.authorization.envrestrictions:environment_flat_files

To use this callable set AUTHORIZATION_CALLABLE to rerest.authorization.envrestrictions:environment_flat_files and set the following items in your configuration file.

Name Type Parent Value
ENVIRONMENT_FLAT_FILES dict None Dictionary holding mapping informationa. key/val is environment name: path to file

Here is an example of what the secion would look like:

1
2
3
4
5
6
7
 {
     "AUTHORIZATION_ENVIRONMENT_CALLABLE": "rerest.authorization.envrestrictions:environment_flat_files",
     "ENVIRONMENT_FLAT_FILES": {
         "somegroup": ["dev", "qa"],
         "superadmins": ["dev", "qa", "stage", "production"]
     }
 }

3.3.5. RE-REST Deployment

3.3.5.1. Apache with mod_wsgi

mod_wsgi can be used with Apache to mount rerest. Example mod_wsgi files are located in contrib/mod_wsgi.

  • rerest.conf: The mod_wsgi configuration file. This should be modified and placed in /etc/httpd/conf.d/.
  • rerest.wsgi: The WSGI file that mod_wsgi will use. This should be modified and placed in the location noted in rerest.conf

3.3.5.2. Gunicorn

Gunicorn (http://gunicorn.org/) is a popular open source Python WSGI server. It’s still recommend to use Apache (or another web server) to handle auth before gunicorn since gunicorn itself is not set up for it.

$ gunicorn --user=YOUR_WORKER_USER --group=YOUR_WORKER_GROUP -D -b 127.0.0.1:5000 --access-logfile=/your/access.log --error-logfile=/your/error.log -e REREST_CONFIG=/full/path/to/settings.json rerest.app:app

3.3.6. Running From Source

To run directly from source in order to test out the server run:

$ python rundevserver.py

The dev server will allow any HTTP Basic Auth user/password combination.

3.3.7. URLs

3.3.7.1. /api/v0/groups/

Note

All authorized users can list all groups. However, authentication is required to see or manipulate the actual playbooks.

  • GET: Gets a list of all groups.
  • Response Type: json
  • Response Example: {"status": "ok", "items": [...]}
  • Input Format: None
  • Inputs: None

3.3.7.2. /api/v0/$GROUP/playbook/$PLAYBOOKID/deployment/

  • PUT: Creates a new deployment.
  • Response Type: json
  • Response Example: {"status": "created", "id": 1}
  • Input Format: None
  • Inputs: optional json

3.3.7.3. /api/v0/playbooks/

  • GET: Gets a list of all playbooks.
  • Response Type: json
  • Response Example: {"status": "ok", "items": [...]}
  • Input Format: None
  • Inputs: None

3.3.7.4. /api/v0/$GROUP/playbook/

  • GET: Gets a list of all playbooks for a group.
  • Response Type: json
  • Response Example: {"status": "ok", "items": [...]}
  • Input Format: None
  • Inputs: None
  • PUT: Creates a new playbook.
  • Response Type: json
  • Response Example: {"status": "created", "id": "53614ccf1370129d6f29c7dd"}
  • Input Format: json/yaml
  • Inputs: Optional format parameter which controls submit type. Can be json or yaml. Default is json.

3.3.7.5. /api/v0/$GROUP/playbook/$ID/

  • GET: Gets a playbooks for a group.
  • Response Type: json/yaml
  • Response Example: {"status": "ok", "item": ...}
  • Input Format: None
  • Inputs: Optional format parameter which controls response type. Can be json or yaml. Default is json.
  • POST: Replace a playbook in a group.
  • Response Type: json
  • Response Example: {"status": "ok", "id": "53614ccf1370129d6f29c7dd"}
  • Input Format: json/yaml
  • Inputs: Optional format parameter which controls response type. Can be json or yaml. Default is json.
  • DELETE: Delete a playbook in a group.
  • Response Type: json
  • Response Example: {"status": "gone"}
  • Input Format: None
  • Inputs: None

3.3.8. Platform Gotchas

3.3.8.1. RHEL 6

You may need to add the following to your PYTHONPATH to be able to use Jinja2:

/usr/lib/python2.6/site-packages/Jinja2-2.6-py2.6.egg

3.3.9. What’s Happening

  1. User requests a new job via the REST endpoint
  2. The REST server creates a temporary response queue and binds it to the exchange with the same name.
  3. The REST server creates a message with a reply_to of the temporary response queue’s topic.
  4. The REST server sends the message to the bus on exchange re and topic job.create. Body Example: {“group”: “nameofgroup”}
  5. The REST server waits on the temporary response queue for a response.
  6. Once a response is returned the REST service loads the body into a json structure and pulls out the id parameter.
  7. The REST service then responds to the user with the job id.
  8. The temporary response queue then is automatically deleted by the bus.

3.3.10. Usage Example

The authentication mechanism used in the front end webserver could be set up to use vastly different schemes. Instead of covering every possible authentication style which could be used we will work with two common ones in usage examples: htacces and kerberos.

Note

Setting up the front end proxy server for authentication is out of scope for this documentation.

3.3.10.1. htaccess / HTTP Basic Auth

$ curl -X PUT --user "USERNAME" -H "Content-Type: application/json" --data @file.json https://rerest.example.com/api/v0/test/playbook/
Password:

... # 201 and json data if exists, otherwise an error code

3.3.10.2. kerberos

$ kinit -f USERNAME
Password for USERNAME@DOMAIN:
$ curl -u 'a:a' -H "Content-Type: application/json" --data @file.json -X PUT https://rerest.example.com/api/v0/test/playbook/

... # 201 and json data if exists, otherwise an error code

3.3.10.3. Dynamic Variables

Passing dynamic variables requires two additions

  1. We must set the Content-Type header (-H ... below) to application/json
  2. We must pass data (-d '{....}' below) for the PUT to send to the server

This example sets the Content-Type and passes two dynamic variables: cart which is the name of a Juicer release cart, and environment, which is the environment to push the release cart contents to.

$ curl -u "user:passwd" -H "Content-Type: application/json" -d '{"cart": "bitmath", "environment": "re"}' -X PUT http://rerest.example.com/api/v0/test/playbook/12345/deployment/

 ... # 201 and json data if exists, otherwise an error code

See also