Extend Ansible functionality with modules

Under the hood of the d2c.io service , we actively use Ansible - from creating virtual machines in provider clouds and installing the necessary software, to managing Docker containers with client applications.

In the article on extending the functionality of Ansible, we partially examined how plugins differ from modules. In short, the main difference is that the former run on the local machine where Ansible is installed, and the latter on the target.

The main task of the plugins is to influence the progress of the playbook, add new features for loading and processing data. The task of the modules is to expand the list of systems and services that Ansible can manage. For example, create a server on the Vultr site - a module vultr, create a user in a home-made authorization system for an office WiFi network - a module mywifiauth_user.

The principle of operation of the modules

A module is a small program that:

• runs on the target host
• can take parameters as input (via parameter file)
• gives a report on its work to standard output in JSON format

The module execution process looks like this:

• Ansible takes the following task from the execution queue. Defines the name of the module to use.
• If an action plug-in of the same name exists, it executes it ( see the article on plugins, part 1 ).
  The plugin can do the preparatory work. For example, it will copy files from the control machine to the target host in advance.
• Prepares a parameter file for the module
• Depending on the type of module:
  
  • for modules that do not use the Ansible Framework: copies the parameter file and the module executable file to the target host.
  • for modules based on AnsibleModule : generates and delivers to the target host a self-extracting Python file containing all the necessary auxiliary Ansible classes, a parameter file and the file of the module itself. Such a “package” is necessary for pipelining mode to work ( see the article about Ansible acceleration ).
• Runs a module or “package” on the remote host.
• The module does useful work.
• Ansible receives the result of the module as a JSON object from standard output.

Most often, the target hosts are remote machines, servers, and devices: for example, the user module manages users exactly on the host on which it starts. Some modules, by contrast, are more often run on the local host using connection: local, local_actionor delegate_to: localhost. A vivid example of this is cloud management modules such as ec2 . They require the setting of credentials, which are most often available precisely on the control machine. The wait_for module to wait for TCP ports to open also often runs on the local machine. It is used, for example, to wait until the remote server restarts and the SSH connection becomes available.

The simplest module

Modules can be created in any language. Starting with Ansible 2.2, modules can be binary executables. Let's make the simplest module on bash:

#!/bin/bash
echo '{"changed":false,"date":"'$(date)'"}'

Save this code to the file ./library/bash_mod.sh and check:

$ ansible localhost -m bash_mod
localhost | SUCCESS => {
    "changed": false,
    "date": "среда, 6 сентября 2017 г. 17:00:32 (MSK)"
}

Ansible receives information about the results of the modules from their standard output. All output must be a valid JSON object (therefore, you cannot debug modules using print statements) . Depending on the value of the service properties, Ansible may make different decisions. One of these properties: changed. For example, you can change in bash_mod its value from falseto trueand see that Ansible now believes that your module has changed something on the target host, the output has turned yellow.

Input parameters

There are several ways to get input parameters for a module:

• Through a file with pairs key=valueseparated by spaces. Used for modules in interpreted languages. The path to the parameter file is passed to the module as the only command line parameter. For example, for our module on bash.
• Through a file with a JSON object. It is used for binary modules, modules based on AnsibleModuleand modules in interpreted languages, in the body of which there is a word WANT_JSON. For example, bashyou can add a comment to our module on # WANT_JSON.
• Through the injection of a JSON object into a file. Used for modules in interpreted languages, in the body of which there is a marker . Before sending the module to the target host, this line will be replaced with a JSON object with parameters.<>

If you create a module based on AnsibleModule, then there is no need to take care of how the parameters are passed - the base class will do all the hood work.

Class AnsibleModule

To simplify the development of custom modules, Ansible has a class AnsibleModule. It takes care of processing the parameters, checking their types and valid values, and also provides a set of auxiliary methods for working with files, checksums and more.

As a basic example, let's look at the module code ping:

from ansible.module_utils.basic import AnsibleModule
def main():
    module = AnsibleModule(
        argument_spec=dict(
            data=dict(required=False, default=None),
        ),
        supports_check_mode=True
    )
    result = dict(ping='pong')
    if module.params['data']:
        if module.params['data'] == 'crash':
            raise Exception("boom")
        result['ping'] = module.params['data']
    module.exit_json(**result)
if __name__ == '__main__':
    main()

A module is described which has a single valid but optional parameter data. It supports verification mode, the key --checkwhen starting Ansible. As a result, returns pongeither the value of the parameter data. If you pass crashas data, the module will "fall" with an error.

For more complex examples, you can refer to the code of other modules . In this regard, it is convenient that Ansible is an open source project.

Module example

One of the convenient cases for writing a module is a wrapper for shell-commands. If you can do some operation on the target host through the command line using the shell/ modules command, but you need to use it often in different playbooks and you want to make the code beautiful and readable. I will analyze a little mythical, but no less working, example with setting the volume level in the operating system:

#!/usr/bin/python
# -*- coding: utf-8 -*-
DOCUMENTATION = '''
---
module: osx_volume
short_description: Set OS X volume level
description:
   - Set OS X volume level or mute flag
options:
    level:
        description:
            - Volume level to be applied
        aliases:
            - volume
        required: false
    muted:
        description:
            - Set mute on/off
        required: false
author:
    - Konstantin Suvorov
'''
EXAMPLES = '''
- name: Set volume to 25
  osx_volume:
    level: 25
- name: Mute
  osx_volume:
    muted: yes
'''
from ansible.module_utils.basic import AnsibleModule
from subprocess import call, check_output
def get_volume():
    level = check_output(['osascript','-e','output volume of (get volume settings)']).strip()
    muted = check_output(['osascript','-e','output muted of (get volume settings)']).strip()
    muted = (muted.lower() == "true")
    return (int(level), muted)
def set_volume(level=None, muted=None):
    if level is not None:
        call(['osascript','-e','set volume output volume {}'.format(level)])
    if muted is not None:
        mute_str = 'true' if muted else 'false'
        call(['osascript','-e','set volume output muted {}'.format(mute_str)])
    return get_volume()
def main():
    module = AnsibleModule(
        argument_spec=dict(
            level=dict(type='int', required=False, default=None, aliases=['volume']),
            muted=dict(type='bool', required=False, default=None)
        ),
        supports_check_mode=True
    )
    req_level = module.params['level']
    req_muted = module.params['muted']
    l, m = get_volume()
    result = dict(level=(req_level if req_level is not None else l),
                  muted=(req_muted if req_muted is not None else m),
                  changed=False)
    if req_level is not None and l != req_level:
        result['changed'] = True
    elif req_muted is not None and m != req_muted:
        result['changed'] = True
    if module.check_mode or not result['changed']:
        module.exit_json(**result)
    new_l, new_m = set_volume(level=req_level, muted=req_muted)
    if req_level is not None and new_l != req_level:
        module.fail_json(msg="Failed to set requested volume level {} (actual {})!".format(req_level, new_l))
    if req_muted is not None and new_m != req_muted:
        module.fail_json(msg="Failed to set requested mute flag {} (actual {})!".format(req_muted, new_m))
    module.exit_json(**result)
if __name__ == '__main__':
    main()

I will not analyze the module code in detail - you can see it yourself. The module adjusts the volume level, controls the mode mutein MacOS. Supports dry-run mode. It will display the status changed=trueif the values ​​should change. The module is idempotent, and if you apply it twice with the same parameters, it will do nothing and display the status changed=false.

Debugging Modules

The option without any preparatory work is to start Ansible with the setting turned on ANSIBLE_KEEP_REMOTE_FILES=1and the level of logging -vvv. In this case, Ansible will not delete the generated module and parameter files, but will leave them in a temporary folder on the target host. An advanced level of logging allows you to see the path of the directory in which the files are located.

Go to the target host by SSH, go to the desired folder, for example cd /tmp/ansible-tmp-1488291604.43-129413612218427. Now we can run, modify and debug our module locally. If the module is written based on AnsibleModule, then it can be started with debugging commands:

• explode - unzip the “archive” into a folder for further modification
• execute - starting the module from the folder obtained in the previous step

For example, if we debug a module in this way ping, then:

• ./ping.py - run the module from the "package"
• ./ping.py explode - unpack the module and parameters to the folder debug_dir
• ./ping.py execute- run the module from the folder debug_dir(with all the changes that we will make there)

Another option is to use the test-module utility . It allows you to prepare the file with the parameters and “pack” the module in the same way as Ansible does in real work. This option allows testing modules locally and much faster than through Ansible; It’s easier to connect a debugger.

Module distribution

In order for Ansible to “see” the module, it must be located on the local machine in the module search paths. By default, this is the directory ./librarynext to the playbook, but this setting can be changed in the configuration file or through environment variables.

If you use role within the role can also be a folder librarywith your unit, for example ./roles/myrole/library/mymodule.py. In this case, if a role was used in the playbook myrole, then it mymodulewill become available. A role can even be empty, without a file tasks/main.yml.

Documenting Modules

Modules are useful to document! If the documentation inside Python modules is described in variables DOCUMENTATIONand EXAMPLESin accordance with a specific format (see the example above), then information about your module can be conveniently viewed using the utility ansible-docfrom the standard Ansible package.

In addition to reference information about the module, this utility can generate “snippets” for inserting playbooks in the code, for example:

$ ansible-doc -s postgresql_db
- name: Add or remove PostgreSQL databases from a remote host.
  action: postgresql_db
      encoding               # Encoding of the database
      lc_collate             # Collation order (LC_COLLATE) to use in the data
      lc_ctype               # Character classification (LC_CTYPE) to use in t
      login_host             # Host running the database
      login_password         # The password used to authenticate with
      login_unix_socket      # Path to a Unix domain socket for local connecti
      login_user             # The username used to authenticate with
      name=                  # name of the database to add or remove
      owner                  # Name of the role to set as owner of the databas
      port                   # Database port to connect to.
      ssl_mode               # Determines whether or with what priority a secu
      ssl_rootcert           # Specifies the name of a file containing SSL cer
      state                  # The database state
      template               # Template used to create the database

Well, it's time to finish the article on modules. Considering the previous articles, you can now expand the functionality of Ansible in all directions! If you have any questions, ask in the comments - I will try to answer or prepare another article. Stay tuned!