SaltStack: Creating Dependent or Referring Service Configurations

What is this article about?

Familiarity with the capabilities of SaltStack to create service configurations dependent on each other; from services located on other or all other subordinate systems, etc. If it is simpler, consider how each subordinate system can receive data from other similar systems at the time of creation and distribution of its configurations.

This is the third article in the SaltStack series, read the first here , the second here .

Task: deploy a cluster of some service, so that in the node configurations there were links to all other nodes of this cluster

The most trivial task for automation is to make many nodes with the previously described configuration of services, place these configurations and get profit. Many guides, including SaltStack, describe such standard cases, so we will not focus on them. All this is fine as long as these nodes do not need to know about the state and parameters of other nodes included in the automation process.

Consider the simplest version of the described task - to write the addresses and names of all other nodes in the cluster to the / etc / hosts file on each node.

How to solve this with SaltStack? The simplest thing that comes to mind is to write the names and addresses of all the nodes into a pillar file and connect it to all the states of all the nodes:

pillar / cluster-nodes.sls

cluster-nodes:
  node1: 
    name: node1.domain.com
    ip: 10.0.0.1
  node2: 
    name: node2.domain.com
    ip: 10.0.0.2
  node3: 
    name: node3.domain.com
    ip: 10.0.0.3
  .......
  nodeN: 
    name: nodeN.domain.com
    ip: 10.0.0.N

pillar / top.sls

base:
  '*':
    - cluster-nodes

Create a state to enter this data into / etc / hosts. For this we will use salt.states.host .

states / cluster-nodes.sls

{% for node in salt['pillar.get']('cluster-nodes', []) %}
cluster-node-{{node['name']}}:
  host.present:
    - ip: {{node['ip']}}
    - names:
      - {{node['name']}}
      - {{node['name'].split('.')[0]}}
{% endfor %}

After applying the state, we get on all nodes something like / etc / hosts :

127.0.0.1     localhost 
10.0.0.1      node1.domain.com   node1
10.0.0.2      node2.domain.com   node2
10.0.0.3      node3.domain.com   node3
.........
10.0.0.N      nodeN.domain.com   nodeN

It would seem - the goal has been achieved - others are visible from all the hosts. This kind of solution, though not elegant, because you will have to make changes to the pillar file when adding new nodes to the cluster, but it has the right to exist in those cases when the names and addresses of all nodes are known and fixed.

But what if each of the nodes gets its address, for example, via DHCP? Or is a IaaS provider like Amazon EC2, GoGrid or Google Grid engaged in node generation? There are no addresses or node names in advance, and you will have to pay extra for fixed addresses.

(Lyrical digression - in the near future I will write an article on how to create your own infrastructure in EC2 using SaltStack) .

In principle, after installing a minion on a node, information about its name and address can be obtained usingsalt grains .

For example, you can get this data on a minion like this:

#salt-call grains.item fqdn fqdn_ip4
local:
    ----------
    fqdn:
        ip-10-6-0-150.ec2.internal
    fqdn_ip4:
        - 10.6.0.150

Or in the description of the state:

{% set host_name = salt['grains.get']('fqdn') %}
{% set host_ip = salt['grains.get']('fqdn_ip4') %}

All would be nothing - but there is one significant but : it is all available on the master and not available on individual minions. That is, at the time of generating the configurations on each of the minions, the data of the minion itself, some of the wizard and absolutely nothing about the other minions are visible.

Here the question arises - how to get data from other minions when generating configurations of a particular minion?

The answer is simple: you can use salt-mine for this . Not much is written in the documentation, but enough to understand the general purpose of this service. How does he work? The master caches a certain set of data from each of the minions with some periodicity and provides access to these cached data to all minions.

How to implement this?
1. We set mine_functions - a description of the functions for receiving and caching individual data from minions. They can be defined using the direct description in / etc / salt / minion on each of the minions, or by connecting a pillar file with a description of these functions on the wizard for each of the minions.
2. Either wait for a while (mine_interval sec. - you can specify in the configuration of the minion) or force update by hand using salt '*' mine.update
3. Use the mine.get function to get the necessary data from the wizard when configuring the current minion.

Consider how to solve the problem with hosts using the steps described. So:

1. Create an entry in the pillar file and connect it to all the minions:

pillar / minefuncs.sls

mine_functions:
  grains.item: [fqdn, fqdn_ip4]

pillar / top.sls

base:
  '*':
    - minefuncs

2. Forsim data collection from minions.

3. Create a state for / etc / hosts :

{% for node, fqdn_data in salt['mine.get']('*', 'grains.item', expr_form='glob').items() %}
cluster-node-{{fqdn_data['fqdn']}}:
  host.present:
    - names:
      - {{fqdn_data['fqdn'].split('.')[0]}}
      - {{fqdn_data['fqdn']}}
    - ip: {{fqdn_data['fqdn_ip4'][0]}}
{% endfor %}

As a result, we get an automatically generated hosts file containing the names and addresses of all minions.

If there is a need to isolate a specific set of minions for which the state will be used (for example, some cluster nodes have one role, others have a different one and you need to isolate only nodes with specific roles), you can use the following recommendations:

1. For all minions, define custom grain (this easy and described in the standard documentation), for example: grains: roles: name_node and grains: roles: data_node.
2. Make a selection in mine.get for the specified roles. For example, like this:

{% for node, fqdn_data in salt['mine.get']('roles:data_node', 'grains.item', expr_form='grain').items() %}
cluster-node-{{fqdn_data['fqdn']}}:
  host.present:
    - names:
      - {{fqdn_data['fqdn'].split('.')[0]}}
      - {{fqdn_data['fqdn']}}
    - ip: {{fqdn_data['fqdn_ip4'][0]}}
{% endfor %}
{% for node, fqdn_data in salt['mine.get']('* and not G@roles:data_node', 'grains.item', expr_form='compound').items() %}
cluster-node-{{fqdn_data['fqdn']}}:
  host.present:
    - names:
      - {{fqdn_data['fqdn'].split('.')[0]}}
      - {{fqdn_data['fqdn']}}
    - ip: {{fqdn_data['fqdn_ip4'][0]}}
{% endfor %}

In these cases, pay attention to expr_form and the description of the expression for the selection.

That, in fact, is the solution to the problem described above - so that it is possible to receive configuration data from minions.

Using the described techniques, you can also transfer various data between minions in the process of generating their configurations.

I hope the article will be useful to everyone who uses SaltStack in fairly non-trivial configurations.

Thank you for reading.