Setting Up Your Development Environment: Coffee Ground (Part 2)

Hello dear reader!
This time I want to share my result of setting up a personal environment for working with various PHP-based projects using Puppet. This article describes the results that were obtained in the process of studying and writing Puppet configuration.

This article is written to show one way to do this. I do not want to call for action, but only share what I encountered, what I did and how I use it.

The article will be very long with a technical bias. I ask for a "cut."

Some time ago I wrote the article “Setting up the development environment: needlework circle (Part 1)” , in which I described my torment with each change of project. During the preparation of this article, some components of the environment were added and removed. At the end of the article there will be a link to the repository with a module that you can ~~tear to pieces~~ and use and modify as you wish.

Purpose: quickly configure the working environment for the current project

The goal is the same, but with a small addition: if you can automate, then do it.

Puppet

This is a good tool to help you manage the configuration of various operating systems.

The choice fell precisely on this tool, since this tool is used by the company, and I need to know what is happening behind the scenes at DevOps / NetOps.

Thus, in the process of describing the configuration, I got the following:

- PHP (5.6, 7.x; pools for each project; extensions; composer)
- NGINX (PHP-FPM upstream for each project; Simple vhost configuration)
- OpenSSL
- MySQL
- Bind9
- NodeJS + NPM
- Memcached
- Redis
- Docker
- Additional software: mc, htop, wget, curl

Configuration

Since I store the real configuration in a private repository, I suggest that you familiarize yourself with the training version , which I will describe in this article.

Caution: the data repository is delivered as is. The further destiny of the project is to become a more flexible foundation or die.

When developing the configuration ~~crutch~~ , I used the capabilities of Puppet Hiera and r10k (a tool for convenient configuration deployment).

Basically, the code that is responsible for installing packages, creating files, and restarting services is located in the 'production' branch. Using the capabilities of Puppet Hiera, I provided the opportunity to configure the configuration of the current node, which is determined by the FQDN of the working machine. Thus, one of the configuration examples can be found along the path:

hieradata / nodes / dev.lo.yaml

---
# Node with all in one
classes:
  - role::all
composer: true
projects:
    warface:
        - {name: 'www', php: php7.0}
        - {name: 'imageproxy', php: php5.6}
    cryengine:
        - {name: www, php: php7.1}
        - {name: shop, php: php7.1}
        - {name: forum, php: php5.6}
php:
  versions: [php5.6, php7.0, php7.1, php7.2]
  packages: [
      opcache, gd, bcmath, curl, intl, json, mbstring,
      mysql, readline, soap, sqlite3, tidy, xml, zip,
      codecoverage, codesniffer, igbinary, geoip, imagick,
      memcache, memcached, redis, xdebug, ssh2
  ]
tools: [imagemagick]
bind9:
  dns: ['8.8.8.8', '8.8.4.4']

to be combined with

hieradata / common.yaml

---
# Puppet Server Tuning
puppet_enterprise::master::puppetserver::jruby_max_requests_per_instance:0classes:
  - role::default
composer:truenginx:domain:"%{::fqdn}"projects:development:
        - name:'www'php:'php7.0'php:versions: [php7.0]
  packages: [
    curl, mbstring, xml, json, intl, xdebug
  ]
tools: [mc, htop, wget, curl]
db:mysql:root_password: root
    remove_default_accounts:trueoverride_options: {}
bind9:dns: ['8.8.8.8', '8.8.4.4']
  zone:"%{::fqdn}"

As a result of this configuration, the entire initial list of components will be installed, as well as the following features on the server:

1) Created a configuration for NGINX + PHP-FPM for the following projects:
- www.warface.lo (php7.0)
- imageproxy.warface .lo (php5.6)
- www.cryengine.lo (php7.1)
- shop.cryengine.lo (php7.1)
- forum.cryengine.lo (php5.6)
2) The following versions of PHP with the appropriate modules are installed: 5.6, 7.0, 7.1, 7.2
3) The imagemagick package will be installed
4) OpenSSL is updated to the latest available version
5) MySQL root / root
6) Redis and Memcached services
7) Latest versions of Composer, NodeJS and NPM
8) Server bind9 + its configuration, which allows you to "resolve" the requests of the * .lo domain to the current host.
9) Docker

Structure

The repository structure combines the following concepts:
master branch - control repository ( control-repo )
production branch - configuration description of 'production'

Installation

The start-up process comes down to a few simple steps:

1) Install git + puppet + r10k
2) Initialize “control-repo”
2) Deploy the configuration using r10k
3) Run puppet apply

bash

#!/bin/bash
echo"Initialize"# https://docs.puppet.com/puppet/5.1/install_linux.html# https://docs.puppet.com/puppet/5.1/puppet_platform.html
wget --no-verbose https://apt.puppetlabs.com/puppet5-release-xenial.deb
dpkg -i --force-confdef puppet5-release-xenial.deb
rm -f puppet5-release-xenial.deb
echo"[APT]: ===="
apt-get update
sudo apt-get upgrade -y
apt install -o Dpkg::Options::="--force-confold" -y git puppet-agent r10k
echo"[APT]: Puppet"export PATH=/opt/puppetlabs/bin:$PATHecho"Puppet version is $(puppet --version)"echo"[PUPPET]: Control Repo"
git clone https://github.com/OxCom/puppet-php-skeleton-dev.git
cp -rf ./puppet-php-skeleton-dev/* /etc/puppetlabs/puppet/
rm -rf ./puppet-php-skeleton-dev
echo"[SSH]: ===="echo"[SSH]: Hosts"
ssh-keygen -R bitbucket.org
ssh-keyscan bitbucket.org >> ~/.ssh/known_hosts
ssh-keygen -R github.com
ssh-keyscan github.com >> ~/.ssh/known_hosts
echo"[PUPPET]: ===="echo"[PUPPET]: Running R10K"cd /etc/puppetlabs/puppet
r10k deploy environment -p -v
echo"[PUPPET]: Running puppet"
puppet apply /etc/puppetlabs/puppet/environments/production/manifests/site.pp --confdir=/etc/puppetlabs/puppet --environment=production --environmentpath=/etc/puppetlabs/puppet/environments/

Further modification

The following is a list of how to improve the current configuration and make it more flexible:

- Add classes that describe the process of project deployment (git clone, specific vhost, application settings, database deployment: user + schema + data)
- Add container launch classes for docker
- Generation of certificates (NGINX + HTTPS)

The implementation is far from ideal and does not always follow the rules, but here I would like to highlight:

- Always think about dependencies, since Puppet does not guarantee the initialization of classes in the order they are connected;
- Describe with hiera parameters that change the behavior of the class;
- Do not forget about the default settings;
- Do not reinvent the wheel: perhaps someone has already made the functionality that you need.

useful links

- Puppet Documentation
- R10K
- Puppet Modules
- Puppet Cookbook
- Setting up the development environment: needlework circle (Part 1)

PS : If you find any things in the repository that can be improved, write me about it and with an example or link, how it can be changed.

Tags: