Build and install Calamari monitoring system packages for CEPH 0.87 distributed storage on Ubuntu 14.04.1 (Trusty Tahr)

Given: Existing CEPH cluster 0.87. Objective: to provide monitoring. Task number 2: Write an instruction (simpler).

I googled and found Calamari. I dug further and found several articles in English on assembly and installation. I tried to deliver, stepped on a few "rakes". As a result, I came to write this article.
I note that the packages collected as a result are hardly fully consistent with the debian-policy for building deb packages. Monitoring is more important.

0. Short description Calamari


Calamari is a CEPH distributed file storage cluster monitoring and management system. It includes:
  • calamari-server - the server itself, the backend
  • calamari-clients - frontend (web interface), installed with the server
  • salt-master - server for managing configuration files, installed with calamari-server
  • salt-minon - the client receiving the configuration files is installed on all machines in the CEPH cluster
  • diamond - statistics collector, installed on all machines of the CEPH cluster

Of course, you will need other software (for example, PostgresQL). But it does not need to be further configured, unlike those mentioned. It is only required to install.

1. Initial distribution settings


1. Install fresh Ubuntu 14.04.1 (Trusty Tahr) .
In my version, the minimum standard server configuration is set, i.e. in /var/log/apt/history.log the following line is present
Commandline: apt-get -o APT::Status-Fd=4 -o APT::Keep-Fds::=5 -o APT::Keep-Fds::=6 -q -y install minimal^ openssh-server^ server^ standard^

Thus, some packages (for example: software-properties-common ) are already present on the system.
Also, despite the fact that some packages required for assembly (for example: libssl-dev ) will come depending on the packages being installed, I will still indicate them for installation, since they are indicated in the documentation and the articles discussed.

For convenience, Midnight Commander is installed .

Created a ceph user with root privileges via sudo .

By default, aptitude is configured to automatically install recommended packages. Leave unchanged.

I assume that everything written earlier, you can do without any help. And then I will try to describe my actions in as much detail as possible.
2. Update:
sudo aptitude update && sudo aptitude -y upgrade

3. If something important has been updated. For example, in my case, this is a package with the linux-image kernel , then we reboot
sudo reboot

4. Change the time zone from Samara to Moscow (optional):
sudo dpkg-reconfigure tzdata

5. Install tools for building packages:
sudo aptitude -y install build-essential debhelper devscripts git make g++

Important: In the next three paragraphs, I build the packages. Each of the packages can be built on a separate machine, but the initial distribution settings must be carried out.

2. Setting the environment for calamari-server and building the package


Install the dependencies for the calamari-server assembly . They are listed in Build-Depends or discovered by me as important for the build process. As already mentioned at the beginning of the article, some of them are already installed.
sudo aptitude -y install libcairo2-dev libpq-dev python-dev python-pip python-virtualenv python-crypto python-m2crypto python-mako \
python-msgpack python-zmq cython libssl-dev lsb-release openssl curl software-properties-common swig libzmq-dev python-cairo \
python-sphinx reprepro

We create a directory in which we will work and clone repositories with calamari-server .
ceph@calamari:~$ mkdir -p ~/dev; cd ~/dev
ceph@calamari:~/dev$ git clone https://github.com/ceph/calamari.git 
ceph@calamari:~/dev$ cd calamari

We are trying to collect
ceph@calamari:~/dev/calamari$ dpkg-buildpackage

We get
dpkg-buildpackage: source package calamari
dpkg-buildpackage: source version 1.0.0-1
dpkg-buildpackage: source distribution precise
dpkg-buildpackage: source changed by Gary Lowell 
dpkg-buildpackage: host architecture amd64
 dpkg-source --before-build calamari
 fakeroot debian/rules clean
dh clean --with python2
   dh_testdir
   dh_auto_clean
        make -j1 clean
make[1]: Entering directory `/home/ceph/dev/calamari'
target: clean
rm -rf venv rest-api/calamari_rest/version.py
make[1]: Leaving directory `/home/ceph/dev/calamari'
   dh_clean
        rm -f debian/calamari-server.substvars
        rm -f debian/calamari-server.*.debhelper
        rm -rf debian/calamari-server/
        rm -f debian/*.debhelper.log
        rm -f debian/files
        find .  \( \( -type f -a \
                \( -name '#*#' -o -name '.*~' -o -name '*~' -o -name DEADJOE \
                 -o -name '*.orig' -o -name '*.rej' -o -name '*.bak' \
                 -o -name '.*.orig' -o -name .*.rej -o -name '.SUMS' \
                 -o -name TAGS -o \( -path '*/.deps/*' -a -name '*.P' \) \
                \) -exec rm -f {} + \) -o \
                \( -type d -a -name autom4te.cache -prune -exec rm -rf {} + \) \)
        rm -f *-stamp
 dpkg-source -b calamari
dpkg-source: error: can't build with source format '3.0 (quilt)': no upstream tarball found at ../calamari_1.0.0.orig.tar.{bz2,gz,lzma,xz}
dpkg-buildpackage: error: dpkg-source -b calamari gave error exit status 255


We see an error
dpkg-source: error: can't build with source format '3.0 (quilt)': no upstream tarball found at ../calamari_1.0.0.orig.tar.{bz2,gz,lzma,xz}
dpkg-buildpackage: error: dpkg-source -b calamari gave error exit status 255

and three lines that callus eyes:
dpkg-buildpackage: source version 1.0.0-1
dpkg-buildpackage: source distribution precise
dpkg-buildpackage: source changed by Gary Lowell 

Calamari version is not the same. The distribution should be trusty . Yes, and I collect the package, not Gary. The error due to which the assembly of the package stopped, I also do not like.

Therefore, I will collect the package in my own way.

BugFix The dpkg-buildpackage error occurs because in the version from the developer, the
version '3.0 (quilt)' is specified as the debian package format . Change the format.
ceph@calamari:~/dev/calamari$ echo '3.0 (native)' > debian/source/format 

Build the package: Change the debian / changelog file in accordance with our name, email address, as well as the version, revision, release and importance of the package.
ceph@calamari:~/dev/calamari$ DEBEMAIL=your@email.com DEBFULLNAME="Your Name" dch \
-v `./get-versions.sh VERSION`-`./get-versions.sh REVISION`-1 -D trusty -u low 'Switch to dpkg-source 3.0 (native) format'

Check
ceph@calamari:~/dev/calamari$ head -5 debian/changelog
calamari (1.2.1-100-ge0b9b21-1) trusty; urgency=low
  * Switch to dpkg-source 3.0 (native) format
 -- Your Name   Wed, 10 Dec 2014 17:05:10 +0300

After that, the package is built without errors, with the correct version, revision, release and importance. For obvious reasons, I do not cite the output of dpkg-buildpackage.
ceph@calamari:~/dev/calamari$ dpkg-buildpackage

Check
ceph@calamari:~/dev/calamari$ ls -la ..
total 12016
drwxrwxr-x  3 ceph ceph     4096 Dec 10 17:20 .
drwxr-xr-x  5 ceph ceph     4096 Dec 10 17:11 ..
drwxrwxr-x 20 ceph ceph     4096 Dec 10 17:11 calamari
-rw-rw-r--  1 ceph ceph     1396 Dec 10 17:20 calamari_1.2.1-100-ge0b9b21-1_amd64.changes
-rw-rw-r--  1 ceph ceph      750 Dec 10 17:11 calamari_1.2.1-100-ge0b9b21-1.dsc
-rw-rw-r--  1 ceph ceph  1278706 Dec 10 17:11 calamari_1.2.1-100-ge0b9b21-1.tar.gz
-rw-r--r--  1 ceph ceph 10998618 Dec 10 17:20 calamari-server_1.2.1-100-ge0b9b21-1_amd64.deb

Go back to the parent directory
ceph@calamari:~/dev/calamari$ cd ..

3. Setting up the environment and assembling diamond


Important: If we collect the package on a separate machine - do not forget about the implementation of 1 paragraph of the article.

Install dependencies for diamond assembly
sudo aptitude -y install python-mock cdbs python-support python-configobj

Download the diamond repository .
mkdir -p ~/dev; cd ~/dev
ceph@calamari:~/dev$ git clone https://github.com/ceph/Diamond.git --branch=calamari
ceph@calamari:~/dev$ cd Diamond
ceph@calamari:~/dev/Diamond$ DEBEMAIL=your@email.com DEBFULLNAME="Your Name" dch -v `./version.sh`-1 -D trusty \
-u low `/bin/echo -n "built on "; date`

Check
ceph@calamari:~/dev/Diamond$ head -5 debian/changelog
diamond (3.4.67-1) trusty; urgency=low
  * built on Wed Dec 10 17:57:02 MSK 2014
 -- Your Name   Wed, 10 Dec 2014 17:57:02 +0300

Collect package
ceph@calamari:~/dev/Diamond$ dpkg-buildpackage

Check
ceph@calamari:~/dev/Diamond$ ls -la ..
total 4572
drwxrwxr-x  3 ceph ceph    4096 Dec 10 18:00 .
drwxr-xr-x  4 ceph ceph    4096 Dec 10 17:54 ..
drwxrwxr-x 11 ceph ceph    4096 Dec 10 18:00 Diamond
-rw-r--r--  1 ceph ceph  232292 Dec 10 18:00 diamond_3.4.67-1_all.deb
-rw-rw-r--  1 ceph ceph    1182 Dec 10 18:00 diamond_3.4.67-1_amd64.changes
-rw-rw-r--  1 ceph ceph     723 Dec 10 18:00 diamond_3.4.67-1.dsc
-rw-rw-r--  1 ceph ceph 4427329 Dec 10 18:00 diamond_3.4.67-1.tar.gz

Go back to the parent directory
ceph@calamari:~/dev/Diamond$ cd ..

4. Setting the environment for calamari-clients and building it


Important: If we collect the package on a separate machine - do not forget about the implementation of 1 paragraph of the article.

Add repositories with node.js :
echo "deb http://ppa.launchpad.net/chris-lea/node.js/ubuntu trusty main" | sudo tee /etc/apt/sources.list.d/nodejs.list
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys B9316A7BC7917B12

Updating package lists:
sudo aptitude update

Install the dependencies for building calamari-clients:
sudo aptitude -y install ruby1.9.1 ruby1.9.1-dev python-software-properties nodejs

Install dependencies using NPM and Gem:
sudo npm install -g bower@1.3.8
sudo npm install -g grunt-cli
sudo gem install compass

In some articles, it is recommended to install gem without sudo , but I swore access rights to system directories. Having stepped on one more "rake", put it through sudo and corrected the consequences of the impact of the root user by the next line
sudo chown -R ceph  ~/.npm

Note: Installing dependencies via NPM and Gem , and building calamari-clients for me coincided with the time of blocking github. Be carefull. Without access to this resource you will catch many problems. Perhaps not resolved without a full-fledged (not anonymous) proxy or vpn.

Download the calamari-clients repository
mkdir -p ~/dev; cd ~/dev
ceph@calamari:~/dev$ git clone https://github.com/ceph/calamari-clients.git
ceph@calamari:~/dev$ cd calamari-clients

Download the patch (to the parent directory)
ceph@calamari:~/dev/calamari-clients$ wget https://raw.githubusercontent.com/avssav/patches/master/calamari-clients/makefile.patch -qO \
 ../makefile.patch

In case the github falls off again I will leave a diff-file here. When copying from the screen, the diff file is created crooked.
../makefile.patch
ceph@calamari:~/dev/calamari-clients$ echo 'diff -ruN a/Makefile b/Makefile
--- a/Makefile  2014-12-03 10:13:32.486463458 +0300
+++ b/Makefile  2014-12-03 10:14:10.994462934 +0300
@@ -45,7 +45,7 @@
 DATESTR=$(shell /bin/echo -n "built on "; date)
 set_deb_version:
        DEBEMAIL=$(DEBEMAIL) dch \
-               --newversion $(VERSION)-$(REVISION)$(BPTAG) \
+               --newversion $(VERSION)-$(REVISION)$(BPTAG) -u low \
                -D $(DIST) --force-bad-version --force-distribution "$(DATESTR)"
 build:' > ../makefile.patch


Apply the patch:
ceph@calamari:~/dev/calamari-clients$ patch -ZEfsp1 < ../makefile.patch

Build the calamari-clients deb package.
ceph@calamari:~/dev/calamari-clients$ DEBEMAIL=your@email.com DEBFULLNAME="Your Name" DIST=trusty BPTAG=-1 REAL_BUILD=y make dpkg

Check
ceph@calamari:~/dev/calamari-clients$ ls -la ..
total 936
drwxrwxr-x  3 ceph ceph   4096 Dec 10 18:54 .
drwxr-xr-x  8 ceph ceph   4096 Dec 10 18:45 ..
drwxrwxr-x 11 ceph ceph   4096 Dec 10 18:41 calamari-clients
-rw-r--r--  1 ceph ceph 838592 Dec 10 18:54 calamari-clients_1.2.1.1-53-gddd7187-1_all.deb
-rw-rw-r--  1 ceph ceph    810 Dec 10 18:54 calamari-clients_1.2.1.1-53-gddd7187-1_amd64.changes
-rw-rw-r--  1 ceph ceph  92902 Dec 10 18:54 calamari-clients-make-dpkg.txt
-rw-rw-r--  1 ceph ceph    431 Dec 10 18:41 makefile.patch

Go back to the parent directory
ceph@calamari:~/dev/calamari-clients$ cd ..

5. Install calamari-server


I install the server on a separate machine. I fulfill all the subparagraphs of paragraph 1, except the last (5).

UPDATE 2015.07:
Saltstack from the developer has been updated to version 2015.07. Calamari does not work with him. Install saltstack as follows
sudo aptitude -y install python-pip
sudo pip install 'salt==2014.7.0'

Installation of salt-master until 2015.07 (I leave for information)
Add repositories with saltstack :
echo "deb http://ppa.launchpad.net/saltstack/salt/ubuntu trusty main" | sudo tee /etc/apt/sources.list.d/saltstack.list
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 4759FA960E27C0A6
sudo aptitude update

Install the configuration management system:
sudo aptitude -y install salt-master salt-minion salt-syndic


Install dependencies for calamari-server
sudo aptitude install -y apache2 libapache2-mod-wsgi libcairo2 supervisor python-cairo libpq5 postgresql python-txamqp python-gevent \
python-sqlalchemy

Install calamari-server and calamari-clients
sudo dpkg -i ./calamari-server_1.2.1-100-ge0b9b21-1_amd64.deb
sudo dpkg -i calamari-clients_1.2.1.1-53-gddd7187-1_all.deb

Calamari setup
ceph@calamari:~$ sudo calamari-ctl initialize
[INFO] Loading configuration..
[INFO] Starting/enabling salt...
[INFO] Starting/enabling postgres...
[INFO] Initializing database...
[INFO] Initializing web interface...
[INFO] You will now be prompted for login details for the administrative user account.  This is the account you will use to log into the web interface once setup is complete.
Username (leave blank to use 'root'): 
Email address: your@email.com
Password:
Password (again):
Superuser created successfully.
[INFO] Starting/enabling services...
[INFO] Restarting services...
[INFO] Complete.

6. Add Ceph nodes to calamari monitoring


UPDATE 2015.07:
Saltstack from the developer has been updated to version 2015.07. Calamari does not work with him. Install saltstack as follows.
On all nodes
Install gdebi
sudo aptitude -y install gdebi-core

Diamond and its settings
sudo gdebi --option=APT::Get::force-yes=1,APT::Get::Assume-Yes=1 -n diamond_3.4.67-1_all.deb
sudo cp /etc/diamond/diamond.conf.example /etc/diamond/diamond.conf

Salt-minion and its settings
wget https://github.com/avssav/packages/raw/master/saltstack/salt-common_2014.7.0%2Bds-2trusty1_all.deb
wget https://github.com/avssav/packages/raw/master/saltstack/salt-minion_2014.7.0%2Bds-2trusty1_all.deb
sudo gdebi --option=APT::Get::force-yes=1,APT::Get::Assume-Yes=1 -n salt-common_2014.7.0+ds-2trusty1_all.deb
sudo gdebi --option=APT::Get::force-yes=1,APT::Get::Assume-Yes=1 -n salt-minion_2014.7.0+ds-2trusty1_all.deb
echo "master: CALAMARI_SERVER_IP" | sudo tee /etc/salt/minion.d/calamari.conf
sudo update-rc.d salt-minion defaults


Setting salt-minion until 2015.07 (I leave for information)
On all nodes
Add a repository with salt-minion
echo "deb http://ppa.launchpad.net/saltstack/salt/ubuntu trusty main" | sudo tee /etc/apt/sources.list.d/saltstack.list
wget -q -O - "http://keyserver.ubuntu.com:11371/pks/lookup?op=get&search=0x4759FA960E27C0A6" | sudo apt-key add -
sudo aptitude update

Install dependencies for installing diamond
sudo aptitude -y install python-support python-configobj

Diamond and its settings
sudo dpkg -i diamond_3.4.67-1_all.deb
sudo cp /etc/diamond/diamond.conf.example /etc/diamond/diamond.conf

Salt-minion and its settings
sudo aptitude -y install salt-minion
echo "master: CALAMARI_SERVER_IP" | sudo tee /etc/salt/minion.d/calamari.conf
sudo update-rc.d salt-minion defaults


Restart
sudo service salt-minion restart
sudo service diamond restart

7. Authorization of the Ceph cluster in Calamari


After completing the previous paragraph, salt-minion clients from Ceph machines will try to upgrade from the wizard.
On a machine with calamari-server and salt-master
ceph@calamari:~$ sudo salt-key -L
Accepted Keys:
Unaccepted Keys:
ceph1
ceph2
ceph3
ceph4
ceph5
Rejected Keys:

We see the list of cluster machines. We allow them access
ceph@calamari:~$ sudo salt-key -A
The following keys are going to be accepted:
Unaccepted Keys:
ceph1
ceph2
ceph3
ceph4
ceph5
Proceed? [n/Y] y
Key for minion ceph1 accepted.
Key for minion ceph2 accepted.
Key for minion ceph3 accepted.
Key for minion ceph4 accepted.
Key for minion ceph5 accepted.

9. Everything should work


We login through the browser to http: // CALAMARI_SERVER_IP with the username and password specified during setup (calamari-ctl initialize).

10. Useful little things


Diamond settings are synchronized on Ceph nodes automatically. However, there is a command to update settings manually
sudo salt-call state.highstate

The following command will check the connectivity of the Ceph node with calamari-server
sudo salt-call ceph.get_heartbeats

Check on salt-master all salt-minions connected to it
sudo salt '*' ceph.get_heartbeats

Note: Some intermediate results are not included in the article. For example, a patch that eliminates an unpleasant error is excluded . But, checking the article, I once again cloned calamari from the github, applied my patch and the patch did not work. In general, as it turned out, while I was writing the article - calamari'vtsy fixed a bug in the main branch.

9. Materials


Compiling Calamari for Ceph on Ubuntu 14.04
Ceph Calamari: Step-by-step
Build package and Install Calamari on Debian Weezy
Ceph Calamari: The Survival Guide
Calamari Documentation
Precise Build Instructions

Also popular now: