Build and install Calamari monitoring system packages for CEPH 0.87 distributed storage on Ubuntu 14.04.1 (Trusty Tahr)
- From the sandbox
- Tutorial
Given: Existing CEPH cluster 0.87. Objective: to provide monitoring. Task number 2: Write an instruction (simpler).
I googled and found Calamari. I dug further and found several articles in English on assembly and installation. I tried to deliver, stepped on a few "rakes". As a result, I came to write this article.
I note that the packages collected as a result are hardly fully consistent with the debian-policy for building deb packages. Monitoring is more important.
Calamari is a CEPH distributed file storage cluster monitoring and management system. It includes:
Of course, you will need other software (for example, PostgresQL). But it does not need to be further configured, unlike those mentioned. It is only required to install.
1. Install fresh Ubuntu 14.04.1 (Trusty Tahr) .
In my version, the minimum standard server configuration is set, i.e. in /var/log/apt/history.log the following line is present
Thus, some packages (for example: software-properties-common ) are already present on the system.
Also, despite the fact that some packages required for assembly (for example: libssl-dev ) will come depending on the packages being installed, I will still indicate them for installation, since they are indicated in the documentation and the articles discussed.
For convenience, Midnight Commander is installed .
Created a ceph user with root privileges via sudo .
By default, aptitude is configured to automatically install recommended packages. Leave unchanged.
I assume that everything written earlier, you can do without any help. And then I will try to describe my actions in as much detail as possible.
2. Update:
3. If something important has been updated. For example, in my case, this is a package with the linux-image kernel , then we reboot
4. Change the time zone from Samara to Moscow (optional):
5. Install tools for building packages:
Important: In the next three paragraphs, I build the packages. Each of the packages can be built on a separate machine, but the initial distribution settings must be carried out.
Install the dependencies for the calamari-server assembly . They are listed in Build-Depends or discovered by me as important for the build process. As already mentioned at the beginning of the article, some of them are already installed.
We create a directory in which we will work and clone repositories with calamari-server .
We are trying to collect
We see an error
and three lines that callus eyes:
Calamari version is not the same. The distribution should be trusty . Yes, and I collect the package, not Gary. The error due to which the assembly of the package stopped, I also do not like.
Therefore, I will collect the package in my own way.
BugFix The dpkg-buildpackage error occurs because in the version from the developer, the
version '3.0 (quilt)' is specified as the debian package format . Change the format.
Build the package: Change the debian / changelog file in accordance with our name, email address, as well as the version, revision, release and importance of the package.
Check
After that, the package is built without errors, with the correct version, revision, release and importance. For obvious reasons, I do not cite the output of dpkg-buildpackage.
Check
Go back to the parent directory
Important: If we collect the package on a separate machine - do not forget about the implementation of 1 paragraph of the article.
Install dependencies for diamond assembly
Download the diamond repository .
Check
Collect package
Check
Go back to the parent directory
Important: If we collect the package on a separate machine - do not forget about the implementation of 1 paragraph of the article.
Add repositories with node.js :
Updating package lists:
Install the dependencies for building calamari-clients:
Install dependencies using NPM and Gem:
In some articles, it is recommended to install gem without sudo , but I swore access rights to system directories. Having stepped on one more "rake", put it through sudo and corrected the consequences of the impact of the root user by the next line
Note: Installing dependencies via NPM and Gem , and building calamari-clients for me coincided with the time of blocking github. Be carefull. Without access to this resource you will catch many problems. Perhaps not resolved without a full-fledged (not anonymous) proxy or vpn.
Download the calamari-clients repository
Download the patch (to the parent directory)
In case the github falls off again I will leave a diff-file here. When copying from the screen, the diff file is created crooked.
Apply the patch:
Build the calamari-clients deb package.
Check
Go back to the parent directory
I install the server on a separate machine. I fulfill all the subparagraphs of paragraph 1, except the last (5).
UPDATE 2015.07:
Saltstack from the developer has been updated to version 2015.07. Calamari does not work with him. Install saltstack as follows
Install dependencies for calamari-server
Install calamari-server and calamari-clients
Calamari setup
UPDATE 2015.07:
Saltstack from the developer has been updated to version 2015.07. Calamari does not work with him. Install saltstack as follows.
On all nodes
Install gdebi
Diamond and its settings
Salt-minion and its settings
Restart
After completing the previous paragraph, salt-minion clients from Ceph machines will try to upgrade from the wizard.
On a machine with calamari-server and salt-master
We see the list of cluster machines. We allow them access
We login through the browser to http: // CALAMARI_SERVER_IP with the username and password specified during setup (calamari-ctl initialize).
Diamond settings are synchronized on Ceph nodes automatically. However, there is a command to update settings manually
The following command will check the connectivity of the Ceph node with calamari-server
Check on salt-master all salt-minions connected to it
Note: Some intermediate results are not included in the article. For example, a patch that eliminates an unpleasant error is excluded . But, checking the article, I once again cloned calamari from the github, applied my patch and the patch did not work. In general, as it turned out, while I was writing the article - calamari'vtsy fixed a bug in the main branch.
Compiling Calamari for Ceph on Ubuntu 14.04
Ceph Calamari: Step-by-step
Build package and Install Calamari on Debian Weezy
Ceph Calamari: The Survival Guide
Calamari Documentation
Precise Build Instructions
I googled and found Calamari. I dug further and found several articles in English on assembly and installation. I tried to deliver, stepped on a few "rakes". As a result, I came to write this article.
I note that the packages collected as a result are hardly fully consistent with the debian-policy for building deb packages. Monitoring is more important.
0. Short description Calamari
Calamari is a CEPH distributed file storage cluster monitoring and management system. It includes:
- calamari-server - the server itself, the backend
- calamari-clients - frontend (web interface), installed with the server
- salt-master - server for managing configuration files, installed with calamari-server
- salt-minon - the client receiving the configuration files is installed on all machines in the CEPH cluster
- diamond - statistics collector, installed on all machines of the CEPH cluster
Of course, you will need other software (for example, PostgresQL). But it does not need to be further configured, unlike those mentioned. It is only required to install.
1. Initial distribution settings
1. Install fresh Ubuntu 14.04.1 (Trusty Tahr) .
In my version, the minimum standard server configuration is set, i.e. in /var/log/apt/history.log the following line is present
Commandline: apt-get -o APT::Status-Fd=4 -o APT::Keep-Fds::=5 -o APT::Keep-Fds::=6 -q -y install minimal^ openssh-server^ server^ standard^
Thus, some packages (for example: software-properties-common ) are already present on the system.
Also, despite the fact that some packages required for assembly (for example: libssl-dev ) will come depending on the packages being installed, I will still indicate them for installation, since they are indicated in the documentation and the articles discussed.
For convenience, Midnight Commander is installed .
Created a ceph user with root privileges via sudo .
By default, aptitude is configured to automatically install recommended packages. Leave unchanged.
I assume that everything written earlier, you can do without any help. And then I will try to describe my actions in as much detail as possible.
2. Update:
sudo aptitude update && sudo aptitude -y upgrade
3. If something important has been updated. For example, in my case, this is a package with the linux-image kernel , then we reboot
sudo reboot
4. Change the time zone from Samara to Moscow (optional):
sudo dpkg-reconfigure tzdata
5. Install tools for building packages:
sudo aptitude -y install build-essential debhelper devscripts git make g++
Important: In the next three paragraphs, I build the packages. Each of the packages can be built on a separate machine, but the initial distribution settings must be carried out.
2. Setting the environment for calamari-server and building the package
Install the dependencies for the calamari-server assembly . They are listed in Build-Depends or discovered by me as important for the build process. As already mentioned at the beginning of the article, some of them are already installed.
sudo aptitude -y install libcairo2-dev libpq-dev python-dev python-pip python-virtualenv python-crypto python-m2crypto python-mako \
python-msgpack python-zmq cython libssl-dev lsb-release openssl curl software-properties-common swig libzmq-dev python-cairo \
python-sphinx reprepro
We create a directory in which we will work and clone repositories with calamari-server .
ceph@calamari:~$ mkdir -p ~/dev; cd ~/dev
ceph@calamari:~/dev$ git clone https://github.com/ceph/calamari.git
ceph@calamari:~/dev$ cd calamari
We are trying to collect
ceph@calamari:~/dev/calamari$ dpkg-buildpackage
We get
dpkg-buildpackage: source package calamari
dpkg-buildpackage: source version 1.0.0-1
dpkg-buildpackage: source distribution precise
dpkg-buildpackage: source changed by Gary Lowell
dpkg-buildpackage: host architecture amd64
dpkg-source --before-build calamari
fakeroot debian/rules clean
dh clean --with python2
dh_testdir
dh_auto_clean
make -j1 clean
make[1]: Entering directory `/home/ceph/dev/calamari'
target: clean
rm -rf venv rest-api/calamari_rest/version.py
make[1]: Leaving directory `/home/ceph/dev/calamari'
dh_clean
rm -f debian/calamari-server.substvars
rm -f debian/calamari-server.*.debhelper
rm -rf debian/calamari-server/
rm -f debian/*.debhelper.log
rm -f debian/files
find . \( \( -type f -a \
\( -name '#*#' -o -name '.*~' -o -name '*~' -o -name DEADJOE \
-o -name '*.orig' -o -name '*.rej' -o -name '*.bak' \
-o -name '.*.orig' -o -name .*.rej -o -name '.SUMS' \
-o -name TAGS -o \( -path '*/.deps/*' -a -name '*.P' \) \
\) -exec rm -f {} + \) -o \
\( -type d -a -name autom4te.cache -prune -exec rm -rf {} + \) \)
rm -f *-stamp
dpkg-source -b calamari
dpkg-source: error: can't build with source format '3.0 (quilt)': no upstream tarball found at ../calamari_1.0.0.orig.tar.{bz2,gz,lzma,xz}
dpkg-buildpackage: error: dpkg-source -b calamari gave error exit status 255
We see an error
dpkg-source: error: can't build with source format '3.0 (quilt)': no upstream tarball found at ../calamari_1.0.0.orig.tar.{bz2,gz,lzma,xz}
dpkg-buildpackage: error: dpkg-source -b calamari gave error exit status 255
and three lines that callus eyes:
dpkg-buildpackage: source version 1.0.0-1
dpkg-buildpackage: source distribution precise
dpkg-buildpackage: source changed by Gary Lowell Calamari version is not the same. The distribution should be trusty . Yes, and I collect the package, not Gary. The error due to which the assembly of the package stopped, I also do not like.
Therefore, I will collect the package in my own way.
BugFix The dpkg-buildpackage error occurs because in the version from the developer, the
version '3.0 (quilt)' is specified as the debian package format . Change the format.
ceph@calamari:~/dev/calamari$ echo '3.0 (native)' > debian/source/format
Build the package: Change the debian / changelog file in accordance with our name, email address, as well as the version, revision, release and importance of the package.
ceph@calamari:~/dev/calamari$ DEBEMAIL=your@email.com DEBFULLNAME="Your Name" dch \
-v `./get-versions.sh VERSION`-`./get-versions.sh REVISION`-1 -D trusty -u low 'Switch to dpkg-source 3.0 (native) format'
Check
ceph@calamari:~/dev/calamari$ head -5 debian/changelog
calamari (1.2.1-100-ge0b9b21-1) trusty; urgency=low
* Switch to dpkg-source 3.0 (native) format
-- Your Name Wed, 10 Dec 2014 17:05:10 +0300
After that, the package is built without errors, with the correct version, revision, release and importance. For obvious reasons, I do not cite the output of dpkg-buildpackage.
ceph@calamari:~/dev/calamari$ dpkg-buildpackage
Check
ceph@calamari:~/dev/calamari$ ls -la ..
total 12016
drwxrwxr-x 3 ceph ceph 4096 Dec 10 17:20 .
drwxr-xr-x 5 ceph ceph 4096 Dec 10 17:11 ..
drwxrwxr-x 20 ceph ceph 4096 Dec 10 17:11 calamari
-rw-rw-r-- 1 ceph ceph 1396 Dec 10 17:20 calamari_1.2.1-100-ge0b9b21-1_amd64.changes
-rw-rw-r-- 1 ceph ceph 750 Dec 10 17:11 calamari_1.2.1-100-ge0b9b21-1.dsc
-rw-rw-r-- 1 ceph ceph 1278706 Dec 10 17:11 calamari_1.2.1-100-ge0b9b21-1.tar.gz
-rw-r--r-- 1 ceph ceph 10998618 Dec 10 17:20 calamari-server_1.2.1-100-ge0b9b21-1_amd64.deb
Go back to the parent directory
ceph@calamari:~/dev/calamari$ cd ..
3. Setting up the environment and assembling diamond
Important: If we collect the package on a separate machine - do not forget about the implementation of 1 paragraph of the article.
Install dependencies for diamond assembly
sudo aptitude -y install python-mock cdbs python-support python-configobj
Download the diamond repository .
mkdir -p ~/dev; cd ~/dev
ceph@calamari:~/dev$ git clone https://github.com/ceph/Diamond.git --branch=calamari
ceph@calamari:~/dev$ cd Diamond
ceph@calamari:~/dev/Diamond$ DEBEMAIL=your@email.com DEBFULLNAME="Your Name" dch -v `./version.sh`-1 -D trusty \
-u low `/bin/echo -n "built on "; date`
Check
ceph@calamari:~/dev/Diamond$ head -5 debian/changelog
diamond (3.4.67-1) trusty; urgency=low
* built on Wed Dec 10 17:57:02 MSK 2014
-- Your Name Wed, 10 Dec 2014 17:57:02 +0300
Collect package
ceph@calamari:~/dev/Diamond$ dpkg-buildpackage
Check
ceph@calamari:~/dev/Diamond$ ls -la ..
total 4572
drwxrwxr-x 3 ceph ceph 4096 Dec 10 18:00 .
drwxr-xr-x 4 ceph ceph 4096 Dec 10 17:54 ..
drwxrwxr-x 11 ceph ceph 4096 Dec 10 18:00 Diamond
-rw-r--r-- 1 ceph ceph 232292 Dec 10 18:00 diamond_3.4.67-1_all.deb
-rw-rw-r-- 1 ceph ceph 1182 Dec 10 18:00 diamond_3.4.67-1_amd64.changes
-rw-rw-r-- 1 ceph ceph 723 Dec 10 18:00 diamond_3.4.67-1.dsc
-rw-rw-r-- 1 ceph ceph 4427329 Dec 10 18:00 diamond_3.4.67-1.tar.gz
Go back to the parent directory
ceph@calamari:~/dev/Diamond$ cd ..
4. Setting the environment for calamari-clients and building it
Important: If we collect the package on a separate machine - do not forget about the implementation of 1 paragraph of the article.
Add repositories with node.js :
echo "deb http://ppa.launchpad.net/chris-lea/node.js/ubuntu trusty main" | sudo tee /etc/apt/sources.list.d/nodejs.list
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys B9316A7BC7917B12
Updating package lists:
sudo aptitude update
Install the dependencies for building calamari-clients:
sudo aptitude -y install ruby1.9.1 ruby1.9.1-dev python-software-properties nodejs
Install dependencies using NPM and Gem:
sudo npm install -g bower@1.3.8
sudo npm install -g grunt-cli
sudo gem install compass
In some articles, it is recommended to install gem without sudo , but I swore access rights to system directories. Having stepped on one more "rake", put it through sudo and corrected the consequences of the impact of the root user by the next line
sudo chown -R ceph ~/.npm
Note: Installing dependencies via NPM and Gem , and building calamari-clients for me coincided with the time of blocking github. Be carefull. Without access to this resource you will catch many problems. Perhaps not resolved without a full-fledged (not anonymous) proxy or vpn.
Download the calamari-clients repository
mkdir -p ~/dev; cd ~/dev
ceph@calamari:~/dev$ git clone https://github.com/ceph/calamari-clients.git
ceph@calamari:~/dev$ cd calamari-clients
Download the patch (to the parent directory)
ceph@calamari:~/dev/calamari-clients$ wget https://raw.githubusercontent.com/avssav/patches/master/calamari-clients/makefile.patch -qO \
../makefile.patch
In case the github falls off again I will leave a diff-file here. When copying from the screen, the diff file is created crooked.
../makefile.patch
ceph@calamari:~/dev/calamari-clients$ echo 'diff -ruN a/Makefile b/Makefile
--- a/Makefile 2014-12-03 10:13:32.486463458 +0300
+++ b/Makefile 2014-12-03 10:14:10.994462934 +0300
@@ -45,7 +45,7 @@
DATESTR=$(shell /bin/echo -n "built on "; date)
set_deb_version:
DEBEMAIL=$(DEBEMAIL) dch \
- --newversion $(VERSION)-$(REVISION)$(BPTAG) \
+ --newversion $(VERSION)-$(REVISION)$(BPTAG) -u low \
-D $(DIST) --force-bad-version --force-distribution "$(DATESTR)"
build:' > ../makefile.patch
Apply the patch:
ceph@calamari:~/dev/calamari-clients$ patch -ZEfsp1 < ../makefile.patch
Build the calamari-clients deb package.
ceph@calamari:~/dev/calamari-clients$ DEBEMAIL=your@email.com DEBFULLNAME="Your Name" DIST=trusty BPTAG=-1 REAL_BUILD=y make dpkg
Check
ceph@calamari:~/dev/calamari-clients$ ls -la ..
total 936
drwxrwxr-x 3 ceph ceph 4096 Dec 10 18:54 .
drwxr-xr-x 8 ceph ceph 4096 Dec 10 18:45 ..
drwxrwxr-x 11 ceph ceph 4096 Dec 10 18:41 calamari-clients
-rw-r--r-- 1 ceph ceph 838592 Dec 10 18:54 calamari-clients_1.2.1.1-53-gddd7187-1_all.deb
-rw-rw-r-- 1 ceph ceph 810 Dec 10 18:54 calamari-clients_1.2.1.1-53-gddd7187-1_amd64.changes
-rw-rw-r-- 1 ceph ceph 92902 Dec 10 18:54 calamari-clients-make-dpkg.txt
-rw-rw-r-- 1 ceph ceph 431 Dec 10 18:41 makefile.patch
Go back to the parent directory
ceph@calamari:~/dev/calamari-clients$ cd ..
5. Install calamari-server
I install the server on a separate machine. I fulfill all the subparagraphs of paragraph 1, except the last (5).
UPDATE 2015.07:
Saltstack from the developer has been updated to version 2015.07. Calamari does not work with him. Install saltstack as follows
sudo aptitude -y install python-pip
sudo pip install 'salt==2014.7.0'
Installation of salt-master until 2015.07 (I leave for information)
Add repositories with saltstack :
Install the configuration management system:
echo "deb http://ppa.launchpad.net/saltstack/salt/ubuntu trusty main" | sudo tee /etc/apt/sources.list.d/saltstack.list
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 4759FA960E27C0A6
sudo aptitude update
Install the configuration management system:
sudo aptitude -y install salt-master salt-minion salt-syndic
Install dependencies for calamari-server
sudo aptitude install -y apache2 libapache2-mod-wsgi libcairo2 supervisor python-cairo libpq5 postgresql python-txamqp python-gevent \
python-sqlalchemy
Install calamari-server and calamari-clients
sudo dpkg -i ./calamari-server_1.2.1-100-ge0b9b21-1_amd64.deb
sudo dpkg -i calamari-clients_1.2.1.1-53-gddd7187-1_all.deb
Calamari setup
ceph@calamari:~$ sudo calamari-ctl initialize
[INFO] Loading configuration..
[INFO] Starting/enabling salt...
[INFO] Starting/enabling postgres...
[INFO] Initializing database...
[INFO] Initializing web interface...
[INFO] You will now be prompted for login details for the administrative user account. This is the account you will use to log into the web interface once setup is complete.
Username (leave blank to use 'root'):
Email address: your@email.com
Password:
Password (again):
Superuser created successfully.
[INFO] Starting/enabling services...
[INFO] Restarting services...
[INFO] Complete.
6. Add Ceph nodes to calamari monitoring
UPDATE 2015.07:
Saltstack from the developer has been updated to version 2015.07. Calamari does not work with him. Install saltstack as follows.
On all nodes
Install gdebi
sudo aptitude -y install gdebi-core
Diamond and its settings
sudo gdebi --option=APT::Get::force-yes=1,APT::Get::Assume-Yes=1 -n diamond_3.4.67-1_all.deb
sudo cp /etc/diamond/diamond.conf.example /etc/diamond/diamond.conf
Salt-minion and its settings
wget https://github.com/avssav/packages/raw/master/saltstack/salt-common_2014.7.0%2Bds-2trusty1_all.deb
wget https://github.com/avssav/packages/raw/master/saltstack/salt-minion_2014.7.0%2Bds-2trusty1_all.deb
sudo gdebi --option=APT::Get::force-yes=1,APT::Get::Assume-Yes=1 -n salt-common_2014.7.0+ds-2trusty1_all.deb
sudo gdebi --option=APT::Get::force-yes=1,APT::Get::Assume-Yes=1 -n salt-minion_2014.7.0+ds-2trusty1_all.deb
echo "master: CALAMARI_SERVER_IP" | sudo tee /etc/salt/minion.d/calamari.conf
sudo update-rc.d salt-minion defaults
Setting salt-minion until 2015.07 (I leave for information)
On all nodes
Add a repository with salt-minion
Install dependencies for installing diamond
Diamond and its settings
Salt-minion and its settings
Add a repository with salt-minion
echo "deb http://ppa.launchpad.net/saltstack/salt/ubuntu trusty main" | sudo tee /etc/apt/sources.list.d/saltstack.list
wget -q -O - "http://keyserver.ubuntu.com:11371/pks/lookup?op=get&search=0x4759FA960E27C0A6" | sudo apt-key add -
sudo aptitude update
Install dependencies for installing diamond
sudo aptitude -y install python-support python-configobj
Diamond and its settings
sudo dpkg -i diamond_3.4.67-1_all.deb
sudo cp /etc/diamond/diamond.conf.example /etc/diamond/diamond.conf
Salt-minion and its settings
sudo aptitude -y install salt-minion
echo "master: CALAMARI_SERVER_IP" | sudo tee /etc/salt/minion.d/calamari.conf
sudo update-rc.d salt-minion defaults
Restart
sudo service salt-minion restart
sudo service diamond restart
7. Authorization of the Ceph cluster in Calamari
After completing the previous paragraph, salt-minion clients from Ceph machines will try to upgrade from the wizard.
On a machine with calamari-server and salt-master
ceph@calamari:~$ sudo salt-key -L
Accepted Keys:
Unaccepted Keys:
ceph1
ceph2
ceph3
ceph4
ceph5
Rejected Keys:
We see the list of cluster machines. We allow them access
ceph@calamari:~$ sudo salt-key -A
The following keys are going to be accepted:
Unaccepted Keys:
ceph1
ceph2
ceph3
ceph4
ceph5
Proceed? [n/Y] y
Key for minion ceph1 accepted.
Key for minion ceph2 accepted.
Key for minion ceph3 accepted.
Key for minion ceph4 accepted.
Key for minion ceph5 accepted.
9. Everything should work
We login through the browser to http: // CALAMARI_SERVER_IP with the username and password specified during setup (calamari-ctl initialize).
10. Useful little things
Diamond settings are synchronized on Ceph nodes automatically. However, there is a command to update settings manually
sudo salt-call state.highstate
The following command will check the connectivity of the Ceph node with calamari-server
sudo salt-call ceph.get_heartbeats
Check on salt-master all salt-minions connected to it
sudo salt '*' ceph.get_heartbeats
Note: Some intermediate results are not included in the article. For example, a patch that eliminates an unpleasant error is excluded . But, checking the article, I once again cloned calamari from the github, applied my patch and the patch did not work. In general, as it turned out, while I was writing the article - calamari'vtsy fixed a bug in the main branch.
9. Materials
Compiling Calamari for Ceph on Ubuntu 14.04
Ceph Calamari: Step-by-step
Build package and Install Calamari on Debian Weezy
Ceph Calamari: The Survival Guide
Calamari Documentation
Precise Build Instructions