Monitoring changes to network device configuration files

    Good day.

    Today I’ll try to talk about a utility developed for monitoring network device configuration files - RANCID (Really Awesome New Cisco config Differ). RANCID - development of the American company Shrubbery Networks, Inc. , this is a team with over 20 years of experience administering, configuring and supporting Unix systems and network devices. The system was developed for Unix, Linux, and MAC OS X. Although Cisco Systems is mentioned in the name, the system also works with equipment of other brands, the off-line quote is “Rancid currently supports Cisco routers, Juniper routers, Catalyst switches, Foundry switches, Redback "NASs, ADC EZT3 muxes, MRTd (and thus likely IRRd), Alteon switches, and HP Procurve switches and a host of others."This product is used by such companies as AOL, Global Crossing, MFN, NTT America, Certainty Solutions Inc. I will consider installing and configuring RANCID to store configurations of Cisco routers.

    First, a few words about how it works.
    RANCID automatically downloads configuration files from network devices and compares them with previous versions, which allows you to track configuration changes and keep track of configuration changes for your network devices. There is the possibility of email alerts when a configuration is changed and configuration files are downloaded to the device. RANCID uses version control systems - Subversion or CVS (Concurrent Version System) to store configuration changes.
    RANCID works according to the following algorithm:

    1. We are connected to the network device (the list of devices is indicated in the router.db file - a certain database of network devices used).
    2. We execute the commands necessary to obtain information from the device that we want to store.
    3. We bring the received information to the necessary form, format.
    4. We check if the configuration of the device has changed and if it has changed, we send a notification by e-mail, if necessary.
    5. And finally, we change the information in the version control system.
    I will consider installation and configuration using the example of version RANCID 2.3.2 (version 2.3.3 is currently available, the list of changes can be found here ), we will install on Ubuntu 10.04 Server Edition LAMP, (for FreeBSD, RANCID can be installed from ports).
    To simplify, let's say that we have several routers in the branches of the company, for the configurations that we need to monitor and store them, taking into account all changes.
    So, in order:

    Installation


    Suppose we have a server with Ubuntu Server 10.04 installed, only one command is required to install RANCID:

    aptitude install rancid

    During the installation, the system will warn about dependencies, and if everything went fine, then we will see the following: As you can see from the listing, except for the rancid package, cvs were installed , expect and tcl. With cvs, everything is clear, it is a version control system that is used by default in rancid. Why expect and tcl? And again, quote :

    root@test:~# aptitude install rancid
    Чтение списков пакетов... Готово
    Построение дерева зависимостей
    Чтение информации о состоянии... Готово
    Reading extended state information
    Initializing package states... Готово
    The following NEW packages will be installed:
    cvs{a} expect{a} libfile-copy-recursive-perl{a} rancid tcl8.5{a} update-inetd{a}
    0 packages upgraded, 6 newly installed, 0 to remove and 27 not upgraded.
    Need to get 3 778kB of archives. After unpacking 10,1MB will be used.
    Do you want to continue? [Y/n/?] Y
    Writing extended state information... Готово
    Fetched 3 778kB in 2с (1 633kB/s)
    Предварительная настройка пакетов ...
    Выбор ранее не выбранного пакета libfile-copy-recursive-perl.
    (Чтение базы данных ... на данный момент установлено 43319 файлов и каталогов.)
    Распаковывается пакет libfile-copy-recursive-perl (из файла .../libfile-copy-recursive-perl_0.38-1_all.deb)...
    Выбор ранее не выбранного пакета update-inetd.
    Распаковывается пакет update-inetd (из файла .../update-inetd_4.35_all.deb)...
    Выбор ранее не выбранного пакета cvs.
    Распаковывается пакет cvs (из файла .../cvs_1%3a1.12.13-12ubuntu1_i386.deb)...
    Выбор ранее не выбранного пакета tcl8.5.
    Распаковывается пакет tcl8.5 (из файла .../tcl8.5_8.5.8-2_i386.deb)...
    Выбор ранее не выбранного пакета expect.
    Распаковывается пакет expect (из файла .../expect_5.44.1.14-5_i386.deb)...
    Выбор ранее не выбранного пакета rancid.
    Распаковывается пакет rancid (из файла .../rancid_2.3.2-1_i386.deb)...
    Обрабатываются триггеры для man-db ...
    Обрабатываются триггеры для install-info ...
    Настраивается пакет libfile-copy-recursive-perl (0.38-1) ...
    Настраивается пакет update-inetd (4.35) ...
    Настраивается пакет cvs (1:1.12.13-12ubuntu1) ...
    Ignoring install-info called from maintainer script
    The package cvs should be rebuilt with new debhelper to get trigger support
    Ignoring install-info called from maintainer script
    The package cvs should be rebuilt with new debhelper to get trigger support
    Настраивается пакет tcl8.5 (8.5.8-2) ...
    update-alternatives: используется `/usr/bin/tclsh8.5' для предоставления `/usr/bin/tclsh' (tclsh) в автоматический режим.
    Настраивается пакет expect (5.44.1.14-5) ...
    Настраивается пакет rancid (2.3.2-1) ...
    Обрабатываются триггеры для libc-bin ...
    ldconfig deferred processing now taking place
    Чтение списков пакетов... Готово
    Построение дерева зависимостей
    Чтение информации о состоянии... Готово
    Reading extended state information
    Initializing package states... Готово
    Writing extended state information... Готово



    “The Expect tool is primarily intended for managing interactive programs that prompt and expect the user to enter a response from the keyboard, such as passwd, ftp, telnet, rlogin, su and many others. In ordinary shell scripts, such programs require the presence of a person entering answers to their requests. Expect allows you to automate the dialogue mode using simple scripts based on the Tcl language, and in addition, it provides step-by-step control of subprocesses: you can check the result of each operation, and determine the input data by the result of the check. ”
    Now that we have installed rancid, we understand the algorithm of its operation, we know the tools used by rancid - we can begin to configure it.

    Customization


    After installation, by default, the files will be in the following directories:
    / var / log / rancid - directory of logs;
    / var / lib / rancid - installation directory RANCID (BASEDIR);
    / usr / share / doc / rancid– documentation and examples of configuration files;
    / etc / rancid - the directory where the configuration file is stored - rancid.conf, this file contains settings for variables, such as job execution delay, the number of attempts to connect to the device, connection timeouts, it also indicates device groups, notification parameters and directory paths RANCID
    It is assumed that network devices will be grouped by some parameters, such as the type of equipment or its geographical location. The group name is specified in the LIST_OF_GROUPS parameter of the rancid.conf file. For example, LIST_OF_GROUPS = “routers”.
    After installation, you need to create a .cloginrc file or edit the sample file located at /usr/share/doc/rancid/examples/cloginrc.sample. This file is placed in the user's home directory with the rights of which RANCID scripts will be launched. For security reasons, only this user needs to set read and write permissions to the file: This is an important point, since RANCID will not work if the file is read / write to other users. You can specify the following connection parameters in .cloginrc:

    root@test:/root# chmod 600 .cloginrc
    root@test:/root# ls -la
    total 32
    drwx------ 5 root root 4096 2010-06-09 12:10 .
    drwxr-xr-x 22 root root 4096 2010-06-09 10:38 ..
    drwx------ 2 root root 4096 2010-06-09 10:58 .aptitude
    -rw-r--r-- 1 root root 3106 2010-04-23 13:45 .bashrc
    -rw------- 1 root root 153 2010-06-09 12:10 .cloginrc
    drwxr-xr-x 2 root root 4096 2010-06-09 10:59 .debtags
    -rw-r--r-- 1 root root 140 2010-04-23 13:45 .profile
    drwx------ 2 root root 4096 2010-06-09 10:51 .ssh




    add method <device name, IP address> {telnet, ssh} - connection method to the device, telnet or ssh, you can specify both, the first one in the list will be more priority;
    add user <device name, IP address> {username} - the username that will be used to connect to the router group;
    add password <device name, IP address> {vtypassword} {enablepassword} - passwords for connection, respectively, the first will be used to connect to the device, the second for privileged mode;
    add autoenable <device name, IP address> {0,1} - allows you to automatically use privileged mode after connecting.
    I got the following file:

    add user 10.5.51.* root
    add password 10.5.51.* Rsduj%0
    add user * admin
    add autoenable * 1
    add password * P@ssw0rd83
    add method * telnet ssh


    To connect to devices whose IP address falls under the mask 10.5.51. * Username will be used - root, password - Rsduj% 0, for all others - admin, P @ ssw0rd83.
    The next step - running the rancid-cvs script: Rancid-cvs - creates all the necessary directories and configuration files for each of the groups specified in the LIST_OF_GROUPS variable of the rancid.conf file and imports them into CVS. This script should be run every time after adding new changes to existing device groups. For each group, a routers.db file is created which stores a list of devices in the group, the type of devices and their status. Routers.db has the following record format: “router: mfg: state” router - FQDN name or ip address of the router; mfg - manufacturer (cisco | cat5 | juniper ..);

    root@test:/var/lib/rancid/bin# ./rancid-cvs

    No conflicts created by this import

    cvs checkout: Updating routers
    Directory /var/lib/rancid/CVS/routers/configs added to the repository
    cvs commit: Examining configs
    cvs add: scheduling file `router.db' for addition
    cvs add: use `cvs commit' to add this file permanently
    /var/lib/rancid/CVS/routers/router.db,v <-- router.db
    initial revision: 1.1
    root@test:/var/lib/rancid/bin#






    state - the state of the device (up or down), if you specify down, the script will simply ignore this device.
    My routers.db has the following form: This completes the setup, you can begin to check and run the script. I gave an example of the simplest setup, there are many more parameters in the rancid.conf file, all of them contain comments, so figuring out what’s easy is not a problem.

    10.5.51.17:cisco:up:"backbone"
    10.5.51.25:cisco:up:"router1"
    10.5.51.33:cisco:up:"router2"
    10.5.52.41:cisco:up:"gateway1"
    10.5.52.49:cisco:up:"gateway2"
    10.5.53.57:cisco:down:"old_"
    10.5.54.105:cisco:up:"851_12"
    10.5.55.113:cisco:up
    10.5.55.129:cisco:up




    Check and run


    We check the availability of the device and the correctness of the parameters specified in cloginrc. We managed to connect to the device and execute the command, which means we configured everything correctly and you can run rancid-run. This script will connect to all routers from the routers.db file for each group. After running rancid-run, a file appears in the log folder: With the following contents: In the routers.db file, 8 routers were specified in the up state, the configs of which were saved in the folder with the group name and one device was in the down state - rancid to it -run did not try to connect. Configuration files are stored in the directory - / var / lib / rancid / routers / configs.

    root@test:/var/lib/rancid/bin# ./clogin -c "sh ver" -f /root/.cloginrc 10.5.51.17
    10.5.51.17
    spawn telnet 10.5.51.17
    Trying 10.5.51.17...
    Connected to 10.5.51.17.
    Escape character is '^]'.
    User Access Verification
    Username: root
    Password:
    VPN_Balkany#
    VPN_Balkany#terminal length 0
    VPN_Balkany#sh ver
    Cisco IOS Software, C850 Software (C850-ADVSECURITYK9-M), Version 12.4(4)T4, RELEASE SOFTWARE (fc2)
    Copyright (c) 1986-2006 by Cisco Systems, Inc.
    Compiled Thu 03-Aug-06 17:16 by kellythw
    ROM: System Bootstrap, Version 12.3(8r)YI3, RELEASE SOFTWARE
    VPN_Balkany uptime is 2 days, 23 hours, 12 minutes
    System returned to ROM by power-on
    System image file is "flash:c850-advsecurityk9-mz.124-4.T4.bin"
    This product contains cryptographic features and is subject to United
    States and local country laws governing import, export, transfer and
    use. Delivery of Cisco cryptographic products does not imply
    third-party authority to import, export, distribute or use encryption.
    Importers, exporters, distributors and users are responsible for
    compliance with U.S. and local country laws. By using this product you
    agree to comply with applicable laws and regulations. If you are unable
    to comply with U.S. and local laws, return this product immediately.
    If you require further assistance please contact us by sending email to
    export@cisco.com.
    Cisco 851 (MPC8272) processor (revision 0x200) with 59392K/6144K bytes of memory.
    Processor board ID FCZ103913RL
    MPC8272 CPU Rev: Part Number 0xC, Mask Number 0x10
    5 FastEthernet interfaces
    128K bytes of non-volatile configuration memory.
    20480K bytes of processor board System flash (Intel Strataflash)
    Configuration register is 0x2102
    VPN_Balkany#exit
    Connection closed by foreign host.





    root@test:/var/lib/rancid/logs# ls -la
    total 52
    drwxr-xr-x 2 root root 4096 2010-06-09 13:51 .
    drwxr-xr-x 13 root root 4096 2010-06-09 10:57 ..
    -rw-r----- 1 root root 41860 2010-06-09 13:51 routers.20100609.135845




    starting: Срд Июн 9 13:58:45 MSD 2010

    /usr/lib/rancid/bin/control_rancid: 308: sendmail: not found
    cvs add: scheduling file `10.5.51.17' for addition
    cvs add: use `cvs commit' to add this file permanently
    /var/lib/rancid/CVS/routers/configs/10.5.51.17,v <-- 10.5.51.17
    initial revision: 1.1
    Added 10.5.51.17
    cvs add: scheduling file `10.5.51.25' for addition
    cvs add: use `cvs commit' to add this file permanently
    /var/lib/rancid/CVS/routers/configs/10.5.51.25,v <-- 10.5.51.25
    initial revision: 1.1
    Added 10.5.51.25
    cvs add: scheduling file `10.5.51.33' for addition
    cvs add: use `cvs commit' to add this file permanently
    /var/lib/rancid/CVS/routers/configs/10.5.51.33,v <-- 10.5.51.33
    initial revision: 1.1
    Added 10.5.51.33
    cvs add: scheduling file `10.5.52.41' for addition
    cvs add: use `cvs commit' to add this file permanently
    /var/lib/rancid/CVS/routers/configs/10.5.52.41,v <-- 10.5.52.41
    initial revision: 1.1
    Added 10.5.52.41
    cvs add: scheduling file `10.5.52.49' for addition
    cvs add: use `cvs commit' to add this file permanently
    /var/lib/rancid/CVS/routers/configs/10.5.52.49,v <-- 10.5.52.49
    initial revision: 1.1
    Added 10.5.52.49
    cvs add: scheduling file `10.5.54.105' for addition
    cvs add: use `cvs commit' to add this file permanently
    /var/lib/rancid/CVS/routers/configs/10.5.54.105,v <-- 10.5.54.105
    initial revision: 1.1
    Added 10.5.54.105
    cvs add: scheduling file `10.5.55.113' for addition
    cvs add: use `cvs commit' to add this file permanently
    /var/lib/rancid/CVS/routers/configs/10.5.55.113,v <-- 10.5.55.113
    initial revision: 1.1
    Added 10.5.55.113
    cvs add: scheduling file `10.5.55.129' for addition
    cvs add: use `cvs commit' to add this file permanently
    /var/lib/rancid/CVS/routers/configs/10.5.55.129,v <-- 10.5.55.129
    initial revision: 1.1
    Added 10.5.55.129

    Trying to get all of the configs.
    All routers sucessfully completed
    cvs diff: Diffing .
    cvs diff: Diffing configs
    cvs commit: Examining .
    cvs commit: Examining configs
    /var/lib/rancid/CVS/routers/router.db,v <-- router.db
    new revision: 1.2; previous revision: 1.1
    /var/lib/rancid/CVS/routers/configs/10.5.51.17,v <-- configs/10.5.51.17
    new revision: 1.2; previous revision: 1.1
    /var/lib/rancid/CVS/routers/configs/10.5.51.25,v <-- configs/10.5.51.25
    new revision: 1.2; previous revision: 1.1
    /var/lib/rancid/CVS/routers/configs/10.5.51.33,v <-- configs/10.5.51.33
    new revision: 1.2; previous revision: 1.1
    /var/lib/rancid/CVS/routers/configs/10.5.52.41,v <-- configs/10.5.52.41
    new revision: 1.2; previous revision: 1.1
    /var/lib/rancid/CVS/routers/configs/10.5.52.49,v <-- configs/10.5.52.49
    new revision: 1.2; previous revision: 1.1
    /var/lib/rancid/CVS/routers/configs/10.5.54.105,v <-- configs/10.5.54.105
    new revision: 1.2; previous revision: 1.1
    /var/lib/rancid/CVS/routers/configs/10.5.55.113,v <-- configs/10.5.55.113
    new revision: 1.2; previous revision: 1.1
    /var/lib/rancid/CVS/routers/configs/10.5.55.129,v <-- configs/10.5.55.129
    new revision: 1.2; previous revision: 1.1
    /usr/lib/rancid/bin/control_rancid: 487: sendmail: not found

    ending: Срд Июн 9 13:59:29 MSD 2010



    We made sure that the configurations are saved, now we will check whether the changes in the configurations of the routers are taken into account. We change something in the settings of the router with the address 10.5.51.17. Run rancid-run again and check the newly appeared log file. Since only the configuration of the router 10.5.51.17 changed, the configuration version increased only for him: We achieved our goals, saved the configuration files and made sure that the changes in the device configuration are displayed in CVS. At first glance, everything may seem complicated, confusing, and not logical, but in fact, everything is simple. Installation is carried out with one command, the adjustment is reduced to creating and editing several files.

    root@test:/# telnet 10.5.51.17
    Trying 10.5.51.17...
    Connected to 10.5.51.17.
    Escape character is '^]'.

    User Access Verification

    Username: root
    Password: .
    VPN_Balkany#conf t
    Enter configuration commands, one per line. End with CNTL/Z.
    VPN_Balkany(config)#int fa4
    VPN_Balkany(config-if)#des
    VPN_Balkany(config-if)#description WAN interface
    VPN_Balkany(config-if)#end
    VPN_Balkany#exit
    Connection closed by foreign host.




    root@test:/var/lib/rancid/logs# ls -la
    total 16
    drwxr-xr-x 2 root root 4096 2010-06-09 14:55 .
    drwxr-xr-x 13 root root 4096 2010-06-09 10:57.
    -rw-r----- 1 root root 3115 2010-06-09 13:59 routers.20100609.135845
    -rw-r----- 1 root root 437 2010-06-09 14:55 routers.20100609.145523




    starting: Срд Июн 9 14:55:23 MSD 2010

    Trying to get all of the configs.
    All routers sucessfully completed.

    cvs diff: Diffing .
    cvs diff: Diffing configs
    cvs commit: Examining .
    cvs commit: Examining configs
    /var/lib/rancid/CVS/routers/configs/10.5.51.17,v <-- configs/10.5.51.17
    new revision: 1.3; previous revision: 1.2
    /usr/lib/rancid/bin/control_rancid: 487: sendmail: not found

    ending: Срд Июн 9 14:55:58 MSD 2010



    Ahead is setting up the web interface for CVS - cvsweb, email notifications when changing the device configuration and running the script on schedule.

    Also popular now: