Linux Quest. Congratulations to the winners and talk about solving problems



    March 25, we opened registration for Linux Quest , this is a game for fans and experts on the Linux operating system. Some statistics: 1117 people registered for the game, 317 of them found at least one key, 241 successfully completed the task of the first stage, 123 of the second and 70 completed the third stage. Today our game has come to an end, and we congratulate our winners!

    • The first place was taken by Alexander Teldekov.
      Alexander said to himself that he is the most typical sysadmin. Lives in Volgograd, administers various Unix-like systems for twenty years. Managed to work in Internet providers, bank, system integrator. Now he works remotely in a small company, he is engaged in cloud infrastructure for a large foreign customer. Likes to read, listen to music. About the game, Alexander said that he liked the game as a whole, he loves such puzzles. In one of the companies at the interview I was engaged in a similar to Hackerrank, it was interesting.
    • Second place - Roman Suslov.
      A novel from Moscow. He's 37 years old. He works as a Linux / Unix engineer at Jet Infosystems. For work, I have to administer and troubleshoot Linux / Unix systems + SAN. Interests are very different: Linux-systems, programming, reverse engineering, information security, Arduino. About the Game, Roman noted that he generally liked the game. “I flexed my gyrus a little and was distracted from the gray everyday life of everyday work. :) I would like more tasks, otherwise I didn’t have time to get a taste of how the game was already over. ”
    • The third is alex3d.
      Alex lives in Moscow, is engaged in software development. "Thanks for the contest, it was interesting to check your google-fu skill."

    Also in the ranking of the top 10 players:

    • Yevgeniy saldayev
    • Markel Mokhnachevsky
    • Konstantin Konosov
    • Pavel Sergeev
    • Vladimir Bovaev
    • Ivan Bubnov
    • Pavlo klets

    We understand that there are many options for solving all our problems, one of the possible solutions is described below.

    1. The first stage


    We called it “Are you sure the admin?”, Because the task was quite simple - to fix a warm lamp service.

    1.1. Interesting Facts:


    Two players found the first key in the first 15 minutes of the game, and in the first hour we had three leaders who coped with the task.

    1.2. The task


    You went to work for a company where for a long time there was no competent IT specialist. Before you begin to clean up, you need to solve a burning problem that blocks the work of the office.

    A cleaning lady mopped the power cable to the server cabinet. Power restored, but a very important website still does not work. The website is important because the company is not very concerned about information security, and on the main page of this you can find the administrator password from the CEO’s computer in clear text.

    The other day, the password was changed, but everyone forgot the new one, the director cannot work. Rumor has it that there were keys on this machine that could help us decrypt the backup of accounting documents.

    Everyone is waiting for a quick solution to the issue!

    1.3. Decision


    1. First of all, you need to change the root password in the virtual machine in order to gain access to it. At startup, we notice that this is Ubuntu 16.04 Server.

    To reset the root password, we restart the machine, at startup, when the grub menu is displayed, go to the editing of the Ubuntu item with the “e” button. We edit the linux line, add it to the end init=/bin/bash. We load through Ctrl + x, we get bash. Remount the root with rw, change the password:

    $ mount -o remount,rw /dev/mapper/ubuntu--vg-root
    $ passwd

    Do not forget about sync, reboot.

    2. The condition says that the web server does not work for us, we look:

    $ curl localhost
    Not Found
    The requested URL / was not found on this server.
    Apache/2.4.18 

    That is, in fact, Apache is running, but responds with code 404. We look at the config:

    $ vim /etc/apache2/sites-enabled/000-default.conf

    There is also a key - StevenPaulSteveJobs.

    Check the path /usr/share/WordPress- this is not, but there is /usr/share/wordpress. We correct the config and restart the Apache.

    $ systemctl restart apache2

    3. We try again, we get an error:

    Warning: mysqli_real_connect(): (HY000/2002): Connection refused in /usr/share/wordpress/wp-includes/wp-db.php on line 1488

    The database is not running?

    $ systemctl status mysql
    Active: active (running)

    What is the matter? It’s necessary to understand. To do this, you need to access MySQL, how to do this is described in the documentation . One of the points in the documentation recommends us to register option skip-grant-tablesc /etc/mysql/mysql.conf.d/mysqld.cnf. Here is also the key - AugustaAdaKingByron.

    We correct the rights to the user 'wp'@'localhost'. We start MySQL, we make it accessible on a network, commenting an option in a config skip-networking.

    4. After the actions taken, the web server starts, but the site still does not work, as

    Warning: require_once(/usr/share/wordpress/wp-content/themes/twentysixteen/footer.php): failed to open stream: Permission denied in /usr/share/wordpress/wp-includes/template.php on line 562

    We correct the rights to the file.

    $ chmod 644 /usr/share/wordpress/wp-content/themes/twentysixteen/footer.php
    

    We refresh the page, go to the site and find the key - BjarneStroustrup! We found all three keys, our director can work, we decrypted the accounting files. Everyone is happy, and you have a lot of work ahead to establish infrastructure, backups and security in the company.

    2. The second stage


    It was necessary to solve the problem of collecting analytics. Everyone loves analytics - who and where and in what quantities go. We came up with a case that all engineers in life may encounter in one form or another.

    2.1. Interesting Facts


    One of our players entered the correct key in the first 10 minutes of the game, and in the first hour we got a leader who coped with the task.

    2.2. The task


    You went to work for the company, managers came to you and asked to find to whom letters were sent from Africa. You need to build the top 21 recipient addresses on them. The first letters of the recipient addresses are the key. One thing: the mail server through which letters were sent does not load. Everyone is waiting for a quick solution to the issue!

    2.3. Decision


    1. The server is not loaded due to the non-existent swap partition in fstab, at boot the system tries to mount it and crashes. How to boot?

    Download the image, we downloaded CentOS 7, boot from the Live CD / DVD (Troubleshooting -> Rescue), mount the system, and edit it /etc/fstab. Here we find the first key - GottfriedWilhelm11646Leibniz!

    Create a swap:

    $ lvcreate -n swap centos -L 256M
    $ sync && reboot

    2. As always, there is no password; you need to change the root password on the virtual machine. We already did this in the first task. We change and successfully go to the server, but it immediately goes into reboot. The server is overloaded at such a speed that you do not even have time to look at all the logs carefully. How to understand what is happening?

    Again we boot from livecd, carefully study the system logs and just in case we look in crowns, since such a frequency. There we find the problem and the second key is Alan1912MathisonTuring!

    You need to /etc/crontabdelete or comment out the line echo b > /proc/sysrq-trigger.

    3. After that, the server booted up, and you can complete the task of managers: "What are the addresses of Africa?" This information is generally publicly available. You can find this information on the Internet using the phrases “ip address africa”, “geoip database”. To solve the problem, you can use freely available address distribution bases (geoip). We used the MaxMind GeoLite2 database , available under the Creative Commons Attribution-ShareAlike 4.0 license, as a reference .

    Let's try to solve our problem using only Linux system utilities, but in general it can be solved in a huge number of ways: using text filtering utilities and using scripts in various programming languages.

    To get started, just get the “IP sender - recipient” pairs from the mail log/var/log/maillog(build a table of email recipients - IP sender). This can be done with the following command:

    $ cat /var/log/maillog | fgrep -e ' connect from' -e 'status=sent' | sed 's/[]<>[]/ /g' | awk '/connect from/ {ip=$11} /status=sent/ {print $10" "ip}' > log1.txt

    And before we continue with the compilation of the database of African addresses, we look at the top IP addresses of senders.

    $ cat log1.txt | cut -d' ' -f1 | sort | uniq -c | sort -r | head -n 40
    5206 L2JhbjAbM67GA99jg@mail.ru
    4165 iHKTBkegOQa6fIALq@mail.ru
    3739 nHkcBl7BdgXxijSYD7@mail.ru
    3405 SMAzPJAzbl9vp4hAXo@mail.ru
    3346 xILz6d7P@mail.ru
    

    Among the all, the first three recipients from the top are clearly distinguished by the number of letters. If you grep the IP addresses of senders that were sent to addresses from this top-3, you can see the clear predominance of certain networks:

    $ cat log1.txt | fgrep 'L2JhbjAbM67GA99jg@mail.ru' | cut -d' ' -f2 | sort | cut -d'.' -f1 | uniq -c | sort -r | head
    831 105
    806 41
    782 197
    664 196
    542 154
    503 102
    266 156
    165 45
    150 160
    108 165

    Most of the networks 105/8, 41/8, 196 / 8,197 / 8 are allocated by AFRINIC - one of the five regional Internet registrars that distribute Internet resources. AFRINIC distributes address space in Africa. And 41/8 refers to AFRINIC completely.

    https://www.nic.ru/whois/?searchWord=105.0.0.0 
    https://www.nic.ru/whois/?searchWord=41.0.0.0
    

    Thus, the answer to the problem, in fact, is in the log itself.

    $ cat log1.txt | fgrep -e '105.' -e '41.' -e '196.' -e '197.' -e '154.' -e '102.' | awk '{print $1}' | sort | uniq -c | sort -r | head -n 21
    4209 L2JhbjAbM67GA99jg@mail.ru
    3313 iHKTBkegOQa6fIALq@mail.ru
    2704 nHkcBl7BdgXxijSYD7@mail.ru
    2215 uvRbp1O@mail.ru
    1774 sPmMsmmFiV@mail.ru
    1448 BtG3aHgQgCKuze2AKuRH@mail.ru
    1233 eQpuuQ2uQdbwRL3@mail.ru
    958 nJT5dpaBZ@mail.ru
    862 ef4WbQiB@mail.ru
    762 dQCqKL6eVminFfH7wLA@mail.ru
    632 ifq6Rd1HxuCQOdO9@mail.ru
    539 cFwm2ssypMmx1sA7@mail.ru
    531 twtTnr4G@mail.ru
    431 TSrczgYASrR11Hs3qCi@mail.ru
    380 o3r3exc3OL@mail.ru
    357 rzmjr2VAHK@mail.ru
    348 vnPr6YjJ3ndw@mail.ru
    312 anOjFXrwOtLP2Rl1Vcz6@mail.ru
    289 dvny5zHmRW8fiT@mail.ru
    282 sgg9jPxFDYvzw8Kr@mail.ru
    274 tKSevzA7GntJ@mail.ru

    At this point, we get the string “LinuxBenedictTorvadst”.

    The correct key is LinusBenedictTorvalds.

    The resulting string contains a typo in relation to the correct key in 3 extreme characters. This is due to the fact that the networks we have chosen are not entirely allocated to African countries and how the emails are distributed by IP addresses in our log.

    With sufficient clarification of the largest networks allocated to African countries, you can get the exact answer:

    $ cat log1.txt | fgrep -e' '105.{30..255}. -e' '41. -e' '196.{64..47}. -e' '196.{248..132}. -e' '197.{160..31}. -e' '154.{127..255}. -e' '102.{70..255}. -e' '156.{155..255}. | awk '{print $1}' | sort | uniq -c | sort -r | head -n 21
    3350 L2JhbjAbM67GA99jg@mail.ru
    2662 iHKTBkegOQa6fIALq@mail.ru
    2105 nHkcBl7BdgXxijSYD7@mail.ru
    1724 uvRbp1O@mail.ru
    1376 sPmMsmmFiV@mail.ru
    1092 BtG3aHgQgCKuze2AKuRH@mail.ru
    849 eQpuuQ2uQdbwRL3@mail.ru
    712 nJT5dpaBZ@mail.ru
    584 ef4WbQiB@mail.ru
    463 dQCqKL6eVminFfH7wLA@mail.ru
    365 ifq6Rd1HxuCQOdO9@mail.ru
    269 cFwm2ssypMmx1sA7@mail.ru
    225 twtTnr4G@mail.ru
    168 TSrczgYASrR11Hs3qCi@mail.ru
    142 o3r3exc3OL@mail.ru
    111 rzmjr2VAHK@mail.ru
     96 vnPr6YjJ3ndw@mail.ru
     78 anOjFXrwOtLP2Rl1Vcz6@mail.ru
     56 lHzWiB7ExvRtSbAcU9@mail.ru
     56 dvny5zHmRW8fiT@mail.ru
     40 sgg9jPxFDYvzw8Kr@mail.ru

    The problem can also be solved in another way.
    Download MaxMind, unpack, and the following three teams also solve our problem.

    $ cat GeoLite2-Country-Locations-ru.csv | grep "Африка" | cut -d',' -f1 > africaIds.txt
    $ grep -Ff africaIds.txt GeoLite2-Country-Blocks-IPv4.csv | cut -d',' -f1 > africaNetworks.txt
    $ grepcidr -f africaNetworks.txt log1.txt | cut -d' ' -f1 | sort | uniq -c | sort -r | head -n21
    

    In one way or another, we eventually counted the statistics, and the managers got the data they needed to work!

    3. The third stage


    The third stage is somewhat similar to the first - you also need to fix the warm lamp service, but it’s more difficult than in the first task.

    3.1. Interesting Facts


    In the first 15 minutes, three players found the first key, 2 hours 20 minutes after the start of the stage, our winner completed the task.

    3.2. The task


    You went to work in a company where all company documents are stored on the internal Wiki server. Last year, an engineer ordered 3 new disks for the server in addition to one existing one, arguing that for system fault tolerance it is necessary to put disks in some arrays. Unfortunately, a few weeks after their installation, the engineer went to rest in India and did not return.

    For several years, the server worked without failures, but a couple of days ago the company's network was hacked. According to the instructions, the security officers removed the disks from the server and sent them to you. During transportation, one disk was irretrievably lost.

    You need to restore the wiki, first of all, the content of the wiki pages is of interest. A piece of text that was on one of the pages of this wiki is a password from a 1C server and is urgently needed to unlock it.

    In addition, somewhere on the wiki pages or elsewhere there were passwords from the log server and the video surveillance server, which are also desirable to recover, without them it is impossible to investigate the incident. As always, they are expected from an operational solution to the issue!

    3.3. Decision


    1. We try to boot in turn from the drives that we have and everywhere we get the same message:

    No bootable medium found! System halted 
    

    It is necessary to boot from something. Again rescues the boot from a Live CD / DVD (Troubleshooting -> Rescue). When booting, we try to find the boot partition, we don’t find it, we get into the shell. We are trying to learn what and how with disks. It is known that there are three of them. There are more tools for this in the 7th version of CentOS, where there are commands blkidor lsblkthat show us all the information about the disks.

    How and what to do:

    $ ls /dev/sd*

    It is immediately clear that

    /dev/sdb1 - ext4
    /dev/sdb2 - часть lvm
    /dev/sda1 и /dev/sdc1 - части рейда
    /dev/sda2 и /dev/sdc2 - про них ничего не известно на текущий момент

    Mount sdb1, it is clear that this is the boot partition of CentOS 6.

    $ mkdir /mnt/sdb1 && mount /dev/sdb1 /mnt/sdb1

    Obviously, we go to the grub section and find the first key there - James191955Gosling in an unusual file.

    2. We study pvs and lvs, as we work with LVM. We see that there should be 2 physical volumes, one is not located and swears at the lost uid. We see that there should be 2 logically volume: root and swap, while root is partially lost (the P attribute of volume). Mount does not work, but sorry! We really need him.

    There are 2 more disks, look at them, assemble and mount:

    $ mdadm --examine --verbose --scan
    $ mdadm --assemble --verbose --scan
    $ mkdir /mnt/md127 && mount /dev/md127  /mnt/md127 

    We look, it is clear that this is the boot partition of CentOS 6 and a take of what is already on /dev/sdb1, and here again the same key is James191955Gosling!

    We look how it is assembled /dev/md127.

    $ mdadm --detail /dev/md127
    

    We see that it was supposed to be assembled from 4 disks, assembled from two /dev/sda1and /dev/sdc1, they should have been numbers 2 and 4 in the system. We assume that from /dev/sda2and /dev/sdc2you can also collect an array. It is unclear why there is no metadata on them, but this is on the conscience of the administrator, who is somewhere in Goa. We assume that there should be RAID10, although there are options. We collect:

    $ mdadm --create --verbose /dev/md0 --assume-clean --level=10 --raid-devices=4 missing /dev/sda2 missing /dev/sdc2
    

    We look at blkid, pvs, lvs. We find that we collected a physical volume, which we previously lacked.

    We immediately fixed lvroot, mount it, but first activate VG:

    $ vgchange -a y
    $ mkdir /mnt/lvroot && mount /dev/mapper/vg_c6m1-lv_root /mnt/lvroot 
    

    And there everything is there, including the DennisBMacAlistairCRitchie key - in the root home directory - /root/sweet.

    3. Nevertheless, we are trying to revive our server so that it starts normally. All logical volume c ours /dev/md0(where we have everything) drag and drop to /dev/sdb2where the whole server worked originally.

    $ pvmove /dev/md0 /dev/sdb2
    $ vgreduce vg_c6m1 /dev/md0

    We extinguish the server, remove 1 and 3 disks, leave the second one, boot from the Live CD / DVD in Rescue. We find the boot section, restore the bootloader in grub:

    root (hd0,0)
    setup (hd0)
    

    We tear off the boot disk and boot successfully, but the site does not work.

    4. There are two options to start the site: configure Apache from scratch or use nginx already configured in advance with php-fpm:

    $ /etc/init.d/nginx start
    $ /etc/init.d/php-fpm start
    

    Finally, you need to start MySQL:

    $ /etc/init.d/mysqld start

    It does not start, and the answer lies in /var/log/mysql. As soon as you solve the problem with MySQL, the site will work, there will be a key on the main page - RichardGCCMatthewGNUStallman! Now we have access to 1C, and employees will be able to get paid. And as always, you have a lot of work ahead to build infrastructure and security in the company.

    We can also once again share a list of books that helped us and our participants prepare for the game: linux.mail.ru/books .

    Thank you for being with us! Follow the announcements of the next games!

    Also popular now: