Linux log files in order

It is impossible to imagine the user and administrator of a server, or even a Linux-based workstation that has never read the log files. The operating system and running applications constantly create various types of messages that are logged in various log files. The ability to determine the desired log file and what to look for in it can significantly save time and eliminate the error faster.

Logging is the main source of information about the operation of the system and its errors. In this short guide, we will cover the main aspects of operating system logging, directory structure, and programs for reading and browsing logs.

Main log files

All log files can be assigned to one of the following categories:

• applications
• events;
• services;
• systemic.

Most log files are contained in a directory /var/log.

• / var / log / syslog or / var / log / messages contains a global system log in which messages are written from the moment the system starts, from the Linux kernel, various services, discovered devices, network interfaces, and much more.
• /var/log/auth.log or / var / log / secure - information about user authorization, including successful and unsuccessful login attempts, as well as the authentication mechanisms involved.
• / var / log / dmesg - device drivers. With the same command you can view the output of the contents of the file. Log size is limited, when the file reaches its limit, old messages will be overwritten with newer ones. By setting the key, --level=you can filter the output by the significance criterion.

Поддерживаемые уровни журналирования (приоритеты):
   emerg - система неиспользуемая
   alert - действие должно быть произведено немедленно
    crit - условия критичности
     err - условия ошибок
    warn - условия предупреждений
  notice - обычные, но значимые условия
    info - информационный
   debug - отладочные сообщения
(5:520)$ dmesg -l err
[1131424.604352] usb 1-1.1: 2:1: cannot get freq at ep 0x1
[1131424.666013] usb 1-1.1: 1:1: cannot get freq at ep 0x81
[1131424.749378] usb 1-1.1: 1:1: cannot get freq at ep 0x81

• /var/log/alternatives.log - The output of the program update-alternativesthat contains symbolic links to commands or libraries by default.
• /var/log/anaconda.log - Entries recorded during system installation.
• / var / log / audit - Records created by the audit service auditd.
• /var/log/boot.log - Information that is written when the operating system boots.
• / var / log / cron - Service report crondon executable commands and messages from the commands themselves.
• / var / log / cups - Everything related to printing and printers.
• / var / log / faillog - Unsuccessful login attempts. It is very useful when checking threats in the security system, hacker attacks, hacking attempts by brute force. You can read the contents using the command faillog.
• var / log / kern.log - The log contains messages from the kernel and warnings that can be useful in resolving errors of user modules built into the kernel.
• / var / log / maillog / or /var/log/mail.log - The log of the mail server used on the OS.
• /var/log/pm-powersave.log - Messages about the battery saving service.
• / var / log / samba / - The logs of the file server Samba, which is used to access Windows shared folders and provide access for Windows users to Linux shared folders.
• / var / log / spooler - For representatives of the old school, contains USENET messages. Most often it is empty and abandoned.
• /var/log/Xorg.0.log - X server logs. Most often useless, but if they have lines starting with EE, then you should pay attention to them.

Each distribution will have a separate package manager log.

• /var/log/yum.log - For programs installed using YumRedHat Linux.
• /var/log/emerge.log - For ebuilds installed from Portageusing emergein Gentoo Linux.
• /var/log/dpkg.log - For programs installed using dpkgDebian Linux and the entire family of related distributions.

And a few binary logs accounting user sessions.

• / var / log / lastlog - The last user session. You can read the command last.
• / var / log / tallylog - Audit failed login attempts. Output to the screen using the utility pam_tally2.
• / var / log / btmp - One more log of failed login attempts. Just so, just in case, if you have not figured out where to look for traces of hacker activity.
• / var / log / utmp - The list of user logins currently.
• / var / log / wtmp - Another log of user login entries. Display on command utmpdump.

(5:535)$ sudo utmpdump /var/log/wtmp
[5] [02187] [l0  ] [        ] [4.0.5-gentoo     ] [0.0.0.0     ] [Вт авг 11 16:50:07 2015]
[1] [00000] [~~  ] [shutdown] [4.0.5-gentoo     ] [0.0.0.0     ] [Вт авг 11 16:50:08 2015]
[2] [00000] [~~  ] [reboot  ] [3.18.12-gentoo   ] [0.0.0.0     ] [Вт авг 11 16:50:57 2015]
[8] [00368] [rc  ] [        ] [3.18.12-gentoo   ] [0.0.0.0     ] [Вт авг 11 16:50:57 2015]
[1] [20019] [~~  ] [runlevel] [3.18.12-gentoo   ] [0.0.0.0     ] [Вт авг 11 16:50:57 2015]

And other magazines

Since the operating system, even such a wonderful one as Linux, does not carry any tangible benefits in itself, most likely a database, a web server, and various applications will be spinning on a server or workstation. Each application or service can have its own file or directory of event and error logs. It is naturally impossible to list all of them, only a few.

• / var / log / mysql / - MySQL database log.
• / var / log / httpd / or / var / log / apache2 / - Apache web server log, access log is in access_log, and errors are in error_log.
• / var / log / lighthttpd / - The web server log lighttpd.

The user's home directory may contain graphical application logs, DE.

• ~ / .xsession-errors - Output to stderrX11 graphical applications.

Initializing  "kcm_input" :  "kcminit_mouse"
Initializing  "kcm_access" :  "kcminit_access"
Initializing  "kcm_kgamma" :  "kcminit_kgamma"
QXcbConnection: XCB error: 3 (BadWindow), sequence: 181, resource id: 10486050, major code: 20 (GetProperty), minor code: 0
kf5.kcoreaddons.kaboutdata: Could not initialize the equivalent properties of Q*Application: no instance (yet) existing.
QXcbConnection: XCB error: 3 (BadWindow), sequence: 181, resource id: 10486050, major code: 20 (GetProperty), minor code: 0
Qt: Session management error: networkIdsList argument is NULL

• ~ / .xfce4-session.verbose-log - XFCE4 desktop messages.

How to browse - lnav

Almost everyone knows about the utility lessand the team tail -f. Also, the editor vimand file manager Midnight Commander will come in handy for these purposes . Everyone has their own drawbacks: it lessdoesn’t matter if it processes logs with long lines, taking them for binaries. Midnight Commander is only suitable for fluent viewing, when there is no need to search for a complex template and go back and forth between matches a lot. The editor vimunderstands and highlights the syntax of many formats, but if the log is updated frequently, distracting messages about changes in the file appear. However, this can easily be circumvented with <:view /path/to/file>.

Recently, I discovered another useful and promising, but still slightly damp, utility - lnav , in the decryption Log File Navigator.

Install the package as usual with one command.

$ aptitude install lnav #Debian/Ubuntu/LinuxMint
$ yum install lnav #RedHat/CentOS
$ dnf install lnav #Fedora
$ emerge -av lnav #Gentoo, нужно добавить в файл package.accept_keywords
$ yaourt -S lnav #Arch

Lnav log navigator understands a number of file formats.

• Access_log web server.
• CUPS page_log
• Syslog
• glog
• dpkg.log
• strace
• Arbitrary time stamped recordings
• gzip, bzip
• VMWare ESXi / vCenter Magazine

What in this case means understanding file formats? The trick is that lnav is more than a utility for viewing text files. The program can do something else. You can open several files at once and switch between them.

(5:471)$ sudo lnav /var/log/pm-powersave.log /var/log/pm-suspend.log

The program can directly open the archive file.

(5:471)$ lnav -r /var/log/Xorg.0.log.old.gz

Shows a histogram of informative messages, warnings and errors if you press a key . This is from my syslog.

Mon May 02 20:25:00        123 normal         3 errors         0 warnings         0 marks
Mon May 02 22:40:00          2 normal         0 errors         0 warnings         0 marks
Mon May 02 23:25:00         10 normal         0 errors         0 warnings         0 marks
Tue May 03 07:25:00         96 normal         3 errors         0 warnings         0 marks
Tue May 03 23:50:00         10 normal         0 errors         0 warnings         0 marks
Wed May 04 07:40:00         96 normal         3 errors         0 warnings         0 marks
Wed May 04 08:30:00          2 normal         0 errors         0 warnings         0 marks
Wed May 04 10:40:00         10 normal         0 errors         0 warnings         0 marks
Wed May 04 11:50:00        126 normal         2 errors         1 warnings         0 marks