August 19, 2014 at 12:09Analysis of network traffic on the server using tshark
- Tutorial
In the practice of system administration, it is often necessary to encounter difficult situations in which neither statistics collection tools (for example, netstat) nor standard utilities based on the ICMP protocol (ping, traceroute and others) help. In such cases, specialized diagnostic utilities are often used, which make it possible to “listen” to network traffic and analyze it at the level of transmission units of individual protocols. They are called traffic analyzers, and in professional jargon, they are called sniffers. With their help, you can, firstly, localize network problems and diagnose them more accurately, and secondly, detect spurious traffic and detect malicious software on the network.
Traffic analyzers are especially useful in cases where network software is poorly documented or uses its own proprietary protocols.
One of the most common and popular traffic analyzers today is Wireshark, distributed under the GNU GPL. There are versions of Wireshark for various operating systems: Linux, Windows, MacOS, FreeBSD, Solaris.
Tshark uses the libpcap library for capturing, which implements the pcap (packet capture) API. This library is also used by the standard tcpdump utility . Files created in tcpdump can be passed to tshark for later analysis.
The obvious advantage of tshark over tcpdump is its clearer and more human-readable output format. In addition, tshark supports a huge number of network protocols (more than 300, which covers almost all types of networks ever invented).
We will tell you about the features of working with tshark in detail in this article.
Beginning of work
Tshark is included in the distributions of most modern Linux systems and is installed using the standard package manager:
$ sudo apt-get install tshark
After completing the installation, run the program:
$ sudo tshark
A list of packets captured in real time will be displayed on the console:
Capturing on eth0 0.000000 88: e0: f3: b6: 47: c0 -> Broadcast ARP Who has 31.186.98.120? Tell 31.186.98.1 0.322046 5a: 58: 74: bf: a9: 9c -> Broadcast ARP Who has 31.186.98.77? Tell 31.186.98.226 0.351801 31.186.98.235 -> 188.93.16.50 SSH Encrypted response packet len = 224 0.352414 188.93.16.50 -> 31.186.98.235 TCP cap> ssh [ACK] Seq = 1 Ack = 225 Win = 331 Len = 0 TSV = 194287231 TSER = 416767897 0.600054 88: e0: f3: b6: 47: c0 -> Broadcast ARP Who has 31.186.98.120? Tell 31.186.98.1 0.622913 Cisco_0d: 0d: 96 -> PVST + STP Conf. Root = 32768/398/00: 21: 1c: 0d: 0d: 80 Cost = 0 Port = 0 × 8016 0.800377 88: e0: f3: b6: 47: c0 -> Broadcast ARP Who has 31.186.98.107? Tell 31.186.98.1 1.320775 31.186.98.235 -> 188.93.16.50 SSH Encrypted response packet len = 528 1.321507 188.93.16.50 -> 31.186.98.235 TCP cap> ssh [ACK] Seq = 1 Ack = 753 Win = 331 Len = 0 TSV = 194287474 TSER = 416768866 1.322109 5a: 58: 74: bf: a9: 9c -> Broadcast ARP Who has 31.186.98.77? Tell 31.186.98.226 1.400654 88: e0: f3: b6: 47: c0 -> Broadcast ARP Who has 31.186.98.107? Tell 31.186.98.1 1.589797 Cisco_0d: 0d: 96 -> PVST + STP Conf. Root = 32768/401/00: 21: 1c: 0d: 0d: 80 Cost = 0 Port = 0 × 8016 2.100769 88: e0: f3: b6: 47: c0 -> Broadcast ARP Who has 31.186.98.107? Tell 31.186.98.1 2.322163 5a: 58: 74: bf: a9: 9c -> Broadcast ARP Who has 31.186.98.77? Tell 31.186.98.226 2.322764 31.186.98.235 -> 188.93.16.50 SSH Encrypted response packet len = 720 2.323594 188.93.16.50 -> 31.186.98.235 TCP cap> ssh [ACK] Seq = 1 Ack = 1473 Win = 331 Len = 0 TSV = 194287724 TSER = 416769868 2.520048 88: e0: f3: b6: 47: c0 -> Broadcast ARP Who has 31.186.98.64? Tell 31.186.98.1 2.635370 Cisco_0d: 0d: 96 -> PVST + STP Conf. Root = 32768/398/00: 21: 1c: 0d: 0d: 80 Cost = 0 Port = 0 × 8016 2003.2002 88: e0: f3: b6: 47: c0 -> Broadcast ARP Who has 31.186.98.64? Tell 31.186.98.1 3.451774 31.186.98.235 -> 188.93.16.50 SSH Encrypted response packet len = 528
As you can see from the above conclusion, tshark shows information about all packages in a row, including those that we currently do not need at all. Using special options, you can make sure that only the information that we really need is displayed on the console.
Filters
Choosing an interface to capture
The -i option captures traffic for a specific interface only. You can view the list of available network interfaces using the command:
$ sudo tshark -D 1. eth0 2. any 3. lo (Loopback) 4.nflog 5. nfqueue 6. usbmon1
After the -i option, any of the available interfaces is indicated:
$ sudo tshark -i eth0
Using additional arguments, you can get more specialized information. For example, using the host argument, you can capture packets for a specific IP address:
$ sudo tshark -i eth0 host 192.168.1.100
The output will include both incoming and outgoing packets. To view information only about incoming packets or only about outgoing packets, the dst and src arguments are used, respectively:
$ sudo tshark -i eth0 src host 192.168.1.100
The command will print to the console a list of packets coming from the address 192.168.1.100
$ sudo tshark -i eth0 dst host 192.168.1.100
A list of packets arriving at 192.168.1.100 will be displayed on the console.
Similarly, you can capture traffic for an entire subnet - the net argument is used for this:
$ sudo tshark -i eth0 src net 192.168.1.0/24
You can also specify the port on which packets will be captured:
$ sudo tshark -i eth0 host 192.168.1.1 and port 80
Tshark allows you to capture packets over a period of time:
$ sudo tshark -i eth0 -a duration: 10 -w traffic.pcap
The -w option was also used in the above example. After it, the path to the file in which the received data will be written is indicated.
Capture filters
These filters are used when capturing traffic on the fly. They are recompiled into a set of rules for pcap, according to which packet filtering is performed. Only information that matches the criteria set by filters is displayed on the console.
In general, the syntax of the tshark command with filters can be represented as follows:
$ sudo tshark -i <interface> -f <filter>
Tshark uses the same syntax for capture filters as tcpdump does. In the framework of this article, we will not consider in detail all existing filters and restrict ourselves to referring to the official manual ).
Reading filters
Tshark can save information about captured packets in files. To extract the necessary information from these files, read filters are used, also called rules (option -R). They can also be used to capture packets on the fly. Information processing is carried out not by pcap, but by tshark itself.
These filters provide much wider possibilities for the selection and specification of information.
However, it should be borne in mind that when analyzing a large amount of information on the fly, they may not be able to cope with the tasks assigned to them: they do not have time to filter and discard packets.
In general terms, the syntax of the rules can be represented as follows:
$ sudo tshark -R "rule" -r "file path"
Consider the features of the formation of rules on specific examples.
So team
$ sudo tshark -R "ip.addr == 192.168.0.1" -r /tmp/capture.cap
indicates that information on outgoing and incoming packets for the IP address 192.168.0.1 should be extracted from the /tmp/capture.cap file.
The following rule will indicate that information about incoming and outgoing packets for all IP addresses except 192.168.0.1 should be extracted from this file:
$ sudo tshark -R "! (ip.addr == 192.168.0.1)" -r /tmp/capture.cap
Similarly, you can set rules for other protocols and ports (filters eth.addr, udp.port, tcp.port):
$ sudo tshark -R "eth.addr == 00: 08: 15: 00: 08: 15" -r /tmp/capture.cap $ sudo tshark -R "udp.port == 80" -r /tmp/capture.cap $ sudo tshark -R "tcp.port == 80" -r /tmp/capture.cap
Rules can also be combined using the logical operators and, or and not:
$ sudo tshark -R "not arp and not (udp.port == 53)" -r /tmp/capture.cap
The above command indicates that you need to retrieve a list of all captured packets except for ARP and DNS (port 53).
Additional settings
Statistics collection
Using the -z option, you can collect and display various static information about packages on the console.
For example, the command
$ sudo tshark -z "proto, colinfo, tcp.srcport, tcp.srcport" -r /tmp/capture.cap
indicates that information about the source port of all packages should be extracted from the /tmp/capture.cap file.
Consider another example:
$ sudo tshark -R "http.response and http.content_type contains image" \ -z "proto, colinfo, http.content_length, http.content_length" \ -z "proto, colinfo, http.content_type, http.content_type" \ -r /tmp/capture.cap
This command will extract information about all packages that contained images from the /tmp/capture.cap file and display the contents of the content_type and content_length fields on the console:
439 12.717117 66.249.89.127 -> 192.168.1.108 HTTP HTTP / 1.1 200 OK (GIF89a) http.content_type == "image / gif" http.content_length == 35 452 12.828186 66.114.48.56 -> 192.168.1.108 HTTP HTTP / 1.1 200 OK (GIF89a) http.content_type == "image / gif" http.content_length == 477 479 13.046184 66.114.48.56 -> 192.168.1.108 HTTP HTTP / 1.1 200 OK (GIF89a) http.content_type == "image / gif" http.content_length == 105 499 13.075361 203.190.124.6 -> 192.168.1.108 HTTP HTTP / 1.1 200 OK (GIF89a) http.content_type == "image / gif" http.content_length == 35 506 13.177414 66.114.48.56 -> 192.168.1.108 HTTP HTTP / 1.1 200 OK (GIF89a) http.content_type == "image / gif" http.content_length == 4039 514 13.190000 66.114.48.56 -> 192.168.1.108 HTTP HTTP / 1.1 200 OK (JPEG JFIF image) http.content_type == "image / jpeg" http.content_length == 11997 519 13.231228 66.114.48.56 -> 192.168.1.108 HTTP HTTP / 1.1 200 OK (JPEG JFIF image) http.content_type == "image / jpeg" http.content_length == 1033 523 13.273888 72.233.69.4 -> 192.168.1.108 HTTP HTTP / 1.1 200 OK (PNG) http.content_type == "image / png" http.content_length == 1974 561 728 19.096984 60.254.185.58 -> 192.168.1.108 HTTP HTTP / 1.1 200 OK (GIF89a) http.content_type == "image / gif" http.content_length == 592 805 19.471444 60.254.185.58 -> 192.168.1.108 HTTP HTTP / 1.1 200 OK (GIF89a) http.content_type == "image / gif" http.content_length == 259
Auto save to multiple files
Imagine that we need to keep traffic statistics for a long period of time. It is not very convenient to save all the output in one file in such a situation: it is then difficult to analyze.
The output can be saved in several files, the number and size of which are specified by the user. Once one file is full, tshark will continue writing to the next. For example, the command:
$ sudo tshark -b filesize: 100 -a files: 20 -w temp.pcap
will save the output in 20 files of 100 kB each.
In the above example, the -b option means that a ring buffer will be involved, and filesize sets the size; the -a option indicates an automatic stop upon reaching the specified limit, files - indicates the number of files.
Auto save after set time
In the example below, tshark will save the captured information in multiple files. A new file will be created automatically if the size exceeds 10240 kB or after an interval of 1 s:
$ sudo tshark -b filesize: 10240 -b duration: 1 -w temp.pcap Capturing on eth0 34 # ls -lrt -rw ------- 1 root root 1863 Apr 10 16:13 temp_00001_20140410161312.pcap -rw ------- 1 root root 1357 Apr 10 16:13 temp_00002_20140410161313.pcap -rw ------- 1 root root 1476 Apr 10 16:13 temp_00003_20140410161314.pcap -rw ------- 1 root root 1216 Apr 10 16:13 temp_00004_20140410161315.pcap
Setting buffer size
This option can be useful in cases where you have to deal with dropping packets. The default buffer size is 1MB; using the -B option, you can set any other size (in megabytes), after which all data will be flushed to disk:
$ sudo tshark -B 2
Display statistics for the selected protocol
Tshark also has the ability to capture only packets transmitted over a protocol specified by the user.
Here, for example, looks like statistics for the HTTP protocol:
$ sudo tshark -q -r a.pcap -R http -z http, tree =================================================== ================== HTTP / Packet Counter value rate percent -------------------------------------------------- ----------------- Total HTTP Packets 7 0.000375 HTTP Request Packets 4 0.000214 57.14% GET 4 0.000214 100.00% HTTP Response Packets 3 0.000161 42.86% 2xx: Success 2 0.000107 66.67% 200 OK 2 0.000107 100.00% 3xx: Redirection 1 0.000054 33.33% 302 Found 1 0.000054 100.00% 5xx: Server Error 0 0.000000 0.00% Other HTTP Packets 0 0.000000 0.00%
Practical examples
In this section, we will look at how tshark can be used to solve everyday administrative tasks.
Monitoring http requests
To list the http requests to the server, use the following command:
$ sudo tshark 'tcp port 80 and (((ip [2: 2] - ((ip [0] & 0xf) <> 2))! = 0)' -R 'http.request.method == "GET" | | http.request.method == "HEAD" '
The output of this command will look like this:
190.302141 192.168.0.199 -> 74.125.77.104 HTTP GET / HTTP / 1.1 190.331454 192.168.0.199 -> 74.125.77.104 HTTP GET /intl/en_com/images/srpr/logo1w.png HTTP / 1.1 190.353211 192.168.0.199 -> 74.125.77.104 HTTP GET /images/srpr/nav_logo13.png HTTP / 1.1 190.400350 192.168.0.199 -> 74.125.77.100 HTTP GET / generate_204 HTTP / 1.1
The following command will print to the console a list of 10 URLs from which http requests are received:
$ tshark -r sample1.cap -R http.request -T fields -e http.host -e http.request.uri | sed -e 's /?.*$//' | sed -e 's # ^ (. *) t (. *) $ # http: // 12 #' | sort | uniq -c | sort -rn | head
View a list of HTTP headers
To view the list of http-headers for server requests, use the command:
$ tshark -r sample1.cap -R http.request -T fields -e http.host -e http.request.uri | sed -e 's /?.*$//' | sed -e 's # ^ (. *) t (. *) $ # http: // 12 #' | sort | uniq -c | sort -rn | head
Accordingly, a list of headers for http responses can be obtained like this:
$ sudo tshark tcp port 80 or tcp port 443 -V -R "http.request"
To include both request and response headers in the list, the following command is used:
$ sudo tshark "tcp port 80 or tcp port 443" -V -R "http.request || http.response"
View a list of files of a particular type
Using tshark, you can view lists of files of a certain type transmitted via the http protocol. Here, for example, you can view a list of images in GIF format:
$ sudo tshark -R "http.response and http.content_type contains image" \ -z "proto, colinfo, http.content_length, http.content_length" \ -z "proto, colinfo, http.content_type, http.content_type" \ -r /tmp/capture.tmp | grep "image / gif" | wc -l
MySQL query monitoring
You can track queries in the MySQL database in real time using the following command:
$ sudo tshark -i eth0 -a duration: 60 -d tcp.port == 3306, mysql -T fields -e mysql.query 'port 3306 ′
Using tshark, you can log information about all requests to MySQL in the log. Start capturing all MySQL traffic with tcpdump:
$ sudo tcpdump -i eth0 port 3306 -s 1500 -w tcpdump.out
From the resulting file using tshark, we extract the list of requests:
$ sudo tshark -r tcpdump.out -d tcp.port == 3306, mysql -T fields -e mysql.query> query_log.out
Delete empty lines from this list and save the edited version in a new file:
$ sudo cat query_log.out | grep -v "^ $" | grep -v "^ commit" | grep -v "^ SET autocommit" | grep -v "^ rollback"> query_log_no_blank.out
Conclusion
Tshark is a tool with very wide capabilities, which is hardly possible to describe in detail in one article. We will be happy to answer all questions in the comments. We will be glad if you share your own experience in diagnosing network problems using tshark.
Readers who cannot post comments here are welcome to join us on our blog .