August 16, 2010 at 01:05Tenant Internet traffic accounting (netflow)
On the issue of statistics in general, and netflow in particular, there are quite a few fairly extensive articles (the study of which requires a decent amount of time). Here I will give my as-is scripts for a not-so-difficult ToR for collecting and sending monthly statistics to tenants.
Several tenants get the Internet through NAT (NAT itself is tied through policy-routing to two providers). Each tenant has his own vilan. Vilan is “deployed” to the tenant’s network through a managed switch. Each tenant is given his own segment of the local network with a ban on going to neighboring segments.
Objective : tell each tenant and building manager at the end of the month how much he ate the Internet and poke on those users who have reused too much traffic. Those. we need: summary statistics for the segment, Internet consumption by each address (destination) in the network taken into account.
Tools: used netflow on tsiska, flow-tools on Linux, MTA, cron.
The router combines information about similar packets into flows. Packets with the same src / dst IP / port combination are considered to be similar. Those. IP packets of the file downloaded from the file wash will be merged into one stream. Information about each stream includes from whom to whom (4 fields, address and port), the number of packets, the interface through which the packet was sent, the size of the transmitted data and time (timestamp). This data is sent periodically in udp packets to the collector- a server ready to receive this data. It is clear that the collector can collect data from several senders, so when saving data, the sender is also usually written there. You need to understand that statistics are collected at all ports where it is said to collect them. If you say “collect everything on all ports”, then the traffic will be counted twice - at the time of reception (as incoming) and at the time of sending to the client interface (as outgoing). Fortunately, nat happens between receiving and sending, so with NAT we will see internal addresses in flows for only one (internal) interface.
I will not quote it in its entirety, I will show only the part related to netflow (i.e. ACLs, NAT, DHCP for those who asked for it, etc. are omitted), in addition, only one tenant interface is shown (the rest are strictly analogous) :
As you know, vlan31 (192.168.31 / 24) are tenants, similarly ± the rest of the tenants. Vlan255 - Vlan through which to send data (service). The remaining values are clear and so. (pay attention to the version of netflow, flow-tools do not support version 9).
It needs to be installed (for debian:)
Please note: netflow eats a LOT of space. Get ready for dozens of gigabytes of data that you also need to store. It is better to put this data on a separate section.
Actually, this is exactly the topic of the article. We need to configure firstly the filters by which we will select information, and secondly, a script that will send this information to anyone. At the moment, there are not very many tenants, so for each everything is configured separately.
Filters are needed for the flow-nfilter program, which takes binary flow as an input, filters it, and again outputs binary flow as an output.
Filters in /etc/flow-tools/cfg/filter.cfg. They consist of primitives (roughly speaking, names for filter elements) and filters. When calling flow-nfilter, it is shown the name of the filter by which to filter.
Here is an example for two tenants:
Actually, the main thing:
Key points: the script sends statistics for the last month. (thanks to the corresponding date option). flow is processed twice (while speed suits me, if it starts to slow down, it will be necessary to do filtering to a file and run two reports separately). The first processing gives us a human-readable number in megabytes, the second - a plate about who eats how much.
If necessary (conflict / dispute) it will be possible by hands (using flow-print and grep) to give full statistics for each chain.
I use cron features in debian, it’s a bit more convenient than usual in unix - it has the /etc/cron.monthly directory. The file /etc/cron.monthly/rent-acc is in it:
This file, at the same time, is also a configuration file for tenants (however, filters for new ones must be registered by hand!). Do not forget about chmod + x on this file.
It looks something like this:
Please note that the provision of communication services requires a license. The only loophole is the re-invoice from the provider for services. That is why statistics should be read. However, all of the above is not certified billing, i.e. requires a certain friendliness on the part of the payer.
Introductory
Several tenants get the Internet through NAT (NAT itself is tied through policy-routing to two providers). Each tenant has his own vilan. Vilan is “deployed” to the tenant’s network through a managed switch. Each tenant is given his own segment of the local network with a ban on going to neighboring segments.
Objective : tell each tenant and building manager at the end of the month how much he ate the Internet and poke on those users who have reused too much traffic. Those. we need: summary statistics for the segment, Internet consumption by each address (destination) in the network taken into account.
Tools: used netflow on tsiska, flow-tools on Linux, MTA, cron.
Theory
The router combines information about similar packets into flows. Packets with the same src / dst IP / port combination are considered to be similar. Those. IP packets of the file downloaded from the file wash will be merged into one stream. Information about each stream includes from whom to whom (4 fields, address and port), the number of packets, the interface through which the packet was sent, the size of the transmitted data and time (timestamp). This data is sent periodically in udp packets to the collector- a server ready to receive this data. It is clear that the collector can collect data from several senders, so when saving data, the sender is also usually written there. You need to understand that statistics are collected at all ports where it is said to collect them. If you say “collect everything on all ports”, then the traffic will be counted twice - at the time of reception (as incoming) and at the time of sending to the client interface (as outgoing). Fortunately, nat happens between receiving and sending, so with NAT we will see internal addresses in flows for only one (internal) interface.
Router configuration
I will not quote it in its entirety, I will show only the part related to netflow (i.e. ACLs, NAT, DHCP for those who asked for it, etc. are omitted), in addition, only one tenant interface is shown (the rest are strictly analogous) :
interface Vlan31 description RENT: OOO AVTOTRADE room 315,316,317,318 ip address 192.168.31.1 255.255.255.0 ip flow ingress ip flow egress ip nat inside ip flow-export source Vlan255 ip flow-export version 5 ip flow-export destination 192.168.255.254 3000
As you know, vlan31 (192.168.31 / 24) are tenants, similarly ± the rest of the tenants. Vlan255 - Vlan through which to send data (service). The remaining values are clear and so. (pay attention to the version of netflow, flow-tools do not support version 9).
Flow-tools configuration
Receiver
It needs to be installed (for debian:)
aptitude install flow-tools. Settings in /etc/flow-tools/capture.conf:-w / srv / netflow / 0/0/3000
Please note: netflow eats a LOT of space. Get ready for dozens of gigabytes of data that you also need to store. It is better to put this data on a separate section.
Statistics processing
Actually, this is exactly the topic of the article. We need to configure firstly the filters by which we will select information, and secondly, a script that will send this information to anyone. At the moment, there are not very many tenants, so for each everything is configured separately.
Filters
Filters are needed for the flow-nfilter program, which takes binary flow as an input, filters it, and again outputs binary flow as an output.
Filters in /etc/flow-tools/cfg/filter.cfg. They consist of primitives (roughly speaking, names for filter elements) and filters. When calling flow-nfilter, it is shown the name of the filter by which to filter.
Here is an example for two tenants:
filter-primitive rent31-range type ip-address-prefix permit 192.168.31.0/24 default deny filter-primitive rent3233-range type ip-address-prefix permit 192.168.32.0/24 permit 192.168.33.0/24 default deny filter-definition rent31-out match dst-ip-addr rent31-range filter-definition rent3233-out match dst-ip-addr ren32t33-range
Script
Actually, the main thing:
#! / bin / sh
#args:
# $ 1 - rent (01,02, ...),
# $ 2 ... $ N - email adresses to be sent
year = `date --date = '- 1 month' +% Y`
month = `date --date = '- 1 month' +% m`
net = $ 1
total = `flow-cat / srv / netflow / $ year / $ year- $ month | flow-nfilter -F rent $ net-out | flow-stat -f 15 | grep -v" # "| awk '{print $ 3 } ''
shift
(echo "Total incoming: $ total Mb"; flow-cat / srv / netflow / $ year / $ year- $ month | flow-nfilter -F rent $ net-out | flow-stat -f 8) | mail -s "report for net 192.168. $ net.0 / 24 for $ year- $ month" $ *
Key points: the script sends statistics for the last month. (thanks to the corresponding date option). flow is processed twice (while speed suits me, if it starts to slow down, it will be necessary to do filtering to a file and run two reports separately). The first processing gives us a human-readable number in megabytes, the second - a plate about who eats how much.
If necessary (conflict / dispute) it will be possible by hands (using flow-print and grep) to give full statistics for each chain.
Cron
I use cron features in debian, it’s a bit more convenient than usual in unix - it has the /etc/cron.monthly directory. The file /etc/cron.monthly/rent-acc is in it:
/ usr / local / bin / rent-acc 31 admin@domain.ru rentuser@example.com manager@company.ru / usr / local / bin / rent-acc 3233 admin@domain.ru rentuser2@foobar.baz manager@company.ru
This file, at the same time, is also a configuration file for tenants (however, filters for new ones must be registered by hand!). Do not forget about chmod + x on this file.
Result
It looks something like this:
Total incoming: 67673.187 Mb # --- ---- ---- Report Information --- --- --- # # Fields: Total # Symbols: Disabled # Sorting: None # Name: Destination IP # # Args: flow-stat -f 8 # # # IPaddr flows octets packets # 192.168.2.16 1321968 4301821906 24858456 192.168.2.2 2118220 62029649663 51512160 192.168.2.3 226383 193301694 1129250 192.168.2.8 105319 350858966 3027875 192.168.2.5 3240 60888068 681056 192.168.2.4 31289 13434866 217217 192.168.2.15 200 32104404 97818 192.168.2.14 1356 137727274 1977666 192.168.2.7 35310 553217946 10290038
Disclamer
Please note that the provision of communication services requires a license. The only loophole is the re-invoice from the provider for services. That is why statistics should be read. However, all of the above is not certified billing, i.e. requires a certain friendliness on the part of the payer.