History of site visits through the Mikrotik router logs
In my work, I often have to use these routers to solve my problems. In this case, it was necessary to provide Internet access through the RB450G router, being able to block certain sites by name mask and save the browsing history. An example of solving this problem using free software will be shown below.
To begin with, it was decided to configure a transparent proxy. The router has its own Web-proxy, to make it transparent, follow the example in the documentation
code> ip firewall nat add in-interface = ether1 dst-port = 80 protocol = tcp action = redirect to-ports = 8080 chain = dstnat
ip proxy set enabled = yes port = 8080
which forwards requests on the 80 (HTTP) port to the 8080 proxy port. Now you can add blocked addresses, for example
In this case, requests containing the word “facebook” will be blocked and redirected to the internal page 192.168.0.1/404 (which of course is optional). After the colon, you can use regular expressions in the dst-host parameter.
Then the question arose of how to actually accumulate and process the visit log. A built-in or other product from the manufacturer was not provided. After reading the documentation and searching the thematic forums, the product Webproxy-log was found. The product is somewhat clumsy, but suggesting a direction (although for a small load it will also work). How it works:
1. In the configuration of the logging destination, a section is added for uploading to the syslog server.
192.168.0.1 is the address of the syslog server to which we will send the log. 192.168.0.3 is the internal address of the router.
2. Create a section of the journal itself, which will use the created destination and forward the proxy server logs there
.
At this stage, the router will send logs of the form
where, with the web-proxy, account prefix, the address of the user who sent the request and the request itself will be recorded.
3. As the server, you can use the above product , which consists of two parts:
WebProxy Log Catcher - application (how the service is not installed) the simple syslog server itself, collecting logs for subsequent processing and adding time stamps.
WebProxy Log - interface for viewing logs, each time it starts, importing the logs accumulated by the collector into the local database.
Configuring the application does not cause difficulties and is sufficiently described in the documentation.
Taking into account the simple log format, it is not difficult to write your own log analyzer sent by the router, and later for your own purposes a small service with a web interface was written using Delphi and ICS components , which stores logs in a database based on SQL server express.
Perhaps all of the above will help you save time in solving such problems and introduce a little to the capabilities of routers of this manufacturer.
To begin with, it was decided to configure a transparent proxy. The router has its own Web-proxy, to make it transparent, follow the example in the documentation
code> ip firewall nat add in-interface = ether1 dst-port = 80 protocol = tcp action = redirect to-ports = 8080 chain = dstnat
ip proxy set enabled = yes port = 8080
which forwards requests on the 80 (HTTP) port to the 8080 proxy port. Now you can add blocked addresses, for example
/ip proxy access add action=deny redirect-to=192.168.0.1/404 dst-host=:facebook
In this case, requests containing the word “facebook” will be blocked and redirected to the internal page 192.168.0.1/404 (which of course is optional). After the colon, you can use regular expressions in the dst-host parameter.
Then the question arose of how to actually accumulate and process the visit log. A built-in or other product from the manufacturer was not provided. After reading the documentation and searching the thematic forums, the product Webproxy-log was found. The product is somewhat clumsy, but suggesting a direction (although for a small load it will also work). How it works:
1. In the configuration of the logging destination, a section is added for uploading to the syslog server.
/system logging action add name=proxylog target=remote remote=192.168.0.1 src-address=192.168.0.3
192.168.0.1 is the address of the syslog server to which we will send the log. 192.168.0.3 is the internal address of the router.
2. Create a section of the journal itself, which will use the created destination and forward the proxy server logs there
.
/system logging add topics=web-proxy action=proxylog
At this stage, the router will send logs of the form
web-proxy, account 192.168.0.59 GET imgcdn.ptvcdn.net/pan_img/appDownload/PandoraService/Service_Info .
xml action = allow cache = MISS
web-proxy, debug GET /pan_img/appDownload/PandoraService/Service_Info.xml HTTP / 1.1
web-proxy, debug Cache-control: no-cache
web-proxy, debug Pragma: no-cache
web- proxy, debug Host: imgcdn.ptvcdn.net
web-proxy, debug Accept: text / html, * / *
web-proxy, debug Accept-Encoding: identity
web-proxy, debug User-Agent: Mozilla / 3.0 (compatible; Indy Library)
web-proxy, debug X-Proxy-ID: 1074695054
web-proxy, debug X-Forwarded-For: 192.168.0.59
web-proxy, debug Via: 1.1 192.168.0.3 (Mikrotik HttpProxy)
web-proxy, debug
where, with the web-proxy, account prefix, the address of the user who sent the request and the request itself will be recorded.
3. As the server, you can use the above product , which consists of two parts:
WebProxy Log Catcher - application (how the service is not installed) the simple syslog server itself, collecting logs for subsequent processing and adding time stamps.
WebProxy Log - interface for viewing logs, each time it starts, importing the logs accumulated by the collector into the local database.
Configuring the application does not cause difficulties and is sufficiently described in the documentation.
Taking into account the simple log format, it is not difficult to write your own log analyzer sent by the router, and later for your own purposes a small service with a web interface was written using Delphi and ICS components , which stores logs in a database based on SQL server express.
Perhaps all of the above will help you save time in solving such problems and introduce a little to the capabilities of routers of this manufacturer.