The basics of transparent proxying using 3proxy and iptables / netfilter or how to "let everything through a proxy"

In this article, I would like to reveal the possibilities of transparent proxying, which allows you to redirect all or part of the traffic through external proxy servers absolutely imperceptibly.

When I started to solve this problem, I was faced with the fact that its implementation has one significant problem - the HTTPS protocol. In the good old days, there were no special problems with transparent HTTP proxying, but when proxying HTTPS, browsers report interference with the protocol, and that is where the happiness ends.

In common instructions to the proxy server, Squid even offers to generate their own certificate and install it to clients, which is complete nonsenseat least irrational and looks like a MITM attack. I know that Squid already knows how to do something like that, but this article is about a proven and working way using 3proxy from the respected 3APA3A.

Next, we will consider in detail the process of building 3proxy from source, its configuration, full and selective proxying using NAT, channel distribution to several external proxy servers, as well as the use of a router and static routes. We use Debian 9 x64 as the OS. Getting started!

Install 3proxy and start a regular proxy server


1. Install ifconfig (from the net-tools package)
apt-get install net-tools
2. Install Midnigth Commander
apt-get install mc
3. We now have 2 interfaces:
enp0s3 - external, looks on the Internet
enp0s8 - internal, should look on the local network
In other Debian-based distributions, interfaces are usually called eth0 and eth1.
ifconfig -a

Interfaces
enp0s3: flags = 4163mtu 1500
inet 192.168.23.11 netmask 255.255.255.0 broadcast 192.168.23.255
inet6 fe80 :: a00: 27ff: fec2: bae4 prefixlen 64 scopeid 0x20 ether 08: 00: 27: c2: ba: e4 txqueuelen 1000 (Ethernet)
RX packets 6412 bytes 8676619 (8.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1726 bytes 289128 (282.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

enp0s8: flags = 4098mtu 1500
ether 08: 00: 27: 79: a7: e3 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags = 73mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 :: 1 prefixlen 128 scopeid 0x10 loop txqueuelen 1 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

The enp0s8 interface is not currently used, we will enable it when we want to use the Proxy NAT or NAT configuration. It is then logical to assign a static ip to it.

4. Let's proceed with the installation of 3proxy

4.1. Installing the basic packages for compiling 3proxy from source

root@debian9:~# apt-get install build-essential libevent-dev libssl-dev -y

4.2. Create a folder for downloading the archive with source

root@debian9:~# mkdir -p /opt/proxy

4.3. Let's go to this folder

root@debian9:~# cd /opt/proxy


4.4. Now download the latest 3proxy package. At the time of writing, the latest stable version was 0.8.12 (04/18/2018) Download it from the official site 3proxy 4.5. Unzip the downloaded archive 4.6. Go to the unpacked directory to build the program 4.7. Next, you need to add a line to the header file so that our server is completely anonymous (it really works, everything is checked, ip clients are hidden) Add a line Press Ctrl + x and Enter to save the changes. 4.8. Let's build the program

root@debian9:/opt/proxy# wget https://github.com/z3APA3A/3proxy/archive/0.8.12.tar.gz




root@debian9:/opt/proxy# tar zxvf 0.8.12.tar.gz



root@debian9:/opt/proxy# cd 3proxy-0.8.12



root@debian9:/opt/proxy/3proxy-0.8.12# nano +29 src/proxy.h



#define ANONYMOUS 1





root@debian9:/opt/proxy/3proxy-0.8.12# make -f Makefile.Linux

Makelog
make [2]: Leaving directory '/opt/proxy/3proxy-0.8.12/src/plugins/TransparentPlugin'
make [1]: Leaving directory '/opt/proxy/3proxy-0.8.12/src'

No errors, continue.

4.9. Install the program in the system

root@debian9:/opt/proxy/3proxy-0.8.12# make -f Makefile.Linux install

4.10. Go to the root directory and check where the 3proxy program has been installed : / usr / local / bin / 3proxy / usr / local / etc / 3proxy 4.11. Create a folder for configuration files and logs in the user’s home directory 4.12. Go to the directory where the 4.13 config should be . Create an empty file and copy the config there

root@debian9:/opt/proxy/3proxy-0.8.12# cd ~/
root@debian9:~# whereis 3proxy





root@debian9:~# mkdir -p /home/joke/proxy/logs



root@debian9:~# cd /home/joke/proxy/



root@debian9:/home/joke/proxy# cat > 3proxy.conf

3proxy.conf
daemon
pidfile /home/joke/proxy/3proxy.pid
nserver 8.8.8.8
nscache 65536
users tester: CL: 1234
timeouts 1 5 30 60 180 1800 16 60
log /home/joke/proxy/logs/3proxy.log D
logformat "- + _L% t.%.% N.% p% E% U% C:% c% R:% r% O% I% h% T "
rotate 3
auth strong
flush
allow tester
socks -p3128
proxy -p8080

To save, press Ctrl + Z

4.14. Create a pid file so that there are no errors at startup.

root@debian9:/home/joke/proxy# cat > 3proxy.pid

To save, press Ctrl + Z

4.15. Launch a proxy server!

root@debian9:/home/joke/proxy# 3proxy /home/joke/proxy/3proxy.conf

4.16. Let's see if the server is listening on ports

root@debian9:~/home/joke/proxy# netstat -nlp

netstat log
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID / Program name
tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 504 / 3proxy
tcp 0 0 0.0.0.0:22 0.0. 0.0: * LISTEN 338 / sshd
tcp 0 0 0.0.0.0 opin128 0.0.0.0:* LISTEN 504 / 3proxy
tcp6 0 0 ::: 22 ::: * LISTEN 338 / sshd
udp 0 0 0.0.0.0:68 0.0.0.0 : * 352 / dhclient

As it was written in the config, the web proxy listens to port 8080, Socks5 proxy listens to 3128.

4.17. To start the proxy service after rebooting, add it to cron.

root@debian9:/home/joke/proxy# crontab -e

Add a line

@reboot /usr/local/bin/3proxy /home/joke/proxy/3proxy.conf

Press Enter, since cron should see the end of line character and save the file.

There should be a message about installing a new crontab.

crontab: installing new crontab

4.18. We will reboot the system and try to connect through the browser to the proxy. For verification, we use the Firefox browser (for web proxy) and the FoxyProxy add-on for socks5 with authentication.

root@debian9:/home/joke/proxy# reboot

4.19. After checking the proxy after rebooting, you can see the logs. This completes the proxy setup.

3 proxy log
1542573996.018 PROXY.8080 00000 tester 192.168.23.10:50915 217.12.15.54ced43 1193 6939 0 CONNECT_ads.yahoo.com-
00-0043_HTTP/1.1 1542574289.634 SOCK5.3128 00000 tester 192.168.23.10∗1193 54.192.13.69-00-0043 0 0 and CONNECT_n .mozilla.net: 443

Configure and run Transparent Proxy NAT configuration


In this configuration, all devices on the internal network will transparently work on the Internet through a remote proxy server. Absolutely all tcp connections will be redirected to one or several (really expanding the channel width, configuration example No. 2!) Proxy servers. DNS will use the capabilities of 3proxy (dnspr). UDP will not go outside because we don’t yet use the forward mechanism (disabled by default in the Linux kernel).

1. It is time to enable the enp0s8 interface

root@debian9:~# nano /etc/network/interfaces

/ etc / network / interfaces file
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces (5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug enp0s3
iface enp0s3 inet dhcp

# The secondary network interface
allow-hotplug enp0s8
iface enp0s8 inet static
address 192.168 .201.254
netmask 255.255.255.0

Here we assigned the static address 192.168.201.254 and the mask 255.255.255.0 to the enp0s8 interface. We save the
Ctrl + X config and reboot

root@debian9:~# reboot

2. Check the interfaces

root@debian9:~# ifconfig

ifconfig log
enp0s3: flags = 4163mtu 1500
inet 192.168.23.11 netmask 255.255.255.0 broadcast 192.168.23.255
inet6 fe80 :: a00: 27ff: fec2: bae4 prefixlen 64 scopeid 0x20 ether 08: 00: 27: c2: ba: e4 txqueuelen 1000 (Ethernet)
RX packets 61 bytes 7873 (7.6 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 65 bytes 10917 (10.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

enp0s8: flags = 4163mtu 1500
inet 192.168.201.254 netmask 255.255.255.0 broadcast 192.168.201.255
inet6 fe80 :: a00: 27ff: fe79: a7e3 prefixlen 64 scopeid 0x20 ether 08: 00: 27: 79: a7: e3 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 8 bytes 648 (648.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags = 73mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 :: 1 prefixlen 128 scopeid 0x10 loop txqueuelen 1 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

3. Everything worked out, now you need to configure 3proxy for transparent proxying.

root@debian9:~# cd /home/joke/proxy/
root@debian9:/home/joke/proxy# cat > 3proxytransp.conf


Example configuration of transparent proxy server # 1
daemon
pidfile /home/joke/proxy/3proxy.pid
nserver 8.8.8.8
nscache 65536
timeouts 1 5 30 60 180 1800 16 60
log /home/joke/proxy/logs/3proxy.log D
logformat "- + _L% t.% .% N.% p% E% U% C:% c% R:% r% O% I% h% T "
rotate 3
flush
auth iponly
dnspr
allow *
parent 1000 socks5 IP_EXT_EXT_PROXY 3128 tester 1234
plugin / opt / proxy / 3proxy-0.8.12 / src / TransparentPlugin.ld.so transparent_plugin
tcppm -i0.0.0.0 888 127.0.0.1 11111

4. Now run 3proxy with the new config
root@debian9:/home/joke/proxy# /usr/local/bin/3proxy /home/joke/proxy/3proxytransp.conf

5. Add it to crontab again 6. Let's see what our proxy is listening to now
root@debian9:/home/joke/proxy# crontab -e
@reboot /usr/local/bin/3proxy /home/joke/proxy/3proxytransp.conf



root@debian9:~# netstat -nlp


netstat log
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID / Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 349 / sshd
tcp 0 0 0.0.0.0:888 0.0. 0.0: * LISTEN 354 / 3proxy
tcp6 0 0 ::: 22 ::: * LISTEN 349 / sshd
udp 0 0 0.0.0.0:53 0.0.0.0:* 354 / 3proxy
udp 0 0 0.0.0.0:68 0.0.0.0: * 367 / dhclient

7. Now the proxy is ready to accept any TCP connections on port 888, DNS on port 53, so that they can then be redirected to the remote socks5 - Google proxy and DNS 8.8.8.8. It remains for us to configure the netfilter (iptables) and DHCP rules for issuing addresses.

8. Install the iptables-persistent package and dhcpd

root@debian9:~# apt-get install iptables-persistent isc-dhcp-server

9. Edit the dhcpd startup file
root@debian9:~# nano /etc/dhcp/dhcpd.conf

dhcpd.conf
# dhcpd.conf
#
# Sample configuration file for ISC dhcpd
#

# option definitions common to all supported networks ...
option domain-name “example.org”;
option domain-name-servers ns1.example.org, ns2.example.org;

default-lease-time 600;
max-lease-time 7200;

ddns-update-style none;

# If this DHCP server is the official DHCP server for the local
# network, the authoritative directive should be uncommented.

authoritative;

# A slightly different configuration for an internal subnet.
subnet 192.168.201.0 netmask 255.255.255.0 {
range 192.168.201.10 192.168.201.250;
option domain-name-servers 192.168.201.254;
option routers 192.168.201.254;
option broadcast-address 192.168.201.255;
default-lease-time 600;
max-lease-time 7200;
}

11. Reboot and check the service on port 67
root@debian9:~# reboot
root@debian9:~# netstat -nlp


netstat log
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID / Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 389 / sshd
tcp 0 0 0.0.0.0:888 0.0. 0.0: * LISTEN 310 / 3proxy
tcp6 0 0 ::: 22 ::: * LISTEN 389 / sshd
udp 0 0 0.0.0.0:20364 0.0.0.0:* 393 / dhcpd
udp 0 0 0.0.0.0:53 0.0.0.0: * 310 / 3proxy
udp 0 0 0.0.0.0:67 0.0.0.0:* 393 / dhcpd
udp 0 0 0.0.0.0:68 0.0.0.0:* 405 / dhclient
udp6 0 0 ::: 31728 ::: * 393 / dhcpd
raw 0 0 0.0.0.0:1 0.0.0.0:* 393 / dhcpd


12. It remains to redirect all tcp requests to port 888 and save the rule in iptables

root@debian9:~# iptables -t nat -A PREROUTING -s 192.168.201.0/24 -p tcp -j REDIRECT --to-ports 888

root@debian9:~# iptables-save > /etc/iptables/rules.v4

13. To expand the channel bandwidth, you can use several proxy servers at once. The total amount should be 1000. New connections are established with a probability of 0.2, 0.2, 0.2, 0.2, 0.1, 0.1 to the specified proxy servers.

Note: if we have a web proxy, then instead of socks5 you need to write connect, if socks4, then socks4 (socks4 DOES NOT SUPPORT LOGIN / PASSWORD AUTHORIZATION!)

Transparent proxy server configuration example # 2
daemon
pidfile /home/joke/proxy/3proxy.pid
nserver 8.8.8.8
nscache 65536
maxconn 500
timeouts 1 5 30 60 180 1800 16 60
log /home/joke/proxy/logs/3proxy.log D
logformat "- + _L% t .%.% N.% p% E% U% C:% c% R:% r% O% I% h% T "
rotate 3
flush
auth iponly
dnspr
allow *

parent 200 socks5 IP_EXT_EXT_PROXY # 1 3128 tester 1234
parent 200 socks5 IP_EXT_PROX_NO_2 3128 tester 1234
parent 200 socks5 IP_EXT_PROX_NO3 3 3128 tester 1234
parent 200 socks5 IP_EXT_PROX_NO_ 4 3128 tester
1234_NEXPEND_3428NER_NEXT_NEXPEND_3428100_NESH_Parent_NEXT_NEXT parent_3434
parent 100 socks5 IP_EXTERNAL_PROXY # 6 3128 tester 1234

plugin /opt/proxy/3proxy-0.8.12/src/TransparentPlugin.ld.so transparent_plugin
tcppm -i0.0.0.0 888 127.0.0.1 11111

Configure and run NAT + Transparent Proxy configuration


In this configuration, we will use the usual NAT mechanism with selective or full transparent proxying of individual addresses or subnets. Users of the internal network will work with certain services / subnets without even realizing that they work through a proxy. All https connections work fine, no certificates need to be generated / replaced.

First, we decide which subnets / services we want to proxy. Suppose external proxies are located where a service such as pandora.com is running. Now it remains to determine its subnet / address.

1. We ping

root@debian9:~# ping pandora.com
PING pandora.com (208.85.40.20) 56 (84) bytes of data.

2. We type in Google BGP 208.85.40.20

Go to the website bgp.he.net/net/208.85.40.0/24#_netinfo
It can be seen that the subnet I am looking for is AS40428 Pandora Media, Inc

bgp.he.net/net/208.85.40.0/24#_netinfo

We open the v4 prefixes

bgp.he.net/AS40428#_prefixes

Here are the desired subnets!

199.116.161.0/24
199.116.162.0/24
199.116.164.0/23
199.116.164.0/24
199.116.165.0/24
208.85.40.0/24
208.85.41.0/24
208.85.42.0/23
208.85.42.0/24
208.85.43.0/24
208.85.44.0/24
208.85.46.0/23
208.85.46.0/24
208.85.47.0/24

3. To reduce the number of subnets, you need to perform aggregation. Go to ip-calculator.ru/aggregate and copy our list there. As a result - 6 subnets instead of 14.

199.116.161.0/24
199.116.162.0/24
199.116.164.0/23
208.85.40.0/22
208.85.44.0/24
208.85.46.0/23

4. We clear the iptables rules Turn on the forward and NAT mechanism So that forward is on all the time after the reboot change the file AND uncomment the line net.ipv4.ip_forward = 1 Ctrl + X to save the file 5. Wrap the pandora.com subnets in the proxy 6. Save the rules

root@debian9:~# iptables -F
root@debian9:~# iptables -X
root@debian9:~# iptables -t nat -F
root@debian9:~# iptables -t nat -X




root@debian9:~# echo 1 > /proc/sys/net/ipv4/ip_forward
root@debian9:~# iptables -A FORWARD -i enp0s3 -o enp0s8 -j ACCEPT
root@debian9:~# iptables -A FORWARD -i enp0s8 -o enp0s3 -j ACCEPT
root@debian9:~# iptables -t nat -A POSTROUTING -o enp0s3 -s 192.168.201.0/24 -j MASQUERADE




root@debian9:~# nano /etc/sysctl.conf









root@debian9:~# iptables -t nat -A PREROUTING -s 192.168.201.0/24 -d 199.116.161.0/24,199.116.162.0/24,199.116.164.0/23,208.85.40.0/22,208.85.44.0/24,208.85.46.0/23 -p tcp -j REDIRECT --to-ports 888



root@debian9:~# iptables-save > /etc/iptables/rules.v4

Configure and run the Transparent Proxy via router configuration


In this configuration, the transparent proxy server can be a separate PC or a virtual machine behind a home / corporate router. It is enough to register static routes on the router or devices and the entire subnet will use proxies without the need for any additional settings.

IMPORTANT! It is necessary that our gateway receive a static IP from the router, or be configured for static itself.

1. Configure a static gateway address (enp0s3 adapter)

root@debian9:~# nano /etc/network/interfaces

/ etc / network / interfaces file
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces (5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug enp0s3
iface enp0s3 inet static
address 192.168.23.2
netmask 255.255.255.0
gateway 192.168.23.254

# The secondary network interface
allow-hotplug enp0s8
iface enp0s8 inet static
address 192.168.201.254
netmask 255.255.255.0

2. We allow devices from the 192.168.23.0/24 subnet to use proxy

root@debian9:~# iptables -t nat -A PREROUTING -s 192.168.23.0/24 -d 199.116.161.0/24,199.116.162.0/24,199.116.164.0/23,208.85.40.0/22,208.85.44.0/24,208.85.46.0/23 -p tcp -j REDIRECT --to-ports 888

3. We will save the rules
root@debian9:~# iptables-save > /etc/iptables/rules.v4

4. We will register the subnets on the router

Router network list
199.116.161.0 255.255.255.0 192.168.23.2
199.116.162.0 255.255.255.0 192.168.23.2
199.116.164.0 255.255.254.0 192.168.23.2
208.85.40.0 255.255.252.0 192.168.23.2
208.85.44.0 255.255.255.0 192.168.23.2
208.85.46. 254.0 192.168.23.2

Materials / Resources Used


1. Official site of the 3proxy program 3proxy.ru

2. Instructions for installing 3proxy from the sources www.ekzorchik.ru/2015/02/how-to-take-your-socks-proxy

3. 3proxy developer branch on GitHub github.com/z3APA3A / 3proxy / issues / 274


Also popular now: