November 14, 2014 at 01:24FreeBSD, dhcp, ip unnumbered and all all all ...
Our small company connects subscribers to the Internet using vlan-per-user technology.
It has historically developed and there are pros and cons in this, but the conversation is not about that now.
Usually for each vlan a network of gray addresses is allocated, which is then released through NAT to the big world. But sometimes subscribers want a real white address, and until recently they were given a / 30 network. Which in the realities of the present is very wasteful and subscribers with real addresses are transferred to the connection using SuperVLAN technology (RFC 3069).
All addresses, both gray and white, are issued to subscribers through DHCP. DHCP, NAT, and shaper services run on the same server running FreeBSD 9.3. The server’s hardware is quite ordinary - Core i7, 8Gb RAM, Intel E1G42ET.
Task:
To give each subscriber the address the same real address that he had, but not with the / 30 mask, but with a new / 24 mask common to all and the xxx1 gateway. Ensure the inability of the subscriber to work with someone else's real address.
For subscribers with gray addresses, continue to issue the network to the subscriber.
We are building SuperVlan ala FreeBSD
Vlan9 - a technological vlan, nothing is connected to it, traffic will go there for real addresses not yet issued to subscribers.
Vlan10 - a subscriber with a gray address
Vlan11 and vlan12 are two subscribers with real addresses.
We put the address 1.1.1.11/24, gw 1.1.1.1 on the vlan11 client with our hands - the Internet is working for the client.
If you need traffic flow between clients with real addresses, enable ProxyARP
(The bridge from private sticky vlan (man bridge) could not be used in this case, the reasons will be discussed below)
Now we will
deal with the most interesting Address distribution via DHCP
Specifically, ISC-DHCPD ver 4.3
Addresses should be issued based on the interface from which the request came.
Unfortunately, the authors of isc-dhcpd are firmly convinced that they certainly know better what the system administrator needs than the system administrator himself. And they do not let him shoot his leg, chopping off the index finger to the sysadmin to the very elbow.
As a result, you can specify a list of interfaces on which I would like to listen to requests from clients, but if in the dhcpd.conf config you do not specify the subnet to which the address on this interface gets, then it will be ignored.
And if the same addresses are specified on several interfaces, then DHCPREQ requests will be processed in one subnet.
And it is impossible to register in dhcpd.conf neither subnet, nor pool, nor host, which would be rigidly tied to the specified interface.
This is a dream, it really doesn’t work like this:
Well ... Well, there is a DHCP relay. We start the relays on the necessary ports, which will add the Option 82 Agent Circuit ID, and will give a dhcpd request.
Attempt 1:
Since dhcp categorically does not want to listen to the loopback interface, we start an additional vlan5
rc.conf:
We start - it does not start ...
dhcpd reports that it cannot bind socket.
Although the different interfaces for the relay and for the daemon are explicitly indicated, both of them just in case want to bind the fallback socket to the address *: 67
which is what this line says at the start of “Sending on Socket / fallback”.
Naturally, the one who starts the second system says: “Busy!”
I could not find how and how to disable this fallback socket.
Attempt 2:
Alternative programs are dhcprelya and dhcprelay.
Again they say that they can not do bind socket. We look into the code and understand their cardinal difference from isc-creations. ISC uses device bpf and snatches its packets from the system almost at the very beginning of the network stack, even before the firewall and other excesses ... Alternative engineers use the standard socket mechanism, but since we have the same addresses on several interfaces, then the second bind to address 1.1.1.1 : 67 the system again says: “Busy!”
Attempt 3:
And let dhcrelay listen on ALL interfaces and forward requests to dhcpd, which will be nailed only to address 192.168.5.1.
We reassemble dhcpd so that it does not use BPF, but connect via socket.
To do this, we quickly correct the Makefile of the isc-dhcp43-server port.
Add to it:
CONFIGURE_ARGS + = - enable-use-sockets
Build, run - started, but does not work. Dhcpd writes in the logs that he assigned the client an address, but the client does not receive a response from dhcpd.
We look at tcpdump as the client tries to get the address on vlan10, the one with gray addresses.
tcpdump –i vlan10 - There is a request from the client - there is no response from the server.
tcpdump –i vlan5 - silence
tcpdump –i lo0 - and here it’s interesting:
we see an udp packet from the relay to the daemon from the address 192.168.5.1 to the address 192.168.5.1
we see the udp response from the daemon to the relay from the address 192.168.5.1 to the address 192.168.10.1
And then it becomes clear that since dhcrelay listens only through BPF - it just doesn’t see packets that pass through the internal lo0 interface, it makes no sense to reassemble dhcrelay to use sockets, then we will return to the situation “Attempt 2”
Attempt 4:
Spread dhcrelay and dhcpd to SEPARATE servers.
Server dhcpd rc.conf:
Server dhcrelay rc.conf:
Does not work.
Oh yes….
Recalls tcpdump from “attempt 3”, register
route add 192.168.10.0/24 192.168.5.2
route add 1.1.1.0/24 192.168.5.2 on the dhcpd server,
now it works. Addresses to the client in vlan10 gives.
Now we add in dhcpd information about clients with real addresses.
The Internet is full of examples with classes in which substrings from agent id and circuit-id are cut, subclasses are created so that it is possible to allocate several addresses to the connection port.
One address is enough for me, so it’s enough to describe host with such a line inside:
host-identifier option agent.circuit-id “vlan11”
If you build SuperVLAN using a bridge, then dhcrelay will forward requests from all interfaces combined into a bridge with agent.circuit-id = "bridge0", so that only interfaces with the same addresses and forced routing.
Total:
Advantages of the solution - the client will automatically receive new settings, while he will have his old address.
Disadvantages - A second
ToDo server was required : Check how the situation with changing the mac address of the client is being worked out. Will dhcpd return the same real address from host if it has not yet expired on the previous poppy.
And returning to the caring and direct authors of isc-dhcpd.
In order for dhcpd to intercept requests coming to the interface through bpf, it is necessary to register in the dhcpd.conf config the subnet section which includes the address on this interface.
At the same time, there is no need to specify which subnet to use for issuing responses via relay - it looks for matches in all.
Here is the dhcpd.conf config
PS: Unexpected rake and solution:
It has historically developed and there are pros and cons in this, but the conversation is not about that now.
Usually for each vlan a network of gray addresses is allocated, which is then released through NAT to the big world. But sometimes subscribers want a real white address, and until recently they were given a / 30 network. Which in the realities of the present is very wasteful and subscribers with real addresses are transferred to the connection using SuperVLAN technology (RFC 3069).
All addresses, both gray and white, are issued to subscribers through DHCP. DHCP, NAT, and shaper services run on the same server running FreeBSD 9.3. The server’s hardware is quite ordinary - Core i7, 8Gb RAM, Intel E1G42ET.
Task:
To give each subscriber the address the same real address that he had, but not with the / 30 mask, but with a new / 24 mask common to all and the xxx1 gateway. Ensure the inability of the subscriber to work with someone else's real address.
For subscribers with gray addresses, continue to issue the network to the subscriber.
We are building SuperVlan ala FreeBSD
Vlan9 - a technological vlan, nothing is connected to it, traffic will go there for real addresses not yet issued to subscribers.
Vlan10 - a subscriber with a gray address
Vlan11 and vlan12 are two subscribers with real addresses.
ifconfig vlan9 inet 1.1.1.1/24
ifconfig vlan10 inet 192.168.10.1/24
ifconfig vlan11 inet 1.1.1.1/32
route add 1.1.1.11 –iface vlan11
ifconfig vlan12 inet 1.1.1.1/32
route add 1.1.1.12 –iface vlan12
We put the address 1.1.1.11/24, gw 1.1.1.1 on the vlan11 client with our hands - the Internet is working for the client.
If you need traffic flow between clients with real addresses, enable ProxyARP
sysctl net.link.ether.inet.proxyall=1
(The bridge from private sticky vlan (man bridge) could not be used in this case, the reasons will be discussed below)
Now we will
deal with the most interesting Address distribution via DHCP
Specifically, ISC-DHCPD ver 4.3
Addresses should be issued based on the interface from which the request came.
Unfortunately, the authors of isc-dhcpd are firmly convinced that they certainly know better what the system administrator needs than the system administrator himself. And they do not let him shoot his leg, chopping off the index finger to the sysadmin to the very elbow.
As a result, you can specify a list of interfaces on which I would like to listen to requests from clients, but if in the dhcpd.conf config you do not specify the subnet to which the address on this interface gets, then it will be ignored.
And if the same addresses are specified on several interfaces, then DHCPREQ requests will be processed in one subnet.
And it is impossible to register in dhcpd.conf neither subnet, nor pool, nor host, which would be rigidly tied to the specified interface.
This is a dream, it really doesn’t work like this:
subnet 1.1.1.11 netmask 255.255.255.255 {
option routers 1.1.1.1;
range 1.1.1.11;
interface vlan11;
}
Well ... Well, there is a DHCP relay. We start the relays on the necessary ports, which will add the Option 82 Agent Circuit ID, and will give a dhcpd request.
Attempt 1:
Since dhcp categorically does not want to listen to the loopback interface, we start an additional vlan5
rc.conf:
ifconfig_vlan5=”inet 192.168.5.1/24”
dhcrelay_flags =”-a”
dhcrelay_servers =”192.168.5.1”
dhcrelay_ifaces =”vlan11 vlan12”
dhcpd_ifaces =”vlan5 vlan10”
We start - it does not start ...
dhcpd reports that it cannot bind socket.
Although the different interfaces for the relay and for the daemon are explicitly indicated, both of them just in case want to bind the fallback socket to the address *: 67
which is what this line says at the start of “Sending on Socket / fallback”.
Naturally, the one who starts the second system says: “Busy!”
I could not find how and how to disable this fallback socket.
Attempt 2:
Alternative programs are dhcprelya and dhcprelay.
Again they say that they can not do bind socket. We look into the code and understand their cardinal difference from isc-creations. ISC uses device bpf and snatches its packets from the system almost at the very beginning of the network stack, even before the firewall and other excesses ... Alternative engineers use the standard socket mechanism, but since we have the same addresses on several interfaces, then the second bind to address 1.1.1.1 : 67 the system again says: “Busy!”
Attempt 3:
And let dhcrelay listen on ALL interfaces and forward requests to dhcpd, which will be nailed only to address 192.168.5.1.
We reassemble dhcpd so that it does not use BPF, but connect via socket.
To do this, we quickly correct the Makefile of the isc-dhcp43-server port.
Add to it:
CONFIGURE_ARGS + = - enable-use-sockets
Build, run - started, but does not work. Dhcpd writes in the logs that he assigned the client an address, but the client does not receive a response from dhcpd.
We look at tcpdump as the client tries to get the address on vlan10, the one with gray addresses.
tcpdump –i vlan10 - There is a request from the client - there is no response from the server.
tcpdump –i vlan5 - silence
tcpdump –i lo0 - and here it’s interesting:
we see an udp packet from the relay to the daemon from the address 192.168.5.1 to the address 192.168.5.1
we see the udp response from the daemon to the relay from the address 192.168.5.1 to the address 192.168.10.1
And then it becomes clear that since dhcrelay listens only through BPF - it just doesn’t see packets that pass through the internal lo0 interface, it makes no sense to reassemble dhcrelay to use sockets, then we will return to the situation “Attempt 2”
Attempt 4:
Spread dhcrelay and dhcpd to SEPARATE servers.
Server dhcpd rc.conf:
ifconfig_vlan5=”inet 192.168.5.1/24”
Server dhcrelay rc.conf:
ifconfig_vlan5=”inet 192.168.5.2/24”
dhcrelay_flags =”-a”
dhcrelay_servers =”192.168.5.1”
Does not work.
Oh yes….
Recalls tcpdump from “attempt 3”, register
route add 192.168.10.0/24 192.168.5.2
route add 1.1.1.0/24 192.168.5.2 on the dhcpd server,
now it works. Addresses to the client in vlan10 gives.
Now we add in dhcpd information about clients with real addresses.
The Internet is full of examples with classes in which substrings from agent id and circuit-id are cut, subclasses are created so that it is possible to allocate several addresses to the connection port.
One address is enough for me, so it’s enough to describe host with such a line inside:
host-identifier option agent.circuit-id “vlan11”
If you build SuperVLAN using a bridge, then dhcrelay will forward requests from all interfaces combined into a bridge with agent.circuit-id = "bridge0", so that only interfaces with the same addresses and forced routing.
Total:
Advantages of the solution - the client will automatically receive new settings, while he will have his old address.
Disadvantages - A second
ToDo server was required : Check how the situation with changing the mac address of the client is being worked out. Will dhcpd return the same real address from host if it has not yet expired on the previous poppy.
And returning to the caring and direct authors of isc-dhcpd.
In order for dhcpd to intercept requests coming to the interface through bpf, it is necessary to register in the dhcpd.conf config the subnet section which includes the address on this interface.
At the same time, there is no need to specify which subnet to use for issuing responses via relay - it looks for matches in all.
Here is the dhcpd.conf config
option domain-name "example.org";
option domain-name-servers 8.8.8.8, 8.8.4.4;
default-lease-time 600;
max-lease-time 7200;
one-lease-per-client true;
stash-agent-options true;
update-conflict-detection false;
authoritative;
log-facility local7;
subnet 192.168.5.0 netmask 255.255.255.0 {
}
subnet 192.168.10.0 netmask 255.255.255.0 {
range 192.168.10.2 192.168.10.254;
option routers 192.168.10.1;
}
subnet 192.168.14.0 netmask 255.255.255.0 {
range 192.168.14.2 192.168.14.254;
option routers 192.168.14.1;
}
subnet 1.1.1.0 netmask 255.255.255.0 {
}
group realip1 {
option routers 1.1.1.1;
host client11 {
host-identifier option agent.circuit-id "vlan11";
fixed-address 1.1.1.11;
}
host client12 {
host-identifier option agent.circuit-id "vlan12";
fixed-address 1.1.1.12;
}
host client28 {
host-identifier option agent.circuit-id "vlan28";
fixed-address 1.1.1.13;
}
}
PS: Unexpected rake and solution:
if anyone needs to raise isc dhcrelay to 8k interfaces,
you need to change the
value of FD_SETSIZE in the files
/usr/src/sys/sys/select.h
/usr/include/sys/select.h
from
#define FD_SETSIZE 1024U
to
#define FD_SETSIZE 16384U
Reconfigure 16384U dhcrelay or isc-dhcpd