June 8, 2016 at 10:46Front-door VRF. Another practical example
Hi habr! There are many articles on the Internet about setting up a VPN with VRF on Cisco equipment. There is a good cheat sheet for setting up IPsec VPN in the form of crypto cards and VTI tunnels together with VRF. In this article, Habr provides an example of DMVPN with VRF. VRF gives great flexibility when setting up equipment and its use cases are large. The main thing is not to forget that we have such a tool. In my article, I decided to consider another interesting problem from our practice, for which the use of Front-door VRF for building IPsec tunnels was also useful. It will be about building parallel VPN tunnels through different Internet providers and distributing traffic over these tunnels.
Virtual Routing and Forwarding (VRF) technology allows you to divide one physical router into several logical (virtual) ones. Thanks to this technology, it becomes possible to have several independent routing tables on the same physical router. VRF technology has become widespread in MPLS networks. However, VRF can also be used without MPLS. In this case, in Cisco terminology, the technology is called VRF lite. To be more precise, VRF-lite is a technology that allows you to build two or more VPN tunnels on one VPN concentrator, in an environment involving overlapping address spaces. In this note, the intersection of address spaces will not be affected.
We will briefly consider what gives us the use of VPN in conjunction with VRF. The protected network, the traffic of which is to be encrypted, may not be located in the global VRF of the router. The VRF of the protected network is called IVRF - Inside VRF. However, by default, the router builds IPsec in global VRF. Internet service providers can be connected to router interfaces that are also not located on the router’s global VRF. This connection of Internet providers allows you to terminate and utilize VPN tunnels from remote points simultaneously on all existing Internet connections of the router.
If ISPs are connected to the same global VRF
In this case, there is a single default route on the router in global VRF. A remote office can build parallel VPN tunnels to the router in question through all available Internet channels. But the response traffic from the router in question will go in accordance with the default route through a single Internet provider. Consequently, only the ISP indicated by the default route will be disposed of in response traffic. Consider an example based on IPsec VTI tunnels:
Tunnel Interface Settings for R1 Router:
Tunnel Interface Settings for Router R2:
For the presented configuration, the tunneled traffic will be distributed among the Internet providers of router R1 as follows:
Tunnel2 IPsec packets from R1 to R2 have a source IP address of 2.2.2.1, but pass through the channel of the first ISP1 Internet provider. Some Internet service providers block traffic with source IP addresses belonging to "foreign" Internet service providers. If the provider does the specified check, Tunnel2 will not be installed (therefore, a part of the Tunnel2 line from R1 to R2 is shown with a dashed line).
Tunnel Interface Settings for R1 Router:
interface GigabitEthernet0/0
ip address 192.168.1.1 255.255.255.0
!
interface GigabitEthernet0/1
ip address 1.1.1.1 255.255.255.252
!
interface GigabitEthernet0/2
ip address 2.2.2.2 255.255.255.252
!
interface Tunnel1
ip unnumbered GigabitEthernet0/0
tunnel source 1.1.1.1
tunnel mode ipsec ipv4
tunnel destination 3.3.3.1
tunnel path-mtu-discovery
tunnel protection ipsec profile ipsec-prof
!
interface Tunnel2
ip unnumbered GigabitEthernet0/0
tunnel source 2.2.2.1
tunnel mode ipsec ipv4
tunnel destination 3.3.3.1
tunnel path-mtu-discovery
tunnel protection ipsec profile ipsec-prof
!
ip route 0.0.0.0 0.0.0.0 1.1.1.2
ip route 0.0.0.0 0.0.0.0 2.2.2.2 10
Tunnel Interface Settings for Router R2:
interface GigabitEthernet0/0
ip address 192.168.2.1 255.255.255.0
!
interface GigabitEthernet0/1
ip address 3.3.3.1 255.255.255.252
!
interface Tunnel1
ip unnumbered GigabitEthernet0/0
tunnel source 3.3.3.1
tunnel mode ipsec ipv4
tunnel destination 1.1.1.1
tunnel path-mtu-discovery
tunnel protection ipsec profile ipsec-prof
!
interface Tunnel2
ip unnumbered GigabitEthernet0/0
tunnel source 3.3.3.1
tunnel mode ipsec ipv4
tunnel destination 2.2.2.1
tunnel path-mtu-discovery
tunnel protection ipsec profile ipsec-prof
!
ip route 0.0.0.0 0.0.0.0 3.3.3.2
For the presented configuration, the tunneled traffic will be distributed among the Internet providers of router R1 as follows:
Tunnel2 IPsec packets from R1 to R2 have a source IP address of 2.2.2.1, but pass through the channel of the first ISP1 Internet provider. Some Internet service providers block traffic with source IP addresses belonging to "foreign" Internet service providers. If the provider does the specified check, Tunnel2 will not be installed (therefore, a part of the Tunnel2 line from R1 to R2 is shown with a dashed line).
VRF, to which the channels of Internet providers are connected and in which tunneled (in particular, encrypted) traffic appears, is called FVRF - Front-door VRF. With the correct configuration of IPsec with VRF, we get the connection between IVRF and FVRF using IPsec. In other words, the protected traffic enters the IVRF, and after encapsulation, the tunnel automatically enters the FVRF, in which other routing rules apply. And it is very convenient.
I turn to the description of my example. There is a distributed network. At the central office (hereinafter referred to as the CH), there are two Internet providers, each of which is connected to its own Cisco router. The first ISP1 ISP is used for VPN to remote offices, the second ISP2 ISP is used to access the Internet. Between routers of the central heating network, the HSRP first hop reservation protocol is configured. In remote offices there are two options for boundary equipment:
- Cisco Router VTI tunnels have been built before these offices.
- The Cisco NPE Router is paired with the Cisco ASA. Prior to these offices, GRE over IPsec tunnels were built. GRE are terminated on routers and encrypted in IPsec on the Cisco ASA.
Network diagram:
Between the central office and the remote offices, routes are transmitted via EIGRP. If ISP1 or router R1 fails, all traffic goes to ISP2 and router R2. If ISP2 or R2 fails, Internet access is via ISP1 and R1, respectively. This scheme is implemented due to EIGRP, HSRP and PBR. PBR is used to redirect traffic to the Internet through ISP2. PBR is configured using ISP2 Health Tracking (IP SLA PBR Object Tracking).
Task
A third ISP3 ISP is connected to the central heating center to router R1. You must distribute the traffic across the tunnels through ISP1 and ISP3 of R1. Also, it is necessary to comply with some of the resiliency requirements for Internet service providers and central routers. I will not write down all the options for failure; I will only mention the most important requirement: ISP1, which runs regular traffic to remote offices (green tunnels in the diagram), should remain as free as possible. That is, if ISP2 fails, Internet access should go to ISP3, and if ISP3 fails, the traffic of the deployed service (red tunnels in the diagram) should go to ISP2.
Network diagram taking into account the new channel (ISP3):
Solution
Short description
Since the R1 router in normal network operation mode (when all Internet providers are operational) must terminate VPN IPsec tunnels at the same time on both Internet providers (ISP1 and ISP3), it was decided to transfer the connection to the new ISP3 Internet provider FVRF At the same time, it was decided to leave the protected (local) networks and the connection to the first Internet provider in the global VRF.
After correctly configuring IPsec together with FVRF, we have parallel operational tunnels through IPS1 and IPS3. An example IPSec configuration with FVRF will be discussed in the next section. Now it remains to distribute traffic flows over existing tunnels. To solve this problem, the following approaches can be distinguished:
- Equal Cost Multi Path (ECMP). Tunnel balancing using routes with the same metric;
- Route different IP subnets of central heating centers and / or remote offices over different tunnels;
- Performance Routing (PfR);
- Policy Based Routing (PBR);
- Use Network Address Translation (NAT) to allocate specific IP addresses and configure routing for these specific IP addresses.
For my task, the last option turned out to be the most preferable - an approach using NAT. This option allows, on the one hand, to rigidly set the distribution of traffic across tunnels (unlike ECMP and PfR), and on the other hand, to reduce the number of configuration lines (unlike PBR). Let's consider it in more detail.
On CO routers, we configure the dynamic address NAT network address translation rule. We use this rule to change the IP addresses of central heating computers to some “special” IP address when they access some previously known servers in remote offices. We use policyNAT, to change the IP addresses of clients only in cases of access to known IP addresses of servers. Further, using the EIGRP configuration, we distribute to the remote offices a “special” IP address with the best metric through ISP3 tunnels, and for tunnels through ISP1 that need to be unloaded, on the contrary, we overstate the metric for the “special” IP address.
Thus, the request from the customers of the central office gets to the server in the remote office with the changed source IP address. In response, the server generates traffic to the changed IP address, and a router in the remote office routes this traffic stream to the required tunnel. At the same time, routers in remote offices do not require additional routing settings: route management is carried out using EIGRP from the console of the central router routers. Below is a flow chart of the request / response. The real IP address of the client is 192.168.1.100. "Special" IP address to which the IP address of the client is converted: 10.10.10.10. The IP address of the server in the remote office: 172.16.1.100.
Asymmetric routing
The diagram shows that we allow asymmetric routing: requests go through tunnels ISP1, responses are returned through tunnels ISP3. In normal operation, this circuit suits. But asymmetric routing is almost always ready to bring problems. Not without difficulties, we have.
Imagine a circuit when ISP1 fails. In this case, requests begin to go through router R2 and ISP2 tunnels, and the request / response flow takes the following form:
As you can see, for request packets, the source IP address is translated on router R2. Response packets arrive through ISP3 tunnels to R1. However, in order to perform the UNNAT procedure and convert the recipient's IP address to the real IP address of the client, you have to send packets back to router R2.
For the UNNAT procedure on router R2 to succeed, the response packets must first go to the interface of the router that has the ip nat outside directive, and then to the interface that has the ip nat inside directive. As the interface with the ip nat outside directive, the Loopback interface is used. The interface with the ip nat inside directive is the internal interface of the router connected to the network core switches. If you redirect traffic from router R1 to router R2 using the existing internal interfaces connected to the network core switches, it turns out that the packets go along the following path: R2 internal interface with ip nat inside, R2 Loopback interface with ip nat outside and back to R2 internal interface c ip nat inside. From the point of view of NAT, the trajectory looks like: ip nat inside -> ip nat outside -> ip nat inside. UNNAT did not work out. It was necessary to exclude the first ip nat inside, that is, the entry of packets subject to UNNAT through the existing internal interface of router R2. I had to configure an additional logical subinterface on router R2 that did not have ip nat inside directives in order to receive traffic from R1 subject to UNNAT, and configure the corresponding routing rules on router R1.
Imagine a circuit when ISP1 fails. In this case, requests begin to go through router R2 and ISP2 tunnels, and the request / response flow takes the following form:
As you can see, for request packets, the source IP address is translated on router R2. Response packets arrive through ISP3 tunnels to R1. However, in order to perform the UNNAT procedure and convert the recipient's IP address to the real IP address of the client, you have to send packets back to router R2.
For the UNNAT procedure on router R2 to succeed, the response packets must first go to the interface of the router that has the ip nat outside directive, and then to the interface that has the ip nat inside directive. As the interface with the ip nat outside directive, the Loopback interface is used. The interface with the ip nat inside directive is the internal interface of the router connected to the network core switches. If you redirect traffic from router R1 to router R2 using the existing internal interfaces connected to the network core switches, it turns out that the packets go along the following path: R2 internal interface with ip nat inside, R2 Loopback interface with ip nat outside and back to R2 internal interface c ip nat inside. From the point of view of NAT, the trajectory looks like: ip nat inside -> ip nat outside -> ip nat inside. UNNAT did not work out. It was necessary to exclude the first ip nat inside, that is, the entry of packets subject to UNNAT through the existing internal interface of router R2. I had to configure an additional logical subinterface on router R2 that did not have ip nat inside directives in order to receive traffic from R1 subject to UNNAT, and configure the corresponding routing rules on router R1.
Description of configuration
IP addressing of the central office:
| Internal network: | 192.168.1.0/24 |
| Connection to ISP1: | 1.1.1.1/30, gateway 1.1.1.2 |
| Connection to ISP2: | 2.2.2.1/30 gateway 2.2.2.2 |
| Connection to ISP3: | 3.3.3.1/30 gateway 3.3.3.2 |
| Client IP Address: | 192.168.1.100 |
Type 1 Remote Office IP Addressing (VTI Tunnels):
| Internal network: | 172.16.1.0/24 |
| Connection to ISP: | 4.4.4.1/30 gateway 4.4.4.2 |
| Server IP: | 172.16.1.100 |
Type 2 Remote Office IP Addressing (GRE over IPsec Tunnels):
| Internal network: | 172.16.2.0/24 |
| Connection to ISP: | 5.5.5.1/30 gateway 5.5.5.2 |
| Subnet between Router and Cisco ASA: | 6.6.6.2 - Router 6.6.6.1 - Cisco ASA Mask 255.255.255.252 |
| Server IP: | 172.16.2.100 |
Network diagram:
Consider the configuration of router R1. First, I’ll give an example of setting up VPN tunnels to remote offices of type 1 (VTI tunnels) and type 2 (GRE over IPsec tunnels). In the diagram and configuration, I use the following VPN tunnel notation:
- Tunnel 10 - VTI tunnel through ISP1;
- Tunnel 11 - VTI tunnel through ISP3 configured using VRF;
- Tunnel 20 - GRE over IPsec tunnel through ISP1;
- Tunnel 21 - GRE over IPsec tunnel through ISP3 configured using VRF.
Configure VRF and interfaces:
! Настройка VRF для подключения нового Интернет-провайдера ISP3
ip vrf ISP3-vrf
!
! Настройка интерфейса для подключения нового Интернет-провайдера ISP3
interface GigabitEthernet0/2
description === ISP3 ===
ip address 3.3.3.1 255.255.255.252
ip vrf forwarding ISP3-vrf
!
! Настройка маршрута по умолчанию для нового Интернет-провайдера ISP3
ip route vrf ISP3-vrf 0.0.0.0 0.0.0.0 3.3.3.2
!
! Настройка внутреннего интерфейса с HSRP
interface GigabitEthernet0/0
description === LAN ===
ip address 192.168.1.2 255.255.255.0
standby 1 ip 192.168.1.1
standby 1 priority 105
standby 1 preempt
!
! Настройка интерфейса для подключения Интернет-провайдера ISP1
interface GigabitEthernet0/1
description === ISP1 ===
ip address 1.1.1.1 255.255.255.252
!
! Настройка маршрута по умолчанию для Интернет-провайдера ISP1
ip route 0.0.0.0 0.0.0.0 1.1.1.2Configuring a VTI tunnel through ISP1 to communicate with remote offices of type 1.
! Политика ISAKMP
crypto isakmp policy 10
encr aes
hash sha
authentication pre-share
group 2
!
! Pre-shared key
crypto isakmp key STRONGKEY address 4.4.4.1 no-xauth
!
! Политика IPsec
crypto ipsec transform-set ESP-AES-SHA esp-aes 256 esp-sha-hmac
mode tunnel
!
! Профиль IPsec
crypto ipsec profile VTI
set transform-set ESP-AES-SHA
!
! Туннельный интерфейс VTI
interface Tunnel10
description === To office Type 1 over ISP1 ===
ip unnumbered GigabitEthernet0/0
tunnel source 1.1.1.1
tunnel mode ipsec ipv4
tunnel destination 4.4.4.1
tunnel path-mtu-discovery
tunnel protection ipsec profile VTIConfiguring a VTI tunnel through the ISP3 provider in the new ISP3-vrf VRF to communicate with remote offices of type 1.
! Keyring
crypto keyring office1-keyring vrf ISP3-vrf
pre-shared-key address 4.4.4.1 key STRONGKEY
!
! Политика ISAKMP
crypto isakmp policy 10
encr aes
hash sha
authentication pre-share
group 2
!
! Профиль ISAKMP
crypto isakmp profile office1-ike-prof
keyring office1-keyring
match identity address 4.4.4.1 255.255.255.255 ISP3-vrf
isakmp authorization list default
local-address GigabitEthernet0/2
!
! Политика IPsec
crypto ipsec transform-set ESP-AES-SHA esp-aes 256 esp-sha-hmac
mode tunnel
!
! Профиль IPsec
crypto ipsec profile office1-ipsec-prof
set transform-set ESP-AES-SHA
set isakmp-profile office1-ike-prof
!
! Туннельный интерфейс VTI. Инкапсулированный трафик попадает в ISP3-vrf
interface Tunnel11
description === To office Type 1 over ISP3 ===
ip unnumbered GigabitEthernet0/0
tunnel source 3.3.3.1
tunnel mode ipsec ipv4
tunnel destination 4.4.4.1
tunnel path-mtu-discovery
tunnel vrf ISP3-vrf
tunnel protection ipsec profile office1-ipsec-prof
Configure the GRE over IPsec tunnel through ISP1 to communicate with type 2 remote offices.
! Политика ISAKMP
crypto isakmp policy 10
encr aes
hash sha
authentication pre-share
group 2
!
! Pre-shared key
crypto isakmp key STRONGKEY address 5.5.5.1 no-xauth
!
! Политика IPsec
crypto ipsec transform-set ESP-AES-SHA esp-aes 256 esp-sha-hmac
mode tunnel
!
! Крипто-карта
crypto map CMAP 10 ipsec-isakmp
description === To office Type 2 over ISP1 ===
set peer 5.5.5.1
set transform-set ESP-AES-SHA
match address cryptomap_10_acl
!
! Туннельный интерфейс GRE
interface Tunnel520
description === To office Type 2 over ISP1 ===
ip unnumbered GigabitEthernet0/0
keepalive 10 3
tunnel source 1.1.1.1
tunnel destination 6.6.6.2
tunnel path-mtu-discovery
!
! Крипто-ACL
ip access-list extended cryptomap_10_acl
permit gre 1.1.1.1 host host 6.6.6.2
!
! Применяем крипто-карту к интерфейсу ISP1
interface GigabitEthernet0/1
crypto map CMAP
Configuring the GRE over IPsec tunnel through the ISP3 provider in the new “ISP3-vrf” VRF to communicate with type 2 remote offices.
crypto keyring office2-keyring vrf ISP3-vrf
pre-shared-key address 5.5.5.1 key STRONGKEY
!
! Политика ISAKMP
crypto isakmp policy 10
encr aes
hash sha
authentication pre-share
group 2
!
! Профиль ISAKMP
crypto isakmp profile office2-ike-prof
vrf ISP3-vrf
keyring office2-keyring
match identity address 5.5.5.1 255.255.255.255 ISP3-vrf
isakmp authorization list default
!
! Политика IPsec
crypto ipsec transform-set ESP-AES-SHA esp-aes 256 esp-sha-hmac
mode tunnel
!
! Крипто-карта
crypto map CMAP-vrf 10 ipsec-isakmp
description === To office Type 2 over ISP3 ===
set peer 5.5.5.1
set transform-set ESP-AES-SHA
set isakmp-profile office2-ike-prof
match address cryptomap-vrf_10_acl
!
interface Tunnel21
description === To office Type 2 over ISP3 ===
ip unnumbered GigabitEthernet0/0
keepalive 10 3
tunnel source 3.3.3.3
tunnel destination 6.6.6.2
tunnel path-mtu-discovery
tunnel vrf ISP3-vrf
!
! Крипто-ACL
ip access-list extended cryptomap-vrf_10_acl
permit gre 3.3.3.3 host host 6.6.6.2
!
! Применяем крипто-карту к интерфейсу ISP3
interface GigabitEthernet0/2
crypto map CMAP-vrf
Using the above examples, VPN tunnels to all remote offices are configured. I will not consider VPN tunnel settings on the R2 router in the central office and on routers and Cisco ASA in remote offices in view of their triviality.
Now that the VPN tunnels to all the remote offices are ready, you can move on to the routing settings. The network uses EIGRP. First you need to configure dynamic policy NAT rules to highlight clients of the deployed system.
! Loopback интерфейс будет использоваться в правиле NAT
! для указания внутреннего глобального адреса
interface Loopback1
description ===For System-Servers routing===
ip address 10.10.10.10 255.255.255.255
!
! В группе объектов прописываются IP-адреса серверов удалённых офисов
object-group network System-Servers
description === System-Servers ===
host 172.16.1.100
host 172.16.2.100
!
! Список доступа для правила NAT
ip access-list extended NAT-System-Servers
permit ip any object-group System-Servers
!
! Карта маршрутизации для правила NAT
route-map RM-NAT-System-Servers permit 10
match ip address NAT-System-Servers
!
! Правило dynamic policy NAT
ip nat inside source route-map RM-NAT-System-Servers interface Loopback1 overload
!
! Директивы ip nat inside и ip nat outside на интерфейсах
interface GigabitEthernet0/0
ip nat inside
!
interface Tunnel10
ip nat outside
!
interface Tunnel11
ip nat outside
interface Tunnel20
ip nat outside
interface Tunnel21
ip nat outside
These NAT settings translate client IP addresses into a “special” IP address 10.10.10.10 when accessing servers in remote offices.
It remains to configure EIGRP on the router in the central office in such a way that the network is announced on 10.10.10.10/32 to remote offices with the best metrics through the tunnels of the ISP3 provider. In our example, these are Tunnel 11 and Tunnel 21. To manage the EIGRP metric, I decided to use the offset-list construct. Let me remind you, using the offset-list directive, you can add the specified value to the announced route metric. EIGRP Settings:
access-list 10 permit 10.10.10.10
!
router eigrp 1
network 192.168.1.0 0.0.0.255
offset-list 10 out 3000000 Tunnel10
offset-list 10 out 3000000 Tunnel20
passive-interface default
no passive-interface Tunnel10
no passive-interface Tunnel11
no passive-interface Tunnel20
no passive-interface Tunnel21
no passive-interface GigabitEthernet0/0
The described configuration makes the procedure of adding new servers in remote offices, the traffic from which must be routed through the tunnels of the new ISP3 central heating provider, as simple as possible. When a new server appears, it is enough to add its IP address to the System-Servers object group:
! В группе объектов прописываются IP-адреса серверов удалённых офисов
object-group network System-Servers
description === System-Servers ===
host 172.16.1.100
host 172.16.2.100
This concludes the review of the configurations of the R1 central heating router. The configuration of the R2 router in the central office and the configuration of network devices in remote offices are not of interest in this article.
Conclusion
Sometimes a simple and logical task is set for a network engineer, but solving the problem leads to the formation of relatively complicated configurations. In my example, the elementary task was considered: to connect a new Internet provider to an existing router and start the traffic of some services from remote offices on it. However, to solve this problem, it was necessary to break the router into VRF, prescribe tricky rules for translating IP addresses, adjust dynamic routing settings, and correct the consequences of asymmetric routing.
Using this task as an example, I looked at using Front-door VRF to build IPsec tunnels. In addition, I showed an option to centralize routing settings for the star topology using NAT rules. Front-door VRF for building IPsec is a convenient tool that allows you to build a bridge to transfer traffic between the VRF of the router: the protected traffic can be in IVRF or global VRF, and encapsulated (encrypted) traffic is in FVRF. FVRF has its own routing rules, which makes it possible to send FVRF traffic through additional communication channels, in particular, through an additional Internet provider.