Introducing Linux Virtual Interfaces: Tunnels
- Transfer
Linux supports many types of tunnels. This confuses beginners who find it difficult to understand the differences in technology and understand which tunnel is better to use in a particular situation. The material, the translation of which we publish today, will give a brief overview of the commonly used tunneling interfaces of the Linux kernel. We will not delve deeply into this topic, considering only the general features of tunnels and options for their use in Linux.
The author of this material believes that what will be discussed here may be of interest to everyone who has something to do with managing computer networks. The list of tunnel interfaces, as well as reference information on a specific configuration, can be obtained using the iproute2 command .
The following commonly used interfaces will be covered here: IPIP, SIT, ip6tnl, VTI and VTI6, GRE and GRETAP, GRE6 and GRE6TAP, FOU, GUE, GENEVE, ERSPAN and IP6ERSPAN.
After reading this article, you will learn about the features of these interfaces and find out the differences between them. You will learn how to create them and learn about situations in which they are best used.
The IPIP tunnel, as its name implies, is a tunnel operating in IP over IP mode ( RFC 2003 ). The IPIP tunnel packet header is as shown below.
IPIP Tunnel Packet Header
These tunnels are typically used to connect two internal IPv4 subnets over a public IPv4 network (Internet). The use of IPIP creates a minimal additional load on the system, but only one-way data transmission (unicast) can be performed on such a tunnel. That is, having built such a tunnel, it will not be possible to use it for multicast data transmission.
IPIP tunnels support IP over IP and MPLS over IP modes.
Please note that when the module is loaded
Here's how to create an IPIP tunnel:
On server A:
On server B:
Please note that when using this configuration, it must be brought into line with real data. In particular, the
SIT (Simple Internet Transition) is a tunneling technology whose main purpose is to connect isolated IPv6 networks via the Internet using IPv4 protocol.
Initially, SIT technology could only work in the “IPv6 over IPv4” tunneling mode. However, over the years of development, it has gained the support of several more regimes. In particular, it
The mode is
The header of the SIT packet is as shown below.
SIT Tunnel Package Header
When the module loads
Here's how to create a SIT tunnel (these steps must be performed on servers A and B):
The ip6tnl interface works in IPv4 / IPv6 over IPv6 mode. It is similar to the IPv6 version of the SIT tunnel. This is what the ip6tnl package header looks like.
Ip6tnl tunnel packet header ip6tnl
tunnels support modes
When the module loads
Here's how to create an ip6tnl tunnel:
The Linux VTI (Virtual Tunnel Interface) interface is similar to the Cisco VTI interface and the Juniper implementation of a secure tunnel (st.xx).
This tunneling driver implements IP encapsulation, which can be used with xfrm to create secure tunnels and then use kernel level routing tunnels over such tunnels.
In general, VTI tunnels work much like IPIP or SIT tunnels. The exception is that they use fwmark and IPsec encapsulation / decapsulation.
VTI6 is the IPv6 equivalent of VTI.
Here's how to create a VTI tunnel:
In addition, IPsec can be configured using libreswan or strongSwan .
GRE (Generic Routing Encapsulation) technology is described in RFC 2784 . In GRE tunneling, an additional GRE header is added between the headers of the internal and external IP packets.
Theoretically, GRE can encapsulate packets of any Layer 3 protocol with a valid Ethernet type. This distinguishes GRE technology from IPIP technology, which only supports encapsulation of IP packets. Here's what the package header looks like when using GRE technology.
GRE tunnel packet header
Note that GRE tunnels allow multicasting and support IPv6.
When the module loads, the
Here's how to create a GRE tunnel:
While GRE tunnels operate at OSI layer 3, GRETAP tunnels operate at OSI layer 2. This means that one of the internal headers of the respective packets is Ethernet headers.
GRETAP Tunnel Package Header
Here's how to create a GRETAP tunnel:
GRE6 is the IPv6 equivalent of GRE. GRE6 tunnels let you encapsulate any layer 3 protocol in IPv6. This is what the GRE6 package header looks like.
GRE6 Tunnel Packet Header GRE6TAP
tunnels, like GRETAP tunnels, have Ethernet headers among the packet's internal headers.
GRE6TAP Tunnel Package Header
Here's how to create a GRE tunnel:
Tunneling can be performed at different levels of the network stack. IPIP, SIT, and GRE tunnels exist at the IP layer. And the FOU tunnels (they are arranged according to the “foo over UDP” scheme) operate at the UDP level.
There are some advantages to using UDP tunneling over IP tunneling. The fact is that the UDP protocol works with the existing hardware infrastructure.
For example, this is RSS in network cards, ECMP in switches, these are technologies for calculating checksums without the participation of a central processor. Applying the appropriate FOU patch to developers shows a significant increase in performance for SIT and IPIP.
Currently, FOU tunnels support protocol encapsulation based on IPIP, SIT, and GRE. Here's what the header of a FOU package might look like.
FOU Tunnel Package Header
Here's how to create a FOU tunnel:
The first command configures the receiving FOU port for IPIP bound to 5555. To use GRE, you must use
Note that FOU tunnels are not supported on Red Hat Enterprise Linux.
Another type of UDP tunneling is introduced by GUE ( Generic UDP Encapsulation ) technology . The difference between FOU and GUE is that GUE has its own header, which contains protocol information and other data.
GUE tunnels currently support internal encapsulation of IPIP, SIT, and GRE. Here's what a GUE package header might look like.
GUE Tunnel Package Header
Here's how to create a GUE tunnel:
Thanks to these commands, a receiving GUE port for IPIP bound to 5555 number and an IPIP tunnel configured for GUE encapsulation will be created.
GUE tunnels are not supported on Red Hat Enterprise Linux.
The GENEVE tunnels (Generic Network Virtualization Encapsulation) support all the features of XLAN, NVGRE, and STT. GENEVE technology is designed to circumvent the identified limitations of these three technologies. Many believe that this technology is capable, in the long run, of completely replacing these three older formats. Here's what the GENEVE tunnel packet header looks like.
GENEVE Tunnel Packet Header
This header is similar to the header of a VXLAN packet. The main difference between the two is that the GENEVE header is more flexible. It makes it very easy to implement new features by expanding headers using Type-Length-Value (TLV) fields.
Details about GENEVE can be found here and here .
GENEVE is used in the Open Virtual Network (OVN) SDN solution as a standard encapsulation tool. Here's how to create a GENEVE tunnel:
Encapsulated Remote Switched Port Analyzer (ERSPAN) technology uses GRE encapsulation to extend the basic capabilities of port mirroring from layer 2 to layer 3. This allows you to forward mirrored traffic over a routed IP network. This is what the ERSPAN package header looks like.
ERSPAN tunnel package header ERSPAN
tunnels allow Linux hosts to act as ERSPAN traffic sources and send mirrored ERSPAN traffic either to a remote host or to an ERSPAN destination that receives and processes ERSPAN packets generated by Cisco switches or other devices that support ERSPAN. A similar system can be used to analyze and diagnose the network, to detect malicious traffic.
Linux currently supports most of the features of the two versions of ERSPAN - v1 (type II) and v2 (type III).
Here's how to create ERSPAN tunnels:
You can also do this:
Add a tc filter to monitor traffic:
We have covered quite a few tunneling technologies in Linux here. Here is a summary table for them.
Please note that all tunnels, the creation examples of which are shown here, exist only until the server reboots. If you want to create a tunnel that recovers after rebooting, consider using a daemon to configure the network, like NetworkManager , or use the appropriate mechanism from the Linux distribution you are using.
Dear readers! What Linux tunnels do you use?
ip link help
The following commonly used interfaces will be covered here: IPIP, SIT, ip6tnl, VTI and VTI6, GRE and GRETAP, GRE6 and GRE6TAP, FOU, GUE, GENEVE, ERSPAN and IP6ERSPAN.
After reading this article, you will learn about the features of these interfaces and find out the differences between them. You will learn how to create them and learn about situations in which they are best used.
IPIP
The IPIP tunnel, as its name implies, is a tunnel operating in IP over IP mode ( RFC 2003 ). The IPIP tunnel packet header is as shown below.
IPIP Tunnel Packet Header
These tunnels are typically used to connect two internal IPv4 subnets over a public IPv4 network (Internet). The use of IPIP creates a minimal additional load on the system, but only one-way data transmission (unicast) can be performed on such a tunnel. That is, having built such a tunnel, it will not be possible to use it for multicast data transmission.
IPIP tunnels support IP over IP and MPLS over IP modes.
Please note that when the module is loaded
ipip
, or when the IPIP device is first created, the Linux kernel will create a default device tunl0
with attributes local=any
and in each namespace remote=any
. When receiving IPIP packets, the kernel, in certain cases, will redirect them totunl0
as the default device. This happens when the kernel cannot find another device whose attributes local/remote
more closely match the source and destination addresses of the packets. Here's how to create an IPIP tunnel:
On server A:
# ip link add name ipip0 type ipip local LOCAL_IPv4_ADDR remote REMOTE_IPv4_ADDR
# ip link set ipip0 up
# ip addr add INTERNAL_IPV4_ADDR/24 dev ipip0
Add a remote internal subnet route if the endpoints don't belong to the same subnet
# ip route add REMOTE_INTERNAL_SUBNET/24 dev ipip0
On server B:
# ip link add name ipip0 type ipip local LOCAL_IPv4_ADDR remote REMOTE_IPv4_ADDR
# ip link set ipip0 up
# ip addr add INTERNAL_IPV4_ADDR/24 dev ipip0
# ip route add REMOTE_INTERNAL_SUBNET/24 dev ipip0
Please note that when using this configuration, it must be brought into line with real data. In particular, the
LOCAL_IPv4_ADDR
, REMOTE_IPv4_ADDR
, INTERNAL_IPV4_ADDR
and REMOTE_INTERNAL_SUBNET
must be replaced by the addresses used in your environment. The same is true for other examples of configurations, which we will consider later.Sit
SIT (Simple Internet Transition) is a tunneling technology whose main purpose is to connect isolated IPv6 networks via the Internet using IPv4 protocol.
Initially, SIT technology could only work in the “IPv6 over IPv4” tunneling mode. However, over the years of development, it has gained the support of several more regimes. In particular, it
ipip
(the same thing happened with the IPIP-tunnel) ip6ip
, mplsip
and any
. The mode is
any
used to work with IP and IPv6 traffic, which may be useful in some situations. SIT tunnels also support ISATAP . Here is an example of using this technology. The header of the SIT packet is as shown below.
SIT Tunnel Package Header
When the module loads
sit
, the Linux kernel creates the default device sit0
. Here's how to create a SIT tunnel (these steps must be performed on servers A and B):
# ip link add name sit1 type sit local LOCAL_IPv4_ADDR remote REMOTE_IPv4_ADDR mode any
# ip link set sit1 up
# ip addr add INTERNAL_IPV4_ADDR/24 dev sit1
Ip6tnl
The ip6tnl interface works in IPv4 / IPv6 over IPv6 mode. It is similar to the IPv6 version of the SIT tunnel. This is what the ip6tnl package header looks like.
Ip6tnl tunnel packet header ip6tnl
tunnels support modes
ip6ip6
, ipip6
and any
. The mode ipip6
is represented by the IPv4 over IPv6 scheme, the mode ip6ip6
is IPv6 over IPv6. The mode any
supports both schemes. When the module loads
ip6tnl
, the Linux kernel creates a default device with the name ip6tnl0
. Here's how to create an ip6tnl tunnel:
# ip link add name ipip6 type ip6tnl local LOCAL_IPv6_ADDR remote REMOTE_IPv6_ADDR mode any
VTI and VTI6
The Linux VTI (Virtual Tunnel Interface) interface is similar to the Cisco VTI interface and the Juniper implementation of a secure tunnel (st.xx).
This tunneling driver implements IP encapsulation, which can be used with xfrm to create secure tunnels and then use kernel level routing tunnels over such tunnels.
In general, VTI tunnels work much like IPIP or SIT tunnels. The exception is that they use fwmark and IPsec encapsulation / decapsulation.
VTI6 is the IPv6 equivalent of VTI.
Here's how to create a VTI tunnel:
# ip link add name vti1 type vti key VTI_KEY local LOCAL_IPv4_ADDR remote REMOTE_IPv4_ADDR
# ip link set vti1 up
# ip addr add LOCAL_VIRTUAL_ADDR/24 dev vti1
# ip xfrm state add src LOCAL_IPv4_ADDR dst REMOTE_IPv4_ADDR spi SPI PROTO ALGR mode tunnel
# ip xfrm state add src REMOTE_IPv4_ADDR dst LOCAL_IPv4_ADDR spi SPI PROTO ALGR mode tunnel
# ip xfrm policy add dir in tmpl src REMOTE_IPv4_ADDR dst LOCAL_IPv4_ADDR PROTO mode tunnel mark VTI_KEY
# ip xfrm policy add dir out tmpl src LOCAL_IPv4_ADDR dst REMOTE_IPv4_ADDR PROTO mode tunnel mark VTI_KEY
In addition, IPsec can be configured using libreswan or strongSwan .
GRE and GRETAP
GRE (Generic Routing Encapsulation) technology is described in RFC 2784 . In GRE tunneling, an additional GRE header is added between the headers of the internal and external IP packets.
Theoretically, GRE can encapsulate packets of any Layer 3 protocol with a valid Ethernet type. This distinguishes GRE technology from IPIP technology, which only supports encapsulation of IP packets. Here's what the package header looks like when using GRE technology.
GRE tunnel packet header
Note that GRE tunnels allow multicasting and support IPv6.
When the module loads, the
gre
Linux kernel creates the default device gre0
. Here's how to create a GRE tunnel:
# ip link add name gre1 type gre local LOCAL_IPv4_ADDR remote REMOTE_IPv4_ADDR [seq] key KEY
While GRE tunnels operate at OSI layer 3, GRETAP tunnels operate at OSI layer 2. This means that one of the internal headers of the respective packets is Ethernet headers.
GRETAP Tunnel Package Header
Here's how to create a GRETAP tunnel:
# ip link add name gretap1 type gretap local LOCAL_IPv4_ADDR remote REMOTE_IPv4_ADDR
GRE6 and GRE6TAP
GRE6 is the IPv6 equivalent of GRE. GRE6 tunnels let you encapsulate any layer 3 protocol in IPv6. This is what the GRE6 package header looks like.
GRE6 Tunnel Packet Header GRE6TAP
tunnels, like GRETAP tunnels, have Ethernet headers among the packet's internal headers.
GRE6TAP Tunnel Package Header
Here's how to create a GRE tunnel:
# ip link add name gre1 type gre6 local LOCAL_IPv6_ADDR remote REMOTE_IPv6_ADDR
# ip link add name gretap1 type gretap6 local LOCAL_IPv6_ADDR remote REMOTE_IPv6_ADDR
Fou
Tunneling can be performed at different levels of the network stack. IPIP, SIT, and GRE tunnels exist at the IP layer. And the FOU tunnels (they are arranged according to the “foo over UDP” scheme) operate at the UDP level.
There are some advantages to using UDP tunneling over IP tunneling. The fact is that the UDP protocol works with the existing hardware infrastructure.
For example, this is RSS in network cards, ECMP in switches, these are technologies for calculating checksums without the participation of a central processor. Applying the appropriate FOU patch to developers shows a significant increase in performance for SIT and IPIP.
Currently, FOU tunnels support protocol encapsulation based on IPIP, SIT, and GRE. Here's what the header of a FOU package might look like.
FOU Tunnel Package Header
Here's how to create a FOU tunnel:
# ip fou add port 5555 ipproto 4
# ip link add name tun1 type ipip remote 192.168.1.1 local 192.168.1.2 ttl 225 encap fou encap-sport auto encap-dport 5555
The first command configures the receiving FOU port for IPIP bound to 5555. To use GRE, you must use
ipproto 47
. The second command configures the new virtual IPIP ( tun1
) interface , which is designed for FOU encapsulation, whose target port is 5555. Note that FOU tunnels are not supported on Red Hat Enterprise Linux.
Gue
Another type of UDP tunneling is introduced by GUE ( Generic UDP Encapsulation ) technology . The difference between FOU and GUE is that GUE has its own header, which contains protocol information and other data.
GUE tunnels currently support internal encapsulation of IPIP, SIT, and GRE. Here's what a GUE package header might look like.
GUE Tunnel Package Header
Here's how to create a GUE tunnel:
# ip fou add port 5555 gue
# ip link add name tun1 type ipip remote 192.168.1.1 local 192.168.1.2 ttl 225 encap gue encap-sport auto encap-dport 5555
Thanks to these commands, a receiving GUE port for IPIP bound to 5555 number and an IPIP tunnel configured for GUE encapsulation will be created.
GUE tunnels are not supported on Red Hat Enterprise Linux.
GENEVE
The GENEVE tunnels (Generic Network Virtualization Encapsulation) support all the features of XLAN, NVGRE, and STT. GENEVE technology is designed to circumvent the identified limitations of these three technologies. Many believe that this technology is capable, in the long run, of completely replacing these three older formats. Here's what the GENEVE tunnel packet header looks like.
GENEVE Tunnel Packet Header
This header is similar to the header of a VXLAN packet. The main difference between the two is that the GENEVE header is more flexible. It makes it very easy to implement new features by expanding headers using Type-Length-Value (TLV) fields.
Details about GENEVE can be found here and here .
GENEVE is used in the Open Virtual Network (OVN) SDN solution as a standard encapsulation tool. Here's how to create a GENEVE tunnel:
# ip link add name geneve0 type geneve id VNI remote REMOTE_IPv4_ADDR
ERSPAN and IP6ERSPAN
Encapsulated Remote Switched Port Analyzer (ERSPAN) technology uses GRE encapsulation to extend the basic capabilities of port mirroring from layer 2 to layer 3. This allows you to forward mirrored traffic over a routed IP network. This is what the ERSPAN package header looks like.
ERSPAN tunnel package header ERSPAN
tunnels allow Linux hosts to act as ERSPAN traffic sources and send mirrored ERSPAN traffic either to a remote host or to an ERSPAN destination that receives and processes ERSPAN packets generated by Cisco switches or other devices that support ERSPAN. A similar system can be used to analyze and diagnose the network, to detect malicious traffic.
Linux currently supports most of the features of the two versions of ERSPAN - v1 (type II) and v2 (type III).
Here's how to create ERSPAN tunnels:
# ip link add dev erspan1 type erspan local LOCAL_IPv4_ADDR remote REMOTE_IPv4_ADDR seq key KEY erspan_ver 1 erspan IDX
You can also do this:
# ip link add dev erspan1 type erspan local LOCAL_IPv4_ADDR remote REMOTE_IPv4_ADDR seq key KEY erspan_ver 2 erspan_dir DIRECTION erspan_hwid HWID
Add a tc filter to monitor traffic:
# tc qdisc add dev MONITOR_DEV handle ffff: ingress
# tc filter add dev MONITOR_DEV parent ffff: matchall skip_hw action mirred egress mirror dev erspan1
Summary
We have covered quite a few tunneling technologies in Linux here. Here is a summary table for them.
Tunnel / Connection Type | Outer header | Encapsulated Header | Inner header |
ipip | IPv4 | None | IPv4 |
sit | IPv4 | None | IPv4 / IPv6 |
ip6tnl | IPv4 | None | IPv4 / IPv6 |
vti | IPv4 | IPsec | IPv4 |
vti6 | IPv6 | IPsec | IPv6 |
gre | IPv4 | GRE | IPv4 / IPv6 |
gretap | IPv4 | GRE | Ether + IPv4 / IPv6 |
gre6 | IPv6 | GRE | IPv4 / IPv6 |
gre6tap | IPv6 | GRE | Ether + IPv4 / IPv6 |
fou | IPv4 / IPv6 | UDP | IPv4 / IPv6 / GRE |
gue | IPv4 / IPv6 | UDP + GUE | IPv4 / IPv6 / GRE |
geneve | IPv4 / IPv6 | UDP + Geneve | Ether + IPv4 / IPv6 |
erspan | IPv4 | GRE + ERSPAN | IPv4 / IPv6 |
ip6erspan | IPv6 | GRE + ERSPAN | IPv4 / IPv6 |
Please note that all tunnels, the creation examples of which are shown here, exist only until the server reboots. If you want to create a tunnel that recovers after rebooting, consider using a daemon to configure the network, like NetworkManager , or use the appropriate mechanism from the Linux distribution you are using.
Dear readers! What Linux tunnels do you use?