Tunnel

This article touches on ‘classic’ IP tunneling protocols.

GRE is often seen as a one size fits all solution when it comes to classic IP tunneling protocols, and for a good reason. However, there are more specialized options, and many of them are supported by VyOS. There are also rather obscure GRE options that can be useful.

All those protocols are grouped under interfaces tunnel in VyOS. Let’s take a closer look at the protocols and options currently supported by VyOS.

Common interface configuration

set interfaces tunnel <interface> address <address>

Configure interface <interface> with one or more interface addresses.

  • address can be specified multiple times as IPv4 and/or IPv6 address, e.g. 192.0.2.1/24 and/or 2001:db8::1/64

Example:

set interfaces tunnel tun0 address 192.0.2.1/24
set interfaces tunnel tun0 address 2001:db8::1/64
set interfaces tunnel <interface> description <description>

Set a human readable, descriptive alias for this connection. Alias is used by e.g. the show interfaces command or SNMP based monitoring tools.

Example:

set interfaces tunnel tun0 description 'This is an awesome interface running on VyOS'
set interfaces tunnel <interface> disable

Disable given <interface>. It will be placed in administratively down (A/D) state.

Example:

set interfaces tunnel tun0 disable
set interfaces tunnel <interface> disable-flow-control

Ethernet flow control is a mechanism for temporarily stopping the transmission of data on Ethernet family computer networks. The goal of this mechanism is to ensure zero packet loss in the presence of network congestion.

The first flow control mechanism, the pause frame, was defined by the IEEE 802.3x standard.

A sending station (computer or network switch) may be transmitting data faster than the other end of the link can accept it. Using flow control, the receiving station can signal the sender requesting suspension of transmissions until the receiver catches up.

Use this command to disable the generation of Ethernet flow control (pause frames).

Example:

set interfaces tunnel tun0 disable-flow-control
set interfaces tunnel <interface> disable-link-detect

Use this command to direct an interface to not detect any physical state changes on a link, for example, when the cable is unplugged.

Default is to detects physical link state changes.

Example:

set interfaces tunnel tun0 disable-link-detect
set interfaces tunnel <interface> mtu <mtu>

Configure MTU on given <interface>. It is the size (in bytes) of the largest ethernet frame sent on this link.

Example:

set interfaces tunnel tun0 mtu 1600
set interfaces tunnel <interface> ip adjust-mss <mss | clamp-mss-to-pmtu>

As Internet wide PMTU discovery rarely works, we sometimes need to clamp our TCP MSS value to a specific value. This is a field in the TCP options part of a SYN packet. By setting the MSS value, you are telling the remote side unequivocally ‘do not try to send me packets bigger than this value’.

Note

This command was introduced in VyOS 1.4 - it was previously called: set firewall options interface <name> adjust-mss <value>

Hint

MSS value = MTU - 20 (IP header) - 20 (TCP header), resulting in 1452 bytes on a 1492 byte MTU.

Instead of a numerical MSS value clamp-mss-to-pmtu can be used to automatically set the proper value.

set interfaces tunnel <interface> ip arp-cache-timeout

Once a neighbor has been found, the entry is considered to be valid for at least for this specific time. An entry’s validity will be extended if it receives positive feedback from higher level protocols.

This defaults to 30 seconds.

Example:

set interfaces tunnel tun0 ip arp-cache-timeout 180
set interfaces tunnel <interface> ip disable-arp-filter

If set the kernel can respond to arp requests with addresses from other interfaces. This may seem wrong but it usually makes sense, because it increases the chance of successful communication. IP addresses are owned by the complete host on Linux, not by particular interfaces. Only for more complex setups like load-balancing, does this behaviour cause problems.

If not set (default) allows you to have multiple network interfaces on the same subnet, and have the ARPs for each interface be answered based on whether or not the kernel would route a packet from the ARP’d IP out that interface (therefore you must use source based routing for this to work).

In other words it allows control of which cards (usually 1) will respond to an arp request.

Example:

set interfaces tunnel tun0 ip disable-arp-filter
set interfaces tunnel <interface> ip disable-forwarding

Configure interface-specific Host/Router behaviour. If set, the interface will switch to host mode and IPv6 forwarding will be disabled on this interface.

set interfaces tunnel tun0 ip disable-forwarding
set interfaces tunnel <interface> ip enable-directed-broadcast

Define different modes for IP directed broadcast forwarding as described in RFC 1812 and RFC 2644.

If configured, incoming IP directed broadcast packets on this interface will be forwarded.

If this option is unset (default), incoming IP directed broadcast packets will not be forwarded.

set interfaces tunnel tun0 ip enable-directed-broadcast
set interfaces tunnel <interface> ip enable-arp-accept

Define behavior for gratuitous ARP frames who’s IP is not already present in the ARP table. If configured create new entries in the ARP table.

Both replies and requests type gratuitous arp will trigger the ARP table to be updated, if this setting is on.

If the ARP table already contains the IP address of the gratuitous arp frame, the arp table will be updated regardless if this setting is on or off.

set interfaces tunnel tun0 ip enable-arp-accept
set interfaces tunnel <interface> ip enable-arp-announce

Define different restriction levels for announcing the local source IP address from IP packets in ARP requests sent on interface.

Use any local address, configured on any interface if this is not set.

If configured, try to avoid local addresses that are not in the target’s subnet for this interface. This mode is useful when target hosts reachable via this interface require the source IP address in ARP requests to be part of their logical network configured on the receiving interface. When we generate the request we will check all our subnets that include the target IP and will preserve the source address if it is from such subnet. If there is no such subnet we select source address according to the rules for level 2.

set interfaces tunnel tun0 ip enable-arp-announce
set interfaces tunnel <interface> ip enable-arp-ignore

Define different modes for sending replies in response to received ARP requests that resolve local target IP addresses:

If configured, reply only if the target IP address is local address configured on the incoming interface.

If this option is unset (default), reply for any local target IP address, configured on any interface.

set interfaces tunnel tun0 ip enable-arp-ignore
set interfaces tunnel <interface> ip enable-proxy-arp

Use this command to enable proxy Address Resolution Protocol (ARP) on this interface. Proxy ARP allows an Ethernet interface to respond with its own MAC address to ARP requests for destination IP addresses on subnets attached to other interfaces on the system. Subsequent packets sent to those destination IP addresses are forwarded appropriately by the system.

Example:

set interfaces tunnel tun0 ip enable-proxy-arp
set interfaces tunnel <interface> ip proxy-arp-pvlan

Private VLAN proxy arp. Basically allow proxy arp replies back to the same interface (from which the ARP request/solicitation was received).

This is done to support (ethernet) switch features, like RFC 3069, where the individual ports are NOT allowed to communicate with each other, but they are allowed to talk to the upstream router. As described in RFC 3069, it is possible to allow these hosts to communicate through the upstream router by proxy_arp’ing.

Note

Does not need to be used together with proxy_arp.

This technology is known by different names:

  • In RFC 3069 it is called VLAN Aggregation

  • Cisco and Allied Telesyn call it Private VLAN

  • Hewlett-Packard call it Source-Port filtering or port-isolation

  • Ericsson call it MAC-Forced Forwarding (RFC Draft)

set interfaces tunnel <interface> ip source-validation <strict | loose | disable>

Enable policy for source validation by reversed path, as specified in RFC 3704. Current recommended practice in RFC 3704 is to enable strict mode to prevent IP spoofing from DDos attacks. If using asymmetric routing or other complicated routing, then loose mode is recommended.

  • strict: Each incoming packet is tested against the FIB and if the interface is not the best reverse path the packet check will fail. By default failed packets are discarded.

  • loose: Each incoming packet’s source address is also tested against the FIB and if the source address is not reachable via any interface the packet check will fail.

  • disable: No source validation

set interfaces tunnel <interface> ipv6 address autoconf

SLAAC RFC 4862. IPv6 hosts can configure themselves automatically when connected to an IPv6 network using the Neighbor Discovery Protocol via ICMPv6 router discovery messages. When first connected to a network, a host sends a link-local router solicitation multicast request for its configuration parameters; routers respond to such a request with a router advertisement packet that contains Internet Layer configuration parameters.

Note

This method automatically disables IPv6 traffic forwarding on the interface in question.

Example:

set interfaces tunnel tun0 ipv6 address autoconf
set interfaces tunnel <interface> ipv6 address eui64 <prefix>

EUI-64 as specified in RFC 4291 allows a host to assign iteslf a unique 64-Bit IPv6 address.

Example:

set interfaces tunnel tun0 ipv6 address eui64 2001:db8:beef::/64
set interfaces tunnel <interface> ipv6 address no-default-link-local

Do not assign a link-local IPv6 address to this interface.

Example:

set interfaces tunnel tun0 ipv6 address no-default-link-local
set interfaces tunnel <interface> ipv6 disable-forwarding

Configure interface-specific Host/Router behaviour. If set, the interface will switch to host mode and IPv6 forwarding will be disabled on this interface.

Example:

set interfaces tunnel tun0 ipv6 disable-forwarding
set interfaces tunnel <interface> ipv6 adjust-mss <mss | clamp-mss-to-pmtu>

As Internet wide PMTU discovery rarely works, we sometimes need to clamp our TCP MSS value to a specific value. This is a field in the TCP options part of a SYN packet. By setting the MSS value, you are telling the remote side unequivocally ‘do not try to send me packets bigger than this value’.

Note

This command was introduced in VyOS 1.4 - it was previously called: set firewall options interface <name> adjust-mss6 <value>

Hint

MSS value = MTU - 40 (IPv6 header) - 20 (TCP header), resulting in 1432 bytes on a 1492 byte MTU.

Instead of a numerical MSS value clamp-mss-to-pmtu can be used to automatically set the proper value.

set interfaces tunnel <interface> ipv6 accept-dad <1-3>

Whether to accept DAD (Duplicate Address Detection).

  • 0: Disable DAD

  • 1: Enable DAD (default)

  • 2: Enable DAD, and disable IPv6 operation if MAC-based duplicate link-local address has been found.

Example:

set interfaces tunnel tun0 ipv6 accept-dad 2
set interfaces tunnel <interface> ipv6 dup-addr-detect-transmits <n>

The amount of Duplicate Address Detection probes to send.

Default: 1

Example:

set interfaces tunnel tun0 ipv6 dup-addr-detect-transmits 5
set interfaces tunnel <interface> vrf <vrf>

Place interface in given VRF instance.

See also

There is an entire chapter about how to configure a VRF, please check this for additional information.

Example:

set interfaces tunnel tun0 vrf red

IPIP

This is one of the simplest types of tunnels, as defined by RFC 2003. It takes an IPv4 packet and sends it as a payload of another IPv4 packet. For this reason, there are no other configuration options for this kind of tunnel.

An example:

set interfaces tunnel tun0 encapsulation ipip
set interfaces tunnel tun0 source-address 192.0.2.10
set interfaces tunnel tun0 remote 203.0.113.20
set interfaces tunnel tun0 address 192.168.100.200/24

IP6IP6

This is the IPv6 counterpart of IPIP. I’m not aware of an RFC that defines this encapsulation specifically, but it’s a natural specific case of IPv6 encapsulation mechanisms described in :rfc:2473`.

It’s not likely that anyone will need it any time soon, but it does exist.

An example:

set interfaces tunnel tun0 encapsulation ip6ip6
set interfaces tunnel tun0 source-address 2001:db8:aa::1
set interfaces tunnel tun0 remote 2001:db8:aa::2
set interfaces tunnel tun0 address 2001:db8:bb::1/64

IPIP6

In the future this is expected to be a very useful protocol (though there are other proposals).

As the name implies, it’s IPv4 encapsulated in IPv6, as simple as that.

An example:

set interfaces tunnel tun0 encapsulation ipip6
set interfaces tunnel tun0 source-address 2001:db8:aa::1
set interfaces tunnel tun0 remote 2001:db8:aa::2
set interfaces tunnel tun0 address 192.168.70.80/24

6in4 (SIT)

6in4 uses tunneling to encapsulate IPv6 traffic over IPv4 links as defined in RFC 4213. The 6in4 traffic is sent over IPv4 inside IPv4 packets whose IP headers have the IP protocol number set to 41. This protocol number is specifically designated for IPv6 encapsulation, the IPv4 packet header is immediately followed by the IPv6 packet being carried. The encapsulation overhead is the size of the IPv4 header of 20 bytes, therefore with an MTU of 1500 bytes, IPv6 packets of 1480 bytes can be sent without fragmentation. This tunneling technique is frequently used by IPv6 tunnel brokers like Hurricane Electric.

An example:

set interfaces tunnel tun0 encapsulation sit
set interfaces tunnel tun0 source-address 192.0.2.10
set interfaces tunnel tun0 remote 192.0.2.20
set interfaces tunnel tun0 address 2001:db8:bb::1/64

A full example of a Tunnelbroker.net config can be found at here.

Generic Routing Encapsulation (GRE)

A GRE tunnel operates at layer 3 of the OSI model and is represented by IP protocol 47. The main benefit of a GRE tunnel is that you are able to carry multiple protocols inside the same tunnel. GRE also supports multicast traffic and supports routing protocols that leverage multicast to form neighbor adjacencies.

A VyOS GRE tunnel can carry both IPv4 and IPv6 traffic and can also be created over either IPv4 (gre) or IPv6 (ip6gre).

Configuration

A basic configuration requires a tunnel source (source-address), a tunnel destination (remote), an encapsulation type (gre), and an address (ipv4/ipv6). Below is a basic IPv4 only configuration example taken from a VyOS router and a Cisco IOS router. The main difference between these two configurations is that VyOS requires you explicitly configure the encapsulation type. The Cisco router defaults to GRE IP otherwise it would have to be configured as well.

VyOS Router:

set interfaces tunnel tun100 address '10.0.0.1/30'
set interfaces tunnel tun100 encapsulation 'gre'
set interfaces tunnel tun100 source-address '198.51.100.2'
set interfaces tunnel tun100 remote '203.0.113.10'

Cisco IOS Router:

interface Tunnel100
ip address 10.0.0.2 255.255.255.252
tunnel source 203.0.113.10
tunnel destination 198.51.100.2

Here is a second example of a dual-stack tunnel over IPv6 between a VyOS router and a Linux host using systemd-networkd.

VyOS Router:

set interfaces tunnel tun101 address '2001:db8:feed:beef::1/126'
set interfaces tunnel tun101 address '192.168.5.1/30'
set interfaces tunnel tun101 encapsulation 'ip6gre'
set interfaces tunnel tun101 source-address '2001:db8:babe:face::3afe:3'
set interfaces tunnel tun101 remote '2001:db8:9bb:3ce::5'

Linux systemd-networkd:

This requires two files, one to create the device (XXX.netdev) and one to configure the network on the device (XXX.network)

# cat /etc/systemd/network/gre-example.netdev
[NetDev]
Name=gre-example
Kind=ip6gre
MTUBytes=14180

[Tunnel]
Remote=2001:db8:babe:face::3afe:3


# cat /etc/systemd/network/gre-example.network
[Match]
Name=gre-example

[Network]
Address=2001:db8:feed:beef::2/126

[Address]
Address=192.168.5.2/30

Tunnel keys

GRE is also the only classic protocol that allows creating multiple tunnels with the same source and destination due to its support for tunnel keys. Despite its name, this feature has nothing to do with security: it’s simply an identifier that allows routers to tell one tunnel from another.

An example:

set interfaces tunnel tun0 source-address 192.0.2.10
set interfaces tunnel tun0 remote 192.0.2.20
set interfaces tunnel tun0 address 10.40.50.60/24
set interfaces tunnel tun0 parameters ip key 10
set interfaces tunnel tun0 source-address 192.0.2.10
set interfaces tunnel tun0 remote 192.0.2.20
set interfaces tunnel tun0 address 172.16.17.18/24
set interfaces tunnel tun0 parameters ip key 20

GRETAP

While normal GRE is for layer 3, GRETAP is for layer 2. GRETAP can encapsulate Ethernet frames, thus it can be bridged with other interfaces to create datalink layer segments that span multiple remote sites.

set interfaces bridge br0 member interface eth0
set interfaces bridge br0 member interface tun0
set interfaces tunnel tun0 encapsulation gretap
set interfaces tunnel tun0 source-address 198.51.100.2
set interfaces tunnel tun0 remote 203.0.113.10

Troubleshooting

GRE is a well defined standard that is common in most networks. While not inherently difficult to configure there are a couple of things to keep in mind to make sure the configuration performs as expected. A common cause for GRE tunnels to fail to come up correctly include ACL or Firewall configurations that are discarding IP protocol 47 or blocking your source/destination traffic.

1. Confirm IP connectivity between tunnel source-address and remote:

vyos@vyos:~$ ping 203.0.113.10 interface 198.51.100.2 count 4
PING 203.0.113.10 (203.0.113.10) from 198.51.100.2 : 56(84) bytes of data.
64 bytes from 203.0.113.10: icmp_seq=1 ttl=254 time=0.807 ms
64 bytes from 203.0.113.10: icmp_seq=2 ttl=254 time=1.50 ms
64 bytes from 203.0.113.10: icmp_seq=3 ttl=254 time=0.624 ms
64 bytes from 203.0.113.10: icmp_seq=4 ttl=254 time=1.41 ms

--- 203.0.113.10 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3007ms
rtt min/avg/max/mdev = 0.624/1.087/1.509/0.381 ms

2. Confirm the link type has been set to GRE:

vyos@vyos:~$ show interfaces tunnel tun100
tun100@NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1476 qdisc noqueue state UNKNOWN group default qlen 1000
  link/gre 198.51.100.2 peer 203.0.113.10
  inet 10.0.0.1/30 brd 10.0.0.3 scope global tun100
     valid_lft forever preferred_lft forever
  inet6 fe80::5efe:c612:2/64 scope link
     valid_lft forever preferred_lft forever

  RX:  bytes    packets     errors    dropped    overrun      mcast
        2183         27          0          0          0          0
  TX:  bytes    packets     errors    dropped    carrier collisions
         836          9          0          0          0          0

3. Confirm IP connectivity across the tunnel:

vyos@vyos:~$ ping 10.0.0.2 interface 10.0.0.1 count 4
PING 10.0.0.2 (10.0.0.2) from 10.0.0.1 : 56(84) bytes of data.
64 bytes from 10.0.0.2: icmp_seq=1 ttl=255 time=1.05 ms
64 bytes from 10.0.0.2: icmp_seq=2 ttl=255 time=1.88 ms
64 bytes from 10.0.0.2: icmp_seq=3 ttl=255 time=1.98 ms
64 bytes from 10.0.0.2: icmp_seq=4 ttl=255 time=1.98 ms

--- 10.0.0.2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3008ms
rtt min/avg/max/mdev = 1.055/1.729/1.989/0.395 ms

Note

There is also a GRE over IPv6 encapsulation available, it is called: ip6gre.