L3VPN for Hub-and-Spoke connectivity with VyOS
IP/MPLS technology is widely used by various service providers and large enterprises in order to achieve better network scalability, manageability and flexibility. It also provides the possibility to deliver different services for the customers in a seamless manner. Layer 3 VPN (L3VPN) is a type of VPN mode that is built and delivered through OSI layer 3 networking technologies. Often the border gateway protocol (BGP) is used to send and receive VPN-related data that is responsible for the control plane. L3VPN utilizes virtual routing and forwarding (VRF) techniques to receive and deliver user data as well as separate data planes of the end-users. It is built using a combination of IP- and MPLS-based information. Generally, L3VPNs are used to send data on back-end VPN infrastructures, such as for VPN connections between data centres, HQs and branches.
An L3VPN consists of multiple access links, multiple VPN routing and forwarding (VRF) tables, and multiple MPLS paths or multiple P2MP LSPs. An L3VPN can be configured to connect two or more customer sites. In hub-and-spoke MPLS L3VPN environments, the spoke routers need to have unique Route Distinguishers (RDs). In order to use the hub site as a transit point for connectivity in such an environment, the spoke sites export their routes to the hub. Spokes can talk to hubs, but never have direct paths to other spokes. All traffic between spokes is controlled and delivered over the hub site.
To deploy a Layer3 VPN with MPLS on VyOS, we should meet a couple requirements in order to properly implement the solution. We’ll use the following nodes in our LAB environment:
2 x Route reflectors (VyOS-RRx)
4 x Provider routers (VyOS-Px)
3 x Provider Edge (VyOs-PEx)
3 x Customer Edge (VyOS-CEx)
The following software was used in the creation of this document:
Operating system: VyOS
Version: 1.4-rolling-202110310317
Image name: vyos-1.4-rolling-202110310317-amd64.iso
NOTE: VyOS Router (tested with VyOS 1.4-rolling-202110310317) – The configurations below are specifically for VyOS 1.4.x.
General information can be found in the L3VPN VRFs chapter.
Topology
How does it work?
As we know the main assumption of L3VPN “Hub and Spoke” is, that the traffic between spokes have to pass via hub, in our scenario VyOS-PE2 is the Hub PE and the VyOS-CE1-HUB is the central customer office device that is responsible for controlling access between all spokes and announcing its network prefixes (10.0.0.100/32). VyOS-PE2 has the main VRF (its name is BLUE_HUB), its own Route-Distinguisher(RD) and route-target import/export lists. Multiprotocol-BGP(MP-BGP) delivers L3VPN related control-plane information to the nodes across network where PEs Spokes import the route-target 60535:1030 (this is export route-target of vrf BLUE_HUB) and export its own route-target 60535:1011(this is vrf BLUE_SPOKE export route-target). Therefore, the Customer edge nodes can only learn the network prefixes of the HUB site [10.0.0.100/32]. For this example VyOS-CE1 has network prefixes [10.0.0.80/32] / VyOS-CE2 has network prefixes [10.0.0.90/32]. Route-Reflector devices VyOS-RR1 and VyOS-RR2 are used to simplify network routes exchange and minimize iBGP peerings between devices.
L3VPN configuration parameters table:
Node |
Role |
VRF |
RD |
RT import |
RT export |
VyOS-PE2 |
Hub |
BLUE_HUB |
10.80.80.1:1011 |
65035:1011 65035:1030 |
65035:1030 |
VyOS-PE1 |
Spoke |
BLUE_SPOKE |
10.50.50.1:1011 |
65035:1030 |
65035:1011 |
VyOS-PE3 |
Spoke |
BLUE_SPOKE |
10.60.60.1:1011 |
65035:1030 |
65035:1011 |
Configuration
Step-1: Configuring IGP and enabling MPLS LDP
At the first step we need to configure the IP/MPLS backbone network using OSPF as IGP protocol and LDP as label-switching protocol for the base connectivity between P (rovider), P (rovider) E (dge) and R (oute) R (eflector) nodes:
VyOS-P1:
# interfaces
set interfaces dummy dum10 address '10.0.0.3/32'
set interfaces ethernet eth0 address '172.16.30.1/24'
set interfaces ethernet eth1 address '172.16.40.1/24'
set interfaces ethernet eth2 address '172.16.90.1/24'
set interfaces ethernet eth3 address '172.16.10.1/24'
set interfaces ethernet eth5 address '172.16.100.1/24'
# protocols ospf+ldp
set protocols mpls interface 'eth1'
set protocols mpls interface 'eth2'
set protocols mpls interface 'eth3'
set protocols mpls interface 'eth5'
set protocols mpls interface 'eth0'
set protocols mpls ldp discovery transport-ipv4-address '10.0.0.3'
set protocols mpls ldp interface 'eth0'
set protocols mpls ldp interface 'eth1'
set protocols mpls ldp interface 'eth2'
set protocols mpls ldp interface 'eth3'
set protocols mpls ldp interface 'eth5'
set protocols mpls ldp router-id '10.0.0.3'
set protocols ospf area 0 network '0.0.0.0/0'
set protocols ospf parameters abr-type 'cisco'
set protocols ospf parameters router-id '10.0.0.3
VyOS-P2:
# interfaces
set interfaces dummy dum10 address '10.0.0.4/32'
set interfaces ethernet eth0 address '172.16.30.2/24'
set interfaces ethernet eth1 address '172.16.20.1/24'
set interfaces ethernet eth2 address '172.16.120.1/24'
set interfaces ethernet eth3 address '172.16.60.1/24'
# protocols ospf+ldp
set protocols mpls interface 'eth1'
set protocols mpls interface 'eth2'
set protocols mpls interface 'eth3'
set protocols mpls interface 'eth0'
set protocols mpls ldp discovery transport-ipv4-address '10.0.0.4'
set protocols mpls ldp interface 'eth0'
set protocols mpls ldp interface 'eth1'
set protocols mpls ldp interface 'eth2'
set protocols mpls ldp interface 'eth3'
set protocols mpls ldp router-id '10.0.0.4'
set protocols ospf area 0 network '0.0.0.0/0'
set protocols ospf parameters abr-type 'cisco'
set protocols ospf parameters router-id '10.0.0.4'
VyOS-P3:
# interfaces
set interfaces dummy dum10 address '10.0.0.5/32'
set interfaces ethernet eth0 address '172.16.110.1/24'
set interfaces ethernet eth1 address '172.16.40.2/24'
set interfaces ethernet eth2 address '172.16.50.1/24'
set interfaces ethernet eth3 address '172.16.70.1/24'
# protocols ospf + ldp
set protocols mpls interface 'eth1'
set protocols mpls interface 'eth2'
set protocols mpls interface 'eth3'
set protocols mpls interface 'eth0'
set protocols mpls ldp discovery transport-ipv4-address '10.0.0.5'
set protocols mpls ldp interface 'eth0'
set protocols mpls ldp interface 'eth1'
set protocols mpls ldp interface 'eth2'
set protocols mpls ldp interface 'eth3'
set protocols mpls ldp router-id '10.0.0.5'
set protocols ospf area 0 network '0.0.0.0/0'
set protocols ospf parameters abr-type 'cisco'
set protocols ospf parameters router-id '10.0.0.5'
VyOS-P4:
# interfaces
set interfaces dummy dum10 address '10.0.0.6/32'
set interfaces ethernet eth0 address '172.16.80.2/24'
set interfaces ethernet eth1 address '172.16.130.1/24'
set interfaces ethernet eth2 address '172.16.50.2/24'
set interfaces ethernet eth3 address '172.16.60.2/24'
set interfaces ethernet eth5 address '172.16.140.1/24'
# protocols ospf + ldp
set protocols mpls interface 'eth1'
set protocols mpls interface 'eth2'
set protocols mpls interface 'eth3'
set protocols mpls interface 'eth0'
set protocols mpls interface 'eth5'
set protocols mpls ldp discovery transport-ipv4-address '10.0.0.6'
set protocols mpls ldp interface 'eth0'
set protocols mpls ldp interface 'eth1'
set protocols mpls ldp interface 'eth2'
set protocols mpls ldp interface 'eth3'
set protocols mpls ldp interface 'eth5'
set protocols mpls ldp router-id '10.0.0.6'
set protocols ospf area 0 network '0.0.0.0/0'
set protocols ospf parameters abr-type 'cisco'
set protocols ospf parameters router-id '10.0.0.6'
VyOS-PE1:
# interfaces
set interfaces dummy dum10 address '10.0.0.7/32'
set interfaces ethernet eth0 address '172.16.90.2/24'
# protocols ospf + ldp
set protocols mpls interface 'eth0'
set protocols mpls ldp discovery transport-ipv4-address '10.0.0.7'
set protocols mpls ldp interface 'eth0'
set protocols mpls ldp router-id '10.0.0.7'
set protocols ospf area 0 network '0.0.0.0/0'
set protocols ospf parameters abr-type 'cisco'
set protocols ospf parameters router-id '10.0.0.7'
VyOS-PE2:
# interfaces
set interfaces dummy dum10 address '10.0.0.8/32'
set interfaces ethernet eth0 address '172.16.110.2/24'
set interfaces ethernet eth1 address '172.16.100.2/24'
set interfaces ethernet eth2 address '172.16.80.1/24'
# protocols ospf + ldp
set protocols mpls interface 'eth0'
set protocols mpls interface 'eth1'
set protocols mpls ldp discovery transport-ipv4-address '10.0.0.8'
set protocols mpls ldp interface 'eth0'
set protocols mpls ldp interface 'eth1'
set protocols mpls ldp router-id '10.0.0.8'
set protocols ospf area 0 network '0.0.0.0/0'
set protocols ospf parameters abr-type 'cisco'
set protocols ospf parameters router-id '10.0.0.8'
VyOS-PE3:
# interfaces
set interfaces dummy dum10 address '10.0.0.10/32'
set interfaces ethernet eth0 address '172.16.140.2/24'
# protocols ospf + ldp
set protocols mpls interface 'eth0'
set protocols mpls ldp discovery transport-ipv4-address '10.0.0.10'
set protocols mpls ldp interface 'eth0'
set protocols mpls ldp router-id '10.0.0.10'
set protocols ospf area 0 network '0.0.0.0/0'
set protocols ospf parameters abr-type 'cisco'
set protocols ospf parameters router-id '10.0.0.10'
VyOS-RR1:
# interfaces
set interfaces ethernet eth1 address '172.16.20.2/24'
set interfaces ethernet eth2 address '172.16.10.2/24'
set interfaces dummy dum10 address '10.0.0.1/32'
# protocols ospf + ldp
set protocols mpls interface 'eth1'
set protocols mpls interface 'eth2'
set protocols mpls ldp discovery transport-ipv4-address '10.0.0.1'
set protocols mpls ldp interface 'eth1'
set protocols mpls ldp interface 'eth2'
set protocols mpls ldp router-id '10.0.0.1'
set protocols ospf area 0 network '0.0.0.0/0'
set protocols ospf parameters abr-type 'cisco'
set protocols ospf parameters router-id '10.0.0.1'
VyOS-RR2:
# interfaces
set interfaces ethernet eth0 address '172.16.80.1/24'
set interfaces ethernet eth1 address '172.16.70.2/24'
set interfaces dummy dum10 address '10.0.0.2/32'
# protocols ospf + ldp
set protocols mpls interface 'eth0'
set protocols mpls interface 'eth1'
set protocols mpls ldp discovery transport-ipv4-address '10.0.0.2'
set protocols mpls ldp interface 'eth1'
set protocols mpls ldp interface 'eth0'
set protocols mpls ldp router-id '10.0.0.2'
set protocols ospf area 0 network '0.0.0.0/0'
set protocols ospf parameters abr-type 'cisco'
set protocols ospf parameters router-id '10.0.0.2'
Step-2: Configuring iBGP for L3VPN control-plane
At this step we are going to enable iBGP protocol on MPLS nodes and Route Reflectors (two routers for redundancy) that will deliver IPv4 VPN (L3VPN) routes between them:
VyOS-RR1:
set protocols bgp system-as '65001'
set protocols bgp neighbor 10.0.0.7 address-family ipv4-vpn route-reflector-client
set protocols bgp neighbor 10.0.0.7 peer-group 'RR_VPNv4'
set protocols bgp neighbor 10.0.0.8 address-family ipv4-vpn route-reflector-client
set protocols bgp neighbor 10.0.0.8 peer-group 'RR_VPNv4'
set protocols bgp neighbor 10.0.0.10 address-family ipv4-vpn route-reflector-client
set protocols bgp neighbor 10.0.0.10 peer-group 'RR_VPNv4'
set protocols bgp parameters cluster-id '10.0.0.1'
set protocols bgp parameters log-neighbor-changes
set protocols bgp parameters router-id '10.0.0.1'
set protocols bgp peer-group RR_VPNv4 remote-as '65001'
set protocols bgp peer-group RR_VPNv4 update-source 'dum10'
VyOS-RR2:
set protocols bgp system-as '65001'
set protocols bgp neighbor 10.0.0.7 address-family ipv4-vpn route-reflector-client
set protocols bgp neighbor 10.0.0.7 peer-group 'RR_VPNv4'
set protocols bgp neighbor 10.0.0.8 address-family ipv4-vpn route-reflector-client
set protocols bgp neighbor 10.0.0.8 peer-group 'RR_VPNv4'
set protocols bgp neighbor 10.0.0.10 address-family ipv4-vpn route-reflector-client
set protocols bgp neighbor 10.0.0.10 peer-group 'RR_VPNv4'
set protocols bgp parameters cluster-id '10.0.0.1'
set protocols bgp parameters log-neighbor-changes
set protocols bgp parameters router-id '10.0.0.2'
set protocols bgp peer-group RR_VPNv4 remote-as '65001'
set protocols bgp peer-group RR_VPNv4 update-source 'dum10'
VyOS-PE1:
set protocols bgp system-as '65001'
set protocols bgp neighbor 10.0.0.1 address-family ipv4-vpn nexthop-self
set protocols bgp neighbor 10.0.0.1 peer-group 'RR_VPNv4'
set protocols bgp neighbor 10.0.0.2 address-family ipv4-vpn nexthop-self
set protocols bgp neighbor 10.0.0.2 peer-group 'RR_VPNv4'
set protocols bgp parameters log-neighbor-changes
set protocols bgp parameters router-id '10.0.0.7'
set protocols bgp peer-group RR_VPNv4 remote-as '65001'
set protocols bgp peer-group RR_VPNv4 update-source 'dum10'
VyOS-PE2:
set protocols bgp system-as '65001'
set protocols bgp neighbor 10.0.0.1 address-family ipv4-vpn nexthop-self
set protocols bgp neighbor 10.0.0.1 peer-group 'RR_VPNv4'
set protocols bgp neighbor 10.0.0.2 address-family ipv4-vpn nexthop-self
set protocols bgp neighbor 10.0.0.2 peer-group 'RR_VPNv4'
set protocols bgp parameters log-neighbor-changes
set protocols bgp parameters router-id '10.0.0.8'
set protocols bgp peer-group RR_VPNv4 remote-as '65001'
set protocols bgp peer-group RR_VPNv4 update-source 'dum10'
VyOS-PE3:
set protocols bgp system-as '65001'
set protocols bgp neighbor 10.0.0.1 address-family ipv4-vpn nexthop-self
set protocols bgp neighbor 10.0.0.1 peer-group 'RR_VPNv4'
set protocols bgp neighbor 10.0.0.2 address-family ipv4-vpn nexthop-self
set protocols bgp neighbor 10.0.0.2 peer-group 'RR_VPNv4'
set protocols bgp parameters log-neighbor-changes
set protocols bgp parameters router-id '10.0.0.10'
set protocols bgp peer-group RR_VPNv4 remote-as '65001'
set protocols bgp peer-group RR_VPNv4 update-source 'dum10'
Step-3: Configuring L3VPN VRFs on PE nodes
This section provides configuration steps for setting up VRFs on our PE nodes including CE facing interfaces, BGP, rd and route-target import/export based on the pre-defined parameters.
VyOS-PE1:
# VRF settings
set vrf name BLUE_SPOKE table '200'
set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast export vpn
set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast import vpn
set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast label vpn export 'auto'
set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast network 10.50.50.0/24
set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast rd vpn export '10.50.50.1:1011'
set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast redistribute connected
set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast route-target vpn export '65035:1011'
set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast route-target vpn import '65035:1030'
set vrf name BLUE_SPOKE protocols bgp system-as '65001'
set vrf name BLUE_SPOKE protocols bgp neighbor 10.50.50.2 address-family ipv4-unicast as-override
set vrf name BLUE_SPOKE protocols bgp neighbor 10.50.50.2 remote-as '65035'
# interfaces
set interfaces ethernet eth3 address '10.50.50.1/24'
set interfaces ethernet eth3 vrf 'BLUE_SPOKE'
VyOS-PE2:
# VRF settings
set vrf name BLUE_HUB table '400'
set vrf name BLUE_HUB protocols bgp address-family ipv4-unicast export vpn
set vrf name BLUE_HUB protocols bgp address-family ipv4-unicast import vpn
set vrf name BLUE_HUB protocols bgp address-family ipv4-unicast label vpn export 'auto'
set vrf name BLUE_HUB protocols bgp address-family ipv4-unicast network 10.80.80.0/24
set vrf name BLUE_HUB protocols bgp address-family ipv4-unicast rd vpn export '10.80.80.1:1011'
set vrf name BLUE_HUB protocols bgp address-family ipv4-unicast redistribute connected
set vrf name BLUE_HUB protocols bgp address-family ipv4-unicast route-target vpn export '65035:1030'
set vrf name BLUE_HUB protocols bgp address-family ipv4-unicast route-target vpn import '65035:1011 65050:2011 65035:1030'
set vrf name BLUE_HUB protocols bgp system-as '65001'
set vrf name BLUE_HUB protocols bgp neighbor 10.80.80.2 address-family ipv4-unicast as-override
set vrf name BLUE_HUB protocols bgp neighbor 10.80.80.2 remote-as '65035'
# interfaces
set interfaces ethernet eth3 address '10.80.80.1/24'
set interfaces ethernet eth3 vrf 'BLUE_HUB'
VyOS-PE3:
# VRF settings
set vrf name BLUE_SPOKE table '200'
set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast export vpn
set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast import vpn
set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast label vpn export 'auto'
set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast network 10.60.60.0/24
set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast rd vpn export '10.60.60.1:1011'
set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast redistribute connected
set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast route-target vpn export '65035:1011'
set vrf name BLUE_SPOKE protocols bgp address-family ipv4-unicast route-target vpn import '65035:1030'
set vrf name BLUE_SPOKE protocols bgp system-as '65001'
set vrf name BLUE_SPOKE protocols bgp neighbor 10.60.60.2 address-family ipv4-unicast as-override
set vrf name BLUE_SPOKE protocols bgp neighbor 10.60.60.2 remote-as '65035'
# interfaces
set interfaces ethernet eth3 address '10.60.60.1/24'
set interfaces ethernet eth3 vrf 'BLUE_SPOKE'
Step-4: Configuring CE nodes
Dynamic routing used between CE and PE nodes and eBGP peering established for the route exchanging between them. All routes received by PEs are then exported to L3VPN and delivered from Spoke sites to Hub and vise-versa based on previously configured L3VPN parameters.
VyOS-CE1-SPOKE:
# interfaces
set interfaces dummy dum20 address '10.0.0.80/32'
set interfaces ethernet eth0 address '10.50.50.2/24'
# BGP for peering with PE
set protocols bgp system-as 65035
set protocols bgp address-family ipv4-unicast network 10.0.0.80/32
set protocols bgp neighbor 10.50.50.1 ebgp-multihop '2'
set protocols bgp neighbor 10.50.50.1 remote-as '65001'
set protocols bgp neighbor 10.50.50.1 update-source 'eth0'
set protocols bgp parameters log-neighbor-changes
set protocols bgp parameters router-id '10.50.50.2'
VyOS-CE1-HUB:
# interfaces
set interfaces dummy dum20 address '10.0.0.100/32'
set interfaces ethernet eth0 address '10.80.80.2/24'
# BGP for peering with PE
set protocols bgp system-as 65035
set protocols bgp address-family ipv4-unicast network 10.0.0.100/32
set protocols bgp address-family ipv4-unicast redistribute connected
set protocols bgp neighbor 10.80.80.1 ebgp-multihop '2'
set protocols bgp neighbor 10.80.80.1 remote-as '65001'
set protocols bgp neighbor 10.80.80.1 update-source 'eth0'
set protocols bgp parameters log-neighbor-changes
set protocols bgp parameters router-id '10.80.80.2'
VyOS-CE2-SPOKE:
# interfaces
set interfaces dummy dum20 address '10.0.0.90/32'
set interfaces ethernet eth0 address '10.60.60.2/24'
# BGP for peering with PE
set protocols bgp system-as 65035
set protocols bgp address-family ipv4-unicast network 10.0.0.90/32
set protocols bgp neighbor 10.60.60.1 ebgp-multihop '2'
set protocols bgp neighbor 10.60.60.1 remote-as '65001'
set protocols bgp neighbor 10.60.60.1 update-source 'eth0'
set protocols bgp parameters log-neighbor-changes
set protocols bgp parameters router-id '10.60.60.2'
Step-5: Verification
This section describes verification commands for MPLS/BGP/LDP protocols and L3VPN related routes as well as diagnosis and reachability checks between CE nodes.
Let’s check IPv4 routing and MPLS information on provider nodes (same procedure for all P nodes):
“show ip ospf neighbor” for checking ospf relationship
vyos@VyOS-P1:~$ show ip ospf neighbor
Neighbor ID Pri State Dead Time Address Interface RXmtL RqstL DBsmL
10.0.0.4 1 Full/Backup 34.718s 172.16.30.2 eth0:172.16.30.1 0 0 0
10.0.0.5 1 Full/Backup 35.132s 172.16.40.2 eth1:172.16.40.1 0 0 0
10.0.0.7 1 Full/Backup 34.764s 172.16.90.2 eth2:172.16.90.1 0 0 0
10.0.0.1 1 Full/Backup 35.642s 172.16.10.2 eth3:172.16.10.1 0 0 0
10.0.0.8 1 Full/Backup 35.484s 172.16.100.2 eth5:172.16.100.1 0 0 0
“show mpls ldp neighbor “ for checking ldp neighbors
vyos@VyOS-P1:~$ show mpls ldp neighbor
AF ID State Remote Address Uptime
ipv4 10.0.0.1 OPERATIONAL 10.0.0.1 07w5d06h
ipv4 10.0.0.4 OPERATIONAL 10.0.0.4 09w3d00h
ipv4 10.0.0.5 OPERATIONAL 10.0.0.5 09w2d23h
ipv4 10.0.0.7 OPERATIONAL 10.0.0.7 03w0d01h
ipv4 10.0.0.8 OPERATIONAL 10.0.0.8 01w3d02h
“show mpls ldp binding” for checking mpls label assignment
vyos@VyOS-P1:~$ show mpls ldp discovery
AF Destination Nexthop Local Label Remote Label In Use
ipv4 10.0.0.1/32 10.0.0.1 23 imp-null yes
ipv4 10.0.0.1/32 10.0.0.4 23 20 no
ipv4 10.0.0.1/32 10.0.0.5 23 17 no
ipv4 10.0.0.1/32 10.0.0.7 23 16 no
ipv4 10.0.0.1/32 10.0.0.8 23 16 no
ipv4 10.0.0.2/32 10.0.0.1 20 16 no
ipv4 10.0.0.2/32 10.0.0.4 20 22 no
ipv4 10.0.0.2/32 10.0.0.5 20 24 yes
ipv4 10.0.0.2/32 10.0.0.7 20 17 no
ipv4 10.0.0.2/32 10.0.0.8 20 17 no
ipv4 10.0.0.3/32 10.0.0.1 imp-null 17 no
ipv4 10.0.0.3/32 10.0.0.4 imp-null 16 no
ipv4 10.0.0.3/32 10.0.0.5 imp-null 18 no
ipv4 10.0.0.3/32 10.0.0.7 imp-null 18 no
ipv4 10.0.0.3/32 10.0.0.8 imp-null 18 no
ipv4 10.0.0.4/32 10.0.0.1 16 18 no
ipv4 10.0.0.4/32 10.0.0.4 16 imp-null yes
ipv4 10.0.0.4/32 10.0.0.5 16 19 no
ipv4 10.0.0.4/32 10.0.0.7 16 19 no
ipv4 10.0.0.4/32 10.0.0.8 16 19 no
ipv4 10.0.0.5/32 10.0.0.1 21 19 no
ipv4 10.0.0.5/32 10.0.0.4 21 17 no
ipv4 10.0.0.5/32 10.0.0.5 21 imp-null yes
ipv4 10.0.0.5/32 10.0.0.7 21 20 no
ipv4 10.0.0.5/32 10.0.0.8 21 20 no
ipv4 10.0.0.6/32 10.0.0.1 17 20 no
ipv4 10.0.0.6/32 10.0.0.4 17 23 yes
ipv4 10.0.0.6/32 10.0.0.5 17 21 yes
ipv4 10.0.0.6/32 10.0.0.7 17 21 no
ipv4 10.0.0.6/32 10.0.0.8 17 21 no
ipv4 10.0.0.7/32 10.0.0.1 22 21 no
ipv4 10.0.0.7/32 10.0.0.4 22 18 no
ipv4 10.0.0.7/32 10.0.0.5 22 20 no
ipv4 10.0.0.7/32 10.0.0.7 22 imp-null yes
ipv4 10.0.0.7/32 10.0.0.8 22 22 no
ipv4 10.0.0.8/32 10.0.0.1 24 22 no
ipv4 10.0.0.8/32 10.0.0.4 24 19 no
ipv4 10.0.0.8/32 10.0.0.5 24 16 no
ipv4 10.0.0.8/32 10.0.0.7 24 22 no
ipv4 10.0.0.8/32 10.0.0.8 24 imp-null yes
ipv4 10.0.0.9/32 10.0.0.1 18 23 no
ipv4 10.0.0.9/32 10.0.0.4 18 21 yes
ipv4 10.0.0.9/32 10.0.0.5 18 22 no
ipv4 10.0.0.9/32 10.0.0.7 18 23 no
ipv4 10.0.0.9/32 10.0.0.8 18 23 no
ipv4 10.0.0.10/32 10.0.0.1 19 24 no
ipv4 10.0.0.10/32 10.0.0.4 19 24 yes
ipv4 10.0.0.10/32 10.0.0.5 19 23 yes
ipv4 10.0.0.10/32 10.0.0.7 19 24 no
ipv4 10.0.0.10/32 10.0.0.8 19 24 no
Now we’re checking iBGP status and routes from route-reflector nodes to other devices:
“show bgp ipv4 vpn summary” for checking BGP VPNv4 neighbors:
vyos@VyOS-RR1:~$ show bgp ipv4 vpn summary
BGP router identifier 10.0.0.1, local AS number 65001 vrf-id 0
BGP table version 0
RIB entries 9, using 1728 bytes of memory
Peers 4, using 85 KiB of memory
Peer groups 1, using 64 bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt
10.0.0.7 4 65001 7719 7733 0 0 0 5d07h56m 2 10
10.0.0.8 4 65001 7715 7724 0 0 0 5d08h28m 4 10
10.0.0.9 4 65001 7713 7724 0 0 0 5d08h28m 2 10
10.0.0.10 4 65001 7713 7724 0 0 0 5d08h28m 2 10
Total number of neighbors 4
“show bgp ipv4 vpn” for checking all VPNv4 prefixes information:
vyos@VyOS-RR1:~$ show bgp ipv4 vpn
BGP table version is 2, local router ID is 10.0.0.1, vrf id 0
Default local pref 100, local AS 65001
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 10.50.50.1:1011
*>i10.50.50.0/24 10.0.0.7 0 100 0 i
UN=10.0.0.7 EC{65035:1011} label=80 type=bgp, subtype=0
*>i80.80.80.80/32 10.0.0.7 0 100 0 65035 i
UN=10.0.0.7 EC{65035:1011} label=80 type=bgp, subtype=0
Route Distinguisher: 10.60.60.1:1011
*>i10.60.60.0/24 10.0.0.10 0 100 0 i
UN=10.0.0.10 EC{65035:1011} label=80 type=bgp, subtype=0
*>i90.90.90.90/32 10.0.0.10 0 100 0 65035 i
UN=10.0.0.10 EC{65035:1011} label=80 type=bgp, subtype=0
Route Distinguisher: 10.80.80.1:1011
*>i10.80.80.0/24 10.0.0.8 0 100 0 i
UN=10.0.0.8 EC{65035:1030} label=80 type=bgp, subtype=0
*>i100.100.100.100/32
10.0.0.8 0 100 0 65035 i
UN=10.0.0.8 EC{65035:1030} label=80 type=bgp, subtype=0
Route Distinguisher: 172.16.80.1:2011
*>i10.110.110.0/24 10.0.0.8 0 100 0 65050 i
UN=10.0.0.8 EC{65050:2011} label=81 type=bgp, subtype=0
*>i172.16.80.0/24 10.0.0.8 0 100 0 i
UN=10.0.0.8 EC{65050:2011} label=81 type=bgp, subtype=0
Route Distinguisher: 172.16.100.1:2011
*>i10.210.210.0/24 10.0.0.9 0 100 0 65050 i
UN=10.0.0.9 EC{65050:2011} label=80 type=bgp, subtype=0
*>i172.16.100.0/24 10.0.0.9 0 100 0 i
UN=10.0.0.9 EC{65050:2011} label=80 type=bgp, subtype=0
“show bgp ipv4 vpn x.x.x.x/x” for checking best path selected for specific VPNv4 destination
vyos@VyOS-RR1:~$ show bgp ipv4 vpn 10.0.0.100/32
BGP routing table entry for 10.80.80.1:1011:10.0.0.100/32
not allocated
Paths: (1 available, best #1)
Advertised to non peer-group peers:
10.0.0.7 10.0.0.8 10.0.0.9 10.0.0.10
65035, (Received from a RR-client)
10.0.0.8 from 10.0.0.8 (10.0.0.8)
Origin incomplete, metric 0, localpref 100, valid, internal, best (First path received)
Extended Community: RT:65035:1030
Remote label: 80
Last update: Tue Oct 19 13:45:32 202
Also we can verify how PE devices receives VPNv4 networks from the RRs and installing them to the specific customer VRFs:
“show bgp ipv4 vpn summary” for checking iBGP neighbors against route-reflector devices:
vyos@VyOS-PE1:~$ show bgp ipv4 vpn summary
BGP router identifier 10.0.0.7, local AS number 65001 vrf-id 0
BGP table version 0
RIB entries 9, using 1728 bytes of memory
Peers 2, using 43 KiB of memory
Peer groups 1, using 64 bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt
10.0.0.1 4 65001 8812 8794 0 0 0 01:18:42 8 2
10.0.0.2 4 65001 8800 8792 0 0 0 6d02h27m 8 2
- “show bgp vrf all” for checking all the prefix learning on BGP
within VRFs:
vyos@VyOS-PE1:~$ show bgp vrf all
Instance default:
No BGP prefixes displayed, 0 exist
Instance BLUE_SPOKE:
BGP table version is 8, local router ID is 10.50.50.1, vrf id 6
Default local pref 100, local AS 65001
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
* 10.50.50.0/24 0.0.0.0 0 32768 ?
*> 0.0.0.0 0 32768 i
*> 10.80.80.0/24 10.0.0.8@0< 0 100 0 i
* 10.0.0.8@0< 0 100 0 i
*> 10.0.0.80/32 10.50.50.2 0 0 65035 i
*> 10.0.0.100/32
10.0.0.8@0< 0 100 0 65035 ?
* 10.0.0.8@0< 0 100 0 65035 ?
- “show bgp vrf BLUE_SPOKE summary” for checking EBGP neighbor
information between PE and CE:
vyos@VyOS-PE1:~$ show bgp vrf BLUE_SPOKE summary
IPv4 Unicast Summary:
BGP router identifier 10.50.50.1, local AS number 65001 vrf-id 6
BGP table version 8
RIB entries 7, using 1344 bytes of memory
Peers 1, using 21 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt
10.50.50.2 4 65035 9019 9023 0 0 0 6d06h12m 1 4
Total number of neighbors 1
- “show ip route vrf BLUE_SPOKE” for viewing the RIB in our Spoke PE.
Using this command we are also able to check the transport and customer label (inner/outer) for Hub network prefix (10.0.0.100/32):
vyos@VyOS-PE1:~$ show ip route vrf BLUE_SPOKE
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
VRF BLUE_SPOKE:
K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 03w0d23h
C>* 10.50.50.0/24 is directly connected, eth3, 03w0d23h
B> 10.80.80.0/24 [200/0] via 10.0.0.8 (vrf default) (recursive), label 80, weight 1, 04:22:00
* via 172.16.90.1, eth0 (vrf default), label 24/80, weight 1, 04:22:00
B>* 10.0.0.80/32 [20/0] via 10.50.50.2, eth3, weight 1, 6d05h30m
B> 10.0.0.100/32 [200/0] via 10.0.0.8 (vrf default) (recursive), label 80, weight 1, 04:22:00
* via 172.16.90.1, eth0 (vrf default), label 24/80, weight 1, 04:22:00
- “show bgp ipv4 vpn x.x.x.x/32” for checking the best-path to the
specific VPNv4 destination including extended community and remotelabel information. This procedure is the same on all Spoke nodes:
vyos@VyOS-PE1:~$ show bgp ipv4 vpn 10.0.0.100/32
BGP routing table entry for 10.80.80.1:1011:10.0.0.100/32
not allocated
Paths: (2 available, best #1)
Not advertised to any peer
65035
10.0.0.8 from 10.0.0.1 (10.0.0.8)
Origin incomplete, metric 0, localpref 100, valid, internal, best (Neighbor IP)
Extended Community: RT:65035:1030
Originator: 10.0.0.8, Cluster list: 10.0.0.1
Remote label: 80
Last update: Tue Oct 19 13:45:26 2021
65035
10.0.0.8 from 10.0.0.2 (10.0.0.8)
Origin incomplete, metric 0, localpref 100, valid, internal
Extended Community: RT:65035:1030
Originator: 10.0.0.8, Cluster list: 10.0.0.1
Remote label: 80
Last update: Wed Oct 13 12:39:34 202
Now, let’s check routing information on out Hub PE:
- “show bgp ipv4 vpn summary” for checking iBGP neighbors again
VyOS-RR1/RR2
vyos@VyOS-PE2:~$ show bgp ipv4 vpn summary
BGP router identifier 10.0.0.8, local AS number 65001 vrf-id 0
BGP table version 0
RIB entries 9, using 1728 bytes of memory
Peers 2, using 43 KiB of memory
Peer groups 1, using 64 bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt
10.0.0.1 4 65001 15982 15949 0 0 0 05:41:28 6 4
10.0.0.2 4 65001 9060 9054 0 0 0 6d06h47m 6 4
Total number of neighbors
“show bgp vrf all” for checking all the prefixes learning on BGP
vyos@VyOS-PE2:~$ show bgp vrf all
Instance default:
No BGP prefixes displayed, 0 exist
Instance BLUE_HUB:
BGP table version is 50, local router ID is 10.80.80.1, vrf id 8
Default local pref 100, local AS 65001
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 10.50.50.0/24 10.0.0.7@0< 0 100 0 i
* 10.0.0.7@0< 0 100 0 i
*> 10.60.60.0/24 10.0.0.10@0< 0 100 0 i
* 10.0.0.10@0< 0 100 0 i
* 10.80.80.0/24 10.80.80.2 0 0 65035 ?
* 0.0.0.0 0 32768 i
*> 0.0.0.0 0 32768 ?
*> 10.110.110.0/24 172.16.80.2@9< 0 0 65050 i
*> 10.210.210.0/24 10.0.0.9@0< 0 100 0 65050 i
* 10.0.0.9@0< 0 100 0 65050 i
*> 10.0.0.80/32 10.0.0.7@0< 0 100 0 65035 i
* 10.0.0.7@0< 0 100 0 65035 i
*> 10.0.0.90/32 10.0.0.10@0< 0 100 0 65035 i
* 10.0.0.10@0< 0 100 0 65035 i
*> 10.0.0.100/32
10.80.80.2 0 0 65035 ?
*> 172.16.80.0/24 0.0.0.0@9< 0 32768 ?
0.0.0.0@9< 0 32768 i
*> 172.16.100.0/24 10.0.0.9@0< 0 100 0 i
* 10.0.0.9@0< 0 100 0 i
- “show bgp vrf BLUE_HUB summary” for checking EBGP neighbor
CE Hub device
vyos@VyOS-PE2:~$ show bgp vrf BLUE_HUB summary
IPv4 Unicast Summary:
BGP router identifier 10.80.80.1, local AS number 65001 vrf-id 8
BGP table version 50
RIB entries 19, using 3648 bytes of memory
Peers 1, using 21 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt
10.80.80.2 4 65035 15954 15972 0 0 0 01w4d01h 2 10
- “show ip route vrf BLUE_HUB” to view the RIB in our Hub PE.
With this command we are able to check the transport and customer label (inner/outer) for network spokes prefixes 10.0.0.80/32 - 10.0.0.90/32
vyos@VyOS-PE2:~$ show ip route vrf BLUE_HUB
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
VRF BLUE_HUB:
K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 01w4d01h
B> 10.50.50.0/24 [200/0] via 10.0.0.7 (vrf default) (recursive), label 144, weight 1, 05:53:15
* via 172.16.100.1, eth1 (vrf default), label 22/144, weight 1, 05:53:15
B> 10.60.60.0/24 [200/0] via 10.0.0.10 (vrf default) (recursive), label 144, weight 1, 05:53:15
* via 172.16.110.1, eth0 (vrf default), label 23/144, weight 1, 05:53:15
C>* 10.80.80.0/24 is directly connected, eth3, 01w4d01h
B>* 10.110.110.0/24 [200/0] via 172.16.80.2, eth2 (vrf GREEN), weight 1, 01w4d01h
B> 10.210.210.0/24 [200/0] via 10.0.0.9 (vrf default) (recursive), label 144, weight 1, 05:53:15
* via 172.16.100.1, eth1 (vrf default), label 18/144, weight 1, 05:53:15
* via 172.16.110.1, eth0 (vrf default), label 22/144, weight 1, 05:53:15
B> 10.0.0.80/32 [200/0] via 10.0.0.7 (vrf default) (recursive), label 144, weight 1, 05:53:15
* via 172.16.100.1, eth1 (vrf default), label 22/144, weight 1, 05:53:15
B> 10.0.0.90/32 [200/0] via 10.0.0.10 (vrf default) (recursive), label 144, weight 1, 05:53:15
* via 172.16.110.1, eth0 (vrf default), label 23/144, weight 1, 05:53:15
B>* 10.0.0.100/32 [20/0] via 10.80.80.2, eth3, weight 1, 01w4d01h
B>* 172.16.80.0/24 [200/0] is directly connected, eth2 (vrf GREEN), weight 1, 01w4d01h
B> 172.16.100.0/24 [200/0] via 10.0.0.9 (vrf default) (recursive), label 144, weight 1, 05:53:15
* via 172.16.100.1, eth1 (vrf default), label 18/144, weight 1, 05:53:15
* via 172.16.110.1, eth0 (vrf default), label 22/144, weight 1, 05:53:15
- “show bgp ipv4 vpn x.x.x.x/32” for checking best-path,
extended community and remote label of specific destination
vyos@VyOS-PE2:~$ show bgp ipv4 vpn 10.0.0.80/32
BGP routing table entry for 10.50.50.1:1011:10.0.0.80/32
not allocated
Paths: (2 available, best #1)
Not advertised to any peer
65035
10.0.0.7 from 10.0.0.1 (10.0.0.7)
Origin IGP, metric 0, localpref 100, valid, internal, best (Neighbor IP)
Extended Community: RT:65035:1011
Originator: 10.0.0.7, Cluster list: 10.0.0.1
Remote label: 144
Last update: Tue Oct 19 13:45:30 2021
65035
10.0.0.7 from 10.0.0.2 (10.0.0.7)
Origin IGP, metric 0, localpref 100, valid, internal
Extended Community: RT:65035:1011
Originator: 10.0.0.7, Cluster list: 10.0.0.1
Remote label: 144
Last update: Wed Oct 13 12:39:37 2021
vyos@VyOS-PE2:~$ show bgp ipv4 vpn 10.0.0.90/32
BGP routing table entry for 10.60.60.1:1011:10.0.0.90/32
not allocated
Paths: (2 available, best #1)
Not advertised to any peer
65035
10.0.0.10 from 10.0.0.1 (10.0.0.10)
Origin IGP, metric 0, localpref 100, valid, internal, best (Neighbor IP)
Extended Community: RT:65035:1011
Originator: 10.0.0.10, Cluster list: 10.0.0.1
Remote label: 144
Last update: Tue Oct 19 13:45:30 2021
65035
10.0.0.10 from 10.0.0.2 (10.0.0.10)
Origin IGP, metric 0, localpref 100, valid, internal
Extended Community: RT:65035:1011
Originator: 10.0.0.10, Cluster list: 10.0.0.1
Remote label: 144
Last update: Wed Oct 13 12:45:44 2021
Finally, let’s check the reachability between CEs:
VyOS-CE1-SPOKE —–> VyOS-CE-HUB
# check rib
vyos@VyOS-CE1-SPOKE:~$ show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
B 10.50.50.0/24 [20/0] via 10.50.50.1 inactive, weight 1, 6d07h53m
C>* 10.50.50.0/24 is directly connected, eth0, 09w0d00h
B>* 10.80.80.0/24 [20/0] via 10.50.50.1, eth0, weight 1, 6d07h53m
C>* 10.0.0.80/32 is directly connected, dum20, 09w0d00h
B>* 10.0.0.100/32 [20/0] via 10.50.50.1, eth0, weight 1, 6d07h53m
# check icmp
vyos@VyOS-CE1-SPOKE:~$ ping 10.0.0.100 interface 10.0.0.80
PING 10.0.0.100 (10.0.0.100) from 10.0.0.80 : 56(84) bytes of data.
64 bytes from 10.0.0.100: icmp_seq=1 ttl=62 time=6.52 ms
64 bytes from 10.0.0.100: icmp_seq=2 ttl=62 time=4.13 ms
64 bytes from 10.0.0.100: icmp_seq=3 ttl=62 time=4.04 ms
64 bytes from 10.0.0.100: icmp_seq=4 ttl=62 time=4.03 ms
^C
--- 10.0.0.100 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 8ms
rtt min/avg/max/mdev = 4.030/4.680/6.518/1.064 ms
# check network path
vyos@VyOS-CE1-SPOKE:~$ traceroute 10.0.0.100
traceroute to 10.0.0.100 (10.0.0.100), 30 hops max, 60 byte packets
1 10.50.50.1 (10.50.50.1) 1.041 ms 1.252 ms 1.835 ms
2 * * *
3 10.0.0.100 (10.0.0.100) 9.225 ms 9.159 ms 9.121 m
VyOS-CE-HUB ——-> VyOS-CE1-SPOKE
VyOS-CE-HUB ——-> VyOS-CE2-SPOKE
# check rib
vyos@VyOS-CE-HUB:~$ show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
B>* 10.50.50.0/24 [20/0] via 10.80.80.1, eth0, weight 1, 6d08h04m
B>* 10.60.60.0/24 [20/0] via 10.80.80.1, eth0, weight 1, 6d08h35m
C>* 10.80.80.0/24 is directly connected, eth0, 01w6d07h
B>* 10.110.110.0/24 [20/0] via 10.80.80.1, eth0, weight 1, 01w4d02h
B>* 10.210.210.0/24 [20/0] via 10.80.80.1, eth0, weight 1, 6d08h35m
B>* 10.0.0.80/32 [20/0] via 10.80.80.1, eth0, weight 1, 6d08h04m
B>* 10.0.0.90/32 [20/0] via 10.80.80.1, eth0, weight 1, 6d08h35m
C>* 10.0.0.100/32 is directly connected, dum20, 01w6d07h
B>* 172.16.80.0/24 [20/0] via 10.80.80.1, eth0, weight 1, 01w4d02h
B>* 172.16.100.0/24 [20/0] via 10.80.80.1, eth0, weight 1, 6d08h35m
# check icmp
vyos@VyOS-CE-HUB:~$ ping 10.0.0.80 interface 10.0.0.100 c 4
PING 10.0.0.80 (10.0.0.80) from 10.0.0.100 : 56(84) bytes of data.
64 bytes from 10.0.0.80: icmp_seq=1 ttl=62 time=3.31 ms
64 bytes from 10.0.0.80: icmp_seq=2 ttl=62 time=4.23 ms
64 bytes from 10.0.0.80: icmp_seq=3 ttl=62 time=3.89 ms
64 bytes from 10.0.0.80: icmp_seq=4 ttl=62 time=3.22 ms
--- 10.0.0.80 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 9ms
rtt min/avg/max/mdev = 3.218/3.661/4.226/0.421 ms
vyos@VyOS-CE-HUB:~$ ping 10.0.0.90 interface 10.0.0.100 c 4
PING 10.0.0.90 (10.0.0.90) from 10.0.0.100 : 56(84) bytes of data.
64 bytes from 10.0.0.90: icmp_seq=1 ttl=62 time=7.46 ms
64 bytes from 10.0.0.90: icmp_seq=2 ttl=62 time=4.43 ms
64 bytes from 10.0.0.90: icmp_seq=3 ttl=62 time=4.60 ms
^C
--- 10.0.0.90 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 6ms
rtt min/avg/max/mdev = 4.430/5.498/7.463/1.391 ms
# check network path
vyos@VyOS-CE-HUB:~$ traceroute 10.0.0.80
traceroute to 10.0.0.80 (10.0.0.80), 30 hops max, 60 byte packets
1 10.80.80.1 (10.80.80.1) 1.563 ms 1.341 ms 1.075 ms
2 * * *
3 10.0.0.80 (10.0.0.80) 8.125 ms 8.019 ms 7.781 ms
vyos@VyOS-CE-HUB:~$ traceroute 10.0.0.90
traceroute to 10.0.0.90 (10.0.0.90), 30 hops max, 60 byte packets
1 10.80.80.1 (10.80.80.1) 1.305 ms 1.137 ms 1.097 ms
2 * * *
3 * * *
4 10.0.0.90 (10.0.0.90) 9.358 ms 9.325 ms 9.292 ms
VyOS-CE2-SPOKE ——-> VyOS-CE-HUB
# check rib
vyos@rt-ce2-SPOKE:~$ show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
B 10.60.60.0/24 [20/0] via 10.60.60.1 inactive, weight 1, 02w6d00h
C>* 10.60.60.0/24 is directly connected, eth0, 02w6d00h
B>* 10.80.80.0/24 [20/0] via 10.60.60.1, eth0, weight 1, 6d08h46m
C>* 10.0.0.90/32 is directly connected, dum20, 02w6d00h
B>* 10.0.0.100/32 [20/0] via 10.60.60.1, eth0, weight 1, 6d08h46m
# check icmp
vyos@rt-ce2-SPOKE:~$ ping 10.0.0.100 interface 10.0.0.90 c 4
PING 10.0.0.100 (10.0.0.100) from 10.0.0.90 : 56(84) bytes of data.
64 bytes from 10.0.0.100: icmp_seq=1 ttl=62 time=4.97 ms
64 bytes from 10.0.0.100: icmp_seq=2 ttl=62 time=4.45 ms
64 bytes from 10.0.0.100: icmp_seq=3 ttl=62 time=4.20 ms
64 bytes from 10.0.0.100: icmp_seq=4 ttl=62 time=4.29 ms
--- 10.0.0.100 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 9ms
rtt min/avg/max/mdev = 4.201/4.476/4.971/0.309 ms
# check network path
vyos@rt-ce2-SPOKE:~$ traceroute 10.0.0.100
traceroute to 10.0.0.100 (10.0.0.100), 30 hops max, 60 byte packets
1 10.60.60.1 (10.60.60.1) 1.343 ms 1.190 ms 1.152 ms
2 * * *
3 * * *
4 10.0.0.100 (10.0.0.100) 7.504 ms 7.480 ms 7.488 ms
Note: At the moment, trace mpls doesn’t show labels/paths. So we’ll see * * * for the transit routers of the mpls backbone.