mpls: simple and advanced inter-vrf routing/leaking

Hello everybody!
Today I will spend some time in demonstrating how to route between separate VRF instances on a local router.

Special prerequisites:
– Understanding of VRF lite

Topology:

Our topology exists of three routers. Quite simple for the beginning but I think with a little bit of configuring we can get this scenario a little more complex ;).

Given configuration:
– routers R2 and R3 are connected via a FastEthernet link to R1
– all routers have loopback addresses configured

Challenge #1:
– R2 and R3 shall use R1 as the default router (to reach the Loopback of the router on the other end)

Challenge #2:
– put the connection to both R2 and R3 in different VRFs on R1 BUT they still should be able to reach each other (first solution with static routing, second one with dynamic routing but everything is done on R1)

So lets begin!

Challenge #1:
First of all we need to (as we are using static routing in our case) configure R1 to be able to reach the loopbacks of R2 and R3.

R1(config)#ip route 2.2.2.2 255.255.255.255 172.16.21.2
R1(config)#ip route 3.3.3.3 255.255.255.255 172.16.31.2

Of course we need to configure the reverse route on R2 and R3 in order to establish bidirectional connectivity. As the task requires to configure a default route we will configure such a route.

R2(config)#ip route 0.0.0.0 0.0.0.0 172.16.21.1

R3(config)#ip route 0.0.0.0 0.0.0.0 172.16.31.1

Lets test the ICMP reachability.

R1#ping 2.2.2.2 so lo0

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 2.2.2.2, timeout is 2 seconds:
Packet sent with a source address of 1.1.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 20/45/88 ms
R1#ping 3.3.3.3 so lo0

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 3.3.3.3, timeout is 2 seconds:
Packet sent with a source address of 1.1.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 8/50/100 ms
R1#

Works prefectly. Lets now check end to end connectivity between R2 and R3

R2#ping 3.3.3.3 so lo0

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 3.3.3.3, timeout is 2 seconds:
Packet sent with a source address of 2.2.2.2
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 56/94/128 ms

Also fine! Challenge one is completed.

Challenge #2:
We will now create two new VRFs (VRF_R2 and VRF_R3) on R1 in order to put the connections into different VRFs.



R1(config)#ip vrf VRF_R2
R1(config-vrf)#rd 2:2
R1(config-vrf)#route-target both 2:2
R1(config-vrf)#ip vrf VRF_R3
R1(config-vrf)#rd 3:3
R1(config-vrf)#route-target both 3:3

Lets put the interfaces fa0/0 and fa0/1 in the corresponding VRFs.

R1(config-vrf)#int fa0/0
R1(config-if)#ip vrf for VRF_R2
% Interface FastEthernet0/0 IP address 172.16.21.1 removed due to enabling VRF VRF_R2
R1(config-if)#ip address 172.16.21.1 255.255.255.252
R1(config-if)#int fa0/1
R1(config-if)#ip vrf for VRF_R3
% Interface FastEthernet0/1 IP address 172.16.31.1 removed due to enabling VRF VRF_R3
R1(config-if)#ip address 172.16.31.1 255.255.255.252

Never forget to verify the config!

R1#sh ip vrf
Name Default RD Interfaces
VRF_R2 2:2 Fa0/0
VRF_R3 3:3 Fa0/1

Looks quite good here! Then lets try the connectivity from the VRFs of R1 to the Loopback of R2 and R3.

R1#ping vrf VRF_R2 2.2.2.2

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 2.2.2.2, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

Strange…does not work. But why? Lets check the routing table of the VRFs.

R1#sh ip route vrf VRF_R2

Routing Table: VRF_R2
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

172.16.0.0/30 is subnetted, 1 subnets
C 172.16.21.0 is directly connected, FastEthernet0/0
R1#sh ip route vrf VRF_R3

Routing Table: VRF_R3
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

172.16.0.0/30 is subnetted, 1 subnets
C 172.16.31.0 is directly connected, FastEthernet0/1

No static routes?! Well that makes sense as we configured the static routes earlier for the global routing table and not for the specific VRFs. We have to correct that.

R1(config)#no ip route 2.2.2.2 255.255.255.255 172.16.21.2
R1(config)#no ip route 3.3.3.3 255.255.255.255 172.16.31.2
R1(config)#ip route vrf VRF_R2 2.2.2.2 255.255.255.255 172.16.21.2
R1(config)#ip route vrf VRF_R3 3.3.3.3 255.255.255.255 172.16.31.2

Lets have a look at the RTs now!

R1#sh ip route vrf VRF_R2

Routing Table: VRF_R2
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

2.0.0.0/32 is subnetted, 1 subnets
S 2.2.2.2 [1/0] via 172.16.21.2
172.16.0.0/30 is subnetted, 1 subnets
C 172.16.21.0 is directly connected, FastEthernet0/0
R1#sh ip route vrf VRF_R3

Routing Table: VRF_R3
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

3.0.0.0/32 is subnetted, 1 subnets
S 3.3.3.3 [1/0] via 172.16.31.2
172.16.0.0/30 is subnetted, 1 subnets
C 172.16.31.0 is directly connected, FastEthernet0/1

Looks better now. And the following ICMP check too!

R1#ping vrf VRF_R2 2.2.2.2

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 2.2.2.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 12/43/72 ms
R1#ping vrf VRF_R3 3.3.3.3

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 3.3.3.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 12/33/72 ms

Lets now check end to end connectivity between R2 and R3.

R2#ping 3.3.3.3 so lo0

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 3.3.3.3, timeout is 2 seconds:
Packet sent with a source address of 2.2.2.2
U.U.U
Success rate is 0 percent (0/5)

Interesting…does not work. But instead of just dotting the ping out…the router gives us a “U” back. This means that the next-hop or what hop ever sends an ICMP unreachable back informing us that the router we have sent the packet to doesnt have a route for the destination prefix. This is quite logical here because R1 has no route for 3.3.3.3/32 in its routing table for VRF_R2. So we need to tune this a little and need to configure a leak between those VRFs. We will generate a static route in a VRF that is pointing to a different VRF. Sounds quite strange but it works, as the router connects the VRFs we are giving him by one -> the name of the VRF we are adding the static route and second -> the VRF the gateway address is participating in.

R1(config)#ip route vrf VRF_R3 2.2.2.2 255.255.255.255 fa0/0 172.16.21.2
R1(config)#ip route vrf VRF_R2 3.3.3.3 255.255.255.255 fa0/1 172.16.31.2

Lets check the tables. Take a look at the routes. I specified an interface AND the gateway ip address. This is necessary because if you only specify the gateway the router looks for this address in the specific vrf, doesnt find it and so does not create the route. When you specify also the interface the router now knows that you wanna route-leak as the interface is in a different vrf and then searches or the following ip address in the vrf of the destination interface.

R1#sh ip route vrf VRF_R2

Routing Table: VRF_R2
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

2.0.0.0/32 is subnetted, 1 subnets
S 2.2.2.2 [1/0] via 172.16.21.2
3.0.0.0/32 is subnetted, 1 subnets
S 3.3.3.3 [1/0] via 172.16.31.2, FastEthernet0/1
172.16.0.0/30 is subnetted, 1 subnets
C 172.16.21.0 is directly connected, FastEthernet0/0
R1#sh ip route vrf VRF_R3

Routing Table: VRF_R3
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

2.0.0.0/32 is subnetted, 1 subnets
S 2.2.2.2 [1/0] via 172.16.21.2, FastEthernet0/0
3.0.0.0/32 is subnetted, 1 subnets
S 3.3.3.3 [1/0] via 172.16.31.2
172.16.0.0/30 is subnetted, 1 subnets
C 172.16.31.0 is directly connected, FastEthernet0/1

There we go. Looks good. Lets now check the end-to-end reachbility again.

R2#ping 3.3.3.3 so lo0

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 3.3.3.3, timeout is 2 seconds:
Packet sent with a source address of 2.2.2.2
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 48/71/104 ms

We made it!

Next step we will do is to…well…solve this a little more dynamic. Imagine that there are lots of disjunct networks you want to connect. Then you are running into the problem that you need a new keyboard after some times because you typed millions of static routes :).
Or protocol of choice in that case is BGP, better MP-BGP. Thats because MP-BGP is capable of handling VRF and so VPN information protocol wide, not only local.

First of all we will remove the static inter-VRF routes from R1.

R1(config)#no ip route vrf VRF_R2 3.3.3.3 255.255.255.255 FastEthernet0/1 172.16.31.2
R1(config)#no ip route vrf VRF_R3 2.2.2.2 255.255.255.255 FastEthernet0/0 172.16.21.2

Lets check if the end-to-end connectivity is broken.

R2#ping 3.3.3.3 so lo0

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 3.3.3.3, timeout is 2 seconds:
Packet sent with a source address of 2.2.2.2
U.U.U
Success rate is 0 percent (0/5)

Ok so far so good. Now we are going to activate MP-BGP by enabling BGP on the router.

R1(config)#router bgp 65000
R1(config-router)#no auto
R1(config-router)#no sync

MP-BGP has the capability of handling different VRFs by using so called address-families. We will now create an address family for each VRF.

R1(config-router)#address-family ipv4 vrf VRF_R2
R1(config-router-af)#address-family ipv4 vrf VRF_R3

Okay. Well what we now have is to torn it down…one BGP instance for the same AS in different VRFs on the router. You recognize its getting a little more complex now :). Every VRF uses (when its transported or better advertised via BGP) route-targets. Those little route-targets are used to tag a bgp routes extended community string. Let me show you this a little bit more on the router. Its easier to explain here.

R1#sh run | sec vrf
ip vrf VRF_R2
rd 2:2
route-target export 2:2
route-target import 2:2
ip vrf VRF_R3
rd 3:3
route-target export 3:3
route-target import 3:3

That means when routes (which we created statically) are sent into the BGP process they are tagged with the extended community 2:2 or 3:3…this is what route-target export means. Route-Target import means that routes coming from BGP peers or locally injected BGP routes (this is what we will abuse) are imported into the VRF routing table/routing processes.
what we will now do…well we want the static route of VRF_R2 in the RT of VRF_R3 and vice versa, but without a static route. Well we first need to import the routes of the opposite VRF.

R1(config)#ip vrf VRF_R2
R1(config-vrf)#route-target import 3:3
R1(config-vrf)#ip vrf VRF_R3
R1(config-vrf)#route-target import 2:2

Okay lets have a look.

R1#sh ip route vrf VRF_R2

Routing Table: VRF_R2
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

2.0.0.0/32 is subnetted, 1 subnets
S 2.2.2.2 [1/0] via 172.16.21.2
172.16.0.0/30 is subnetted, 1 subnets
C 172.16.21.0 is directly connected, FastEthernet0/0

Hmm nothing there but why? Remember that we wanna use BGP as the leaking method here. So we first need to handover those static routes into the BGP process, so that BGP can handover the routes between the two VRFs.

R1(config)#router bgp 65000
R1(config-router)#address-family ipv4 vrf VRF_R2
R1(config-router-af)#redistribute static
R1(config-router-af)#redistribute connected
R1(config-router-af)#address-family ipv4 vrf VRF_R3
R1(config-router-af)#redistribute static
R1(config-router-af)#redistribute connected

Lets check if BGP has the routes in its routing-process (by the way I also redistributed the connected interfaces to get full convergence of the link addresses as well).

R1#sh ip bgp vpnv4 all
BGP table version is 13, local router ID is 1.1.1.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 2:2 (default for vrf VRF_R2)
*> 2.2.2.2/32 172.16.21.2 0 32768 ?
*> 3.3.3.3/32 172.16.31.2 0 32768 ?
*> 172.16.21.0/30 0.0.0.0 0 32768 ?
*> 172.16.31.0/30 0.0.0.0 0 32768 ?
Route Distinguisher: 3:3 (default for vrf VRF_R3)
*> 2.2.2.2/32 172.16.21.2 0 32768 ?
*> 3.3.3.3/32 172.16.31.2 0 32768 ?
*> 172.16.21.0/30 0.0.0.0 0 32768 ?
*> 172.16.31.0/30 0.0.0.0 0 32768 ?

Looks quite good, lets check the RTs.

R1#sh ip route vrf VRF_R2

Routing Table: VRF_R2
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

2.0.0.0/32 is subnetted, 1 subnets
S 2.2.2.2 [1/0] via 172.16.21.2
3.0.0.0/32 is subnetted, 1 subnets
B 3.3.3.3 [20/0] via 172.16.31.2 (VRF_R3), 00:01:27
172.16.0.0/30 is subnetted, 2 subnets
B 172.16.31.0 is directly connected, 00:01:27, FastEthernet0/1
C 172.16.21.0 is directly connected, FastEthernet0/0
R1#sh ip route vrf VRF_R3

Routing Table: VRF_R3
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

2.0.0.0/32 is subnetted, 1 subnets
B 2.2.2.2 [20/0] via 172.16.21.2 (VRF_R2), 00:01:44
3.0.0.0/32 is subnetted, 1 subnets
S 3.3.3.3 [1/0] via 172.16.31.2
172.16.0.0/30 is subnetted, 2 subnets
C 172.16.31.0 is directly connected, FastEthernet0/1
B 172.16.21.0 is directly connected, 00:01:31, FastEthernet0/0

Beautiful…BGP routes :). Now if everything is configured correctly the connection should work.

R2#ping 3.3.3.3 so lo0

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 3.3.3.3, timeout is 2 seconds:
Packet sent with a source address of 2.2.2.2
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 28/73/116 ms

Indeed it does!
I hope you enjoyed it and as always you are welcome to comment on that!

Regards!
Markus

Advertisements

About markus.wirth

Living near Limburg in Germany, working as a Network Engineer around Frankfurt am Main.
This entry was posted in MPLS, Uncategorized and tagged , , , , , , , , , , , , , , , , , , . Bookmark the permalink.

8 Responses to mpls: simple and advanced inter-vrf routing/leaking

  1. Simon Brooks says:

    Excellent explanantion mate, thanks.

  2. juan says:

    why do u do ping with the source command?

  3. Dillip says:

    Excellent Thanks for explaining.

  4. Details says:

    For the network between R3 and R1, your diagrams show
    172.16.32.x
    But your commands and commentary show
    172.16.31.x

  5. Eagerness says:

    Great Explanation

  6. Per G says:

    I must say that this is the best explanation i have come accross on internet. Even Cisco blogs dont explain it good enough. Not even in the book. thanks Mate 🙂

  7. sparkmiracle says:

    I replicated the above lab beautifully, end to end connectivity works, but you can ping Loopback 1.1.1.1 from R2 or R3 as well as 2.2.2.2 form R1 Lo 1.1.1.1 or 3.3.3.3 from lo 1.1.1.1
    are you able to supply the additional configs to complete the lab please. In the beginning it was possible pinging from R1 source Lo 1.1.1.1 but after MP-BGP it wasn’t. Thanks

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s