Something that took me ages to grasp was the difference between BGP route-distinguishers and route-targets in MPLS VPNs. And even when I thought I understood, I still didn’t see why we needed both. Why not just have the one number to fit both tasks?
When I finally understood it, it all clicked together perfectly, like the expensive Lego I stole from the kids next door. But getting to that stage involved reading about 30 different blog posts, forum questions and Wikipedia articles – because it turns out that the answer is complicated. So, let me try to save you time. Let’s talk about it all in detail, with some examples.
This post assumes you understand the basic concepts of BGP, address families, and MPLS VPNs (what some people in networking call “The Sexy Trinity”.) (No-one calls it that.)
Here’s the topology we’ll use. Routers 1 and 4 are PE (Provider Edge) routers – the routers at the other end of our customer WAN links. Routers 2 and 3 are our P (Provider) routers, acting as BGP route reflectors. So:
— Router 1 just peers with 2 and 3
— Router 4 just peers with 2 and 3
FUN BONUS FACT: The VRFs themselves aren’t actually configured on Routers 2 and 3. As P routers, they don’t need to know the exact VRFs in our network; they just need to know how to pass the information on to the PE routers.
WHAT IS A ROUTE TARGET?
So, you’ll know already that MPLS VPNs allow routers to hold multiple routing tables, called VRFs. This allows ISPs to run multiple private networks for different customers, so customer A and customer B can use the same private IP address space. It doesn’t matter if the subnets overlap; each customer has their own unique routing table.
Now, if customer A decided to make a new network, 10.0.0.0/24, and Router 1 advertises this to Router 4, how does Router 4 if it’s meant for VRF CUSTOMER_A or CUSTOMER_B? That’s where route targets come in. In short, a route target is some extra information, added to an advertised prefix, to tell the receiving router which VRF to put it in.
Here’s how it works: You’ll know already that when BGP advertises a prefix, it adds on some “path attributes” – things like the AS_PATH, or the MED. Well, BGP’s Multi-Protocol Extensions give us even more attributes – and one of them is the Route Target.
They take the format of two numbers, from 0 to 65535. Here’s an example VRF (ignore the line that says “rd 64512:500” just for now – we’ll come back to it):
PROVIDER EDGE ROUTER 1: ip vrf CUSTOMER_A rd 64512:500 route-target export 64512:900 route-target import 64512:100
Look at the export line. What this config says is: when BGP advertises (or “exports”) a prefix, Router 1 will add a route-target to the advertisement of 64512:900. Why that number? Well, that’s completely up to you. It’s common to put your autonomous system in the first number, but the whole thing can be whatever you want. Choose something meaningful to you.
Now let’s look at the VRF configuration on the router (Router 4) that will receive this prefix:
PROVIDER EDGE ROUTER 4: ip vrf CUSTOMER_A rd 64512:100 route-target export 64512:100 route-target import 64512:900
This time, take a look at the import route-target. The numbers match the export route-target on Router 1! This is how Router 4 knows which VRF to put it into: it imports any prefixes with a route-target of 64512:900 into the CUSTOMER_A routing table.
HANG ON: WHY NOT JUST USE THE VRF IN THE ADVERTISEMENT?
Fair point: why do we have these route-target numbers? Why can’t BGP just say “Add this route to VRF CUSTOMER_A?
The reason is that the VRF label CUSTOMER_A is only locally significant – in other words, the VRF name only matters on the router itself. There’s nothing stopping you from giving a customer’s VRF a totally different name on each router in your network! As such, sending the VRF name wouldn’t be useful.
This is why we use the route-target number system – which actually allows a lot of flexibility. For example, there’s nothing stopping me from taking the advertisement from one router and adding it into multiple VRFs on another router! You might do this for management purposes, to allow yourself to manage all your customer equipment from one location, regardless of what VRF they’re in.
Let’s test this.
You’ll remember a moment ago that we configured Router 4 to import any routes it receives with a route-target of 64512:900, into VRF CUSTOMER_A.
Now, let’s make a test VRF on Router 4, and configure this VRF to *also* import these routes. Le’s add this configuration to router 4 (again, ignore the line with rd in it for now):
ip vrf TEST_CUSTOMER rd 64512:666 route-target import 64512:900
After a minute, let’s look at the routing table:
Router4#show ip route vrf TEST_CUSTOMER Routing Table: TEST_CUSTOMER [ edited for brevity ] Gateway of last resort is not set 10.0.0.0/24 is subnetted, 1 subnets B 10.1.1.0 [200/0] via 192.168.1.251, 00:01:46
It worked! The route in Router 1’s CUSTOMER_A VRF is now in two VRFs on Router 4.
OKAY OKAY, SO WHAT DOES A ROUTE DISTINGUISHER DO THEN?
Another fair question! Right now, it seems like we have all we need for a working network:
— We have VRFs on our Provider Edge routers, that make each routing table unique on that router.
— We also have a way of advertising our customer’s prefixes between our routers, putting them into the customer’s VRF on another router – even when the VRF names don’t match.
— We even have a way of importing the prefixes into more than one VRF!
What more could you want?
Well, listen here, Jimmy: this is the 21st century. People always want more. They’re never satisfied. Give them a Mega Drive, and they’ll want a SNES. Give them an ice cream, and they’ll want a pension. Why were route distinguishers made? To satisfy our opulent western greed. We should all hang our heads in shame.
Confession time: everything I just said then was a lie. Because the truth is a bit confusing – so I thought I’d “put a smile on your face” before telling you the actual answer.
You see, to our advanced human brains, everything here is perfect. But there’s a quirk of BGP that means this won’t quite work.
Imagine that Router 4 receives a BGP update with a new route: 10.0.0.0/24. Let’s say that it had a route-target of 64512:900 – which you and I would say as belonging to the VRF CUSTOMER_A.
Now imagine Router 4 receives another update. This one also mentions a prefix of 10.0.0.0/24, but this one has a different route-target – let’s say 64512:800. The prefix is the same, but the route-targets are totally different. This should be enough for the router to tell these two prefixes, apart, right?
Well, that’s where you’re wrong, Jimmy. Damn wrong. You see, although you and I can see that these prefixes are different, the router can’t. Why? Because we’re smarter than routers, Jimmy.
Okay, the real reason is because the router actually needs the prefix itself to be unique. It doesn’t matter what’s in the route-target – if the prefix itself isn’t unique, BGP gets confused.
Remember, the route-target is just an attribute, like the AS_PATH, or the MED. If the router receives an advertisement for 10.0.0.0/24 and is told to put it in one VRF, that’s fine. But if it then receives an advertisement for 10.0.0.0/24 for a different VRF, by default it will over-write the first advertisement!
Here’s a screenshot of a BGP update message, captured by Wireshark (you can download the message here), going from Router 1 to Router 4. You’ll see I’ve highlighted the Route Target – it’s just underneath the LOCAL_PREF.
Then, at the bottom, you’ll see the actual NLRI (Network Layer Reachability Information). This is the only thing that matters for making the prefix unique. And, look what we see: the route distinguisher!
OKAY, I’M READY TO LEARN: WHAT IS A ROUTE DISTINGUISHER?
Here’s the concept: each VRF on a router is configured with its own unique route-distinguisher – and it’s this RD number that makes its routes unique.
Now, this is worth emphasising, because it confuses a lot of people: when a router receives a route, it doesn’t use the route distinguisher to put the route into a particular VRF. That’s the job of the route target. Literally the only job of the route distinguisher is to make the route globally unique – or, in other words, to distinguish the route.
The route distinguisher is added to the start of the IP address being advertised, which makes the prefix globally unique. Let’s see what it looks like, by doing a show ip bgp vpnv4 vrf CUSTOMER_A 10.2.2.0 on Router 1, to see if it’s learning the 10.2.2.0/24 network that lives on Router 4 – and what that advertisement looks like. Check out the bit I’ve highlighted:
Yep! It’s there! We now have a unique prefix, that won’t confuse BGP.
Now, let me tell you something that might blow your mind: the route distinguisher numbers don’t even have to match between routers! Remember, all the RD does is make the prefix unique. In fact, it can be a good idea to have totally different RDs for each VRF on each router – it means that you can quickly identify exactly which router the prefix originated from.
You probably noticed that the route distinguisher has exactly the same format as a route-target. No wonder people get so confused about it! But now you know what RTs and RDs do, you can probably see the advantage of having them use the same numbering system. For example, in a small network you may choose to use the same numbers for your RD, your route-target import, and your route-target export. In a large network, you may choose unique numbers for all of these things, to help you know exactly what goes where. It’s entirely up to you. Isn’t freedom great? Answer: no, it is a curse.
So, that’s it! I hope this was useful to you. If it was, please think about making a post on Twitter, Facebook or LinkedIn to share it. The more people that read this blog, the more worthwhile it is for me to make even more posts.