ROUTE DISTINGUISHERS vs ROUTE TARGETS: WHAT’S THE DIFFERENCE? (JUNOS FOR CISCO IOS ENGINEERS)

(This is actually a complete re-write of my first ever blog post. I’ve expanded it to include both Cisco IOS and Juniper Junos, and in doing so I’ve accidentally made it a part of my Junos for Cisco IOS Engineers series! If you know Cisco already and you’d like to know how Junos works, give that link a click to find out more about my fun new series!)

~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~

When Bill Gates invented Micro-Soft in the year 1753, he had one simple goal: to give every single country their own website. What a guy!

But the internet isn’t just about websites. You probably know that many organisations have a requirement for devices at their various offices, shops, factories, casinos and slaughterhouses to be able to talk to each other, even though they’re in physically different locations, and even though the sites are using exclusively private IP addresses. In such situations a VPN is required, and the type of VPN we’re interested in today is MPLS Layer 3 VPNs.

If you’re just starting to learn about these L3VPNs then you might have noticed a thing in the configuration called a route target, and also something called a route distinguisher. They look almost identical, right? In fact, sometimes they actually are identical. So what’s the difference? Why do we need both?

I struggled with that concept at first, too. But when I finally understood it, it all clicked together perfectly, like the expensive Lego I stole last week from the kids next door. My need is greater than theirs. But forget I said that: today we’ll learn the difference between the two.

In this blog post I’m going to be using both Cisco IOS and Juniper configuration, with a big focus on Juniper. Why? Because, although I bet lots of readers of this post are studying for Cisco certs, I want to give you a taste of how Junos does it – and in doing so, hopefully I’ll convince you that the Juniper config is a little bit more readable and understandable. And when I say “a bit”, I do of course mean “a LOT”.

Not all of you have time to read a post this long – so if you’re just after a quick answer, this next section is for you.

 

THE SHORT ANSWER

Route targets are used to indicate which VRFs a VPN prefix should be imported into.

In fact, when a PE router exports a VPN prefix, we can actually tag it with multiple route-targets, which allows us to import that prefix into multiple routing tables around our ISP network!

By contrast, a route distinguisher serves only one purposes: to make the prefix unique. This is important when two VPN customers connected to the same PE router might be using the exact same private IP range.

It might seem odd that the target isn’t enough to make the prefix unique. The reason is basically to do with the way that BGP was designed. The “original” BGP would actually just overwrite one customer’s prefix with another customer’s prefix, because as far as BGP is concerned, they’re the exact same address. For all the receiver knows, the route-target could have just changed, and the 2nd prefix is just a newer version of the 1st prefix.

By adding a unique route-distinguisher to each advertisement, and by adding that route-distinguisher to the start of the prefix, two VRFs can advertise the same prefix, and the receiving routers don’t override one prefix with the other. For example, instead of advertising just 192.168.10.0/24, a router might instead advertise something like “64512:900:192.168.10.0/24”. In this example, “64512:900” might be the route-distinguisher assigned to a particular customer. The route distinguisher literally distinguishes between two of the “same” prefix belonging to different VPN customers, and therefore makes it clear that ultimately they’re different prefixes.

What the route distinguisher does NOT do is tell the receiving router which VRF the prefix is a part of. That is exclusively the job of the route target.

There. If you’re after a quick answer, you can stop reading now.

But if you’re new to MPLS VPN studies, and you want to truly understand the moving parts, then read on.

 

LET’S LOOK AT OUR TOPOLOGY & DEFINE SOME TERMS

Click this pic to make it bigger!

Hey look: here’s a network diagram for our pretend Internet Service Provider!

Notice that there’s three PE routers. PE stands for Provider Edge, and it’s the name we give to the routers in the ISP’s network that host our customer WAN links.

Our ISP has only two customers, and therefore faces guaranteed economic ruin. Hooray!!

Notice that I’ve named our customers CUSTOMER_A and CUSTOMER_B. Pretty original naming convention! You’re welcome. CUSTOMER_A’s sites are in yellow. CUSTOMER_B’s sites are in green. You probably didn’t need me to tell you that though, right?

You’ll have spotted that each customer is using the same IP address space within their LANS. How are they doing that? The answer involves one of the major pieces of an MPLS VPN: each customer has their own VRF, which stands for Virtual Routing and Forwarding.

You may know already that VRFs are kind of like virtual routers within one single physical router. Each VRF represents a self-contained routing table, and by default each VRF has no interaction with any other VRFs on the router it lives on. As a result, these customers can both use the exact same private IP address space, and it doesn’t even matter if the subnets overlap because the VRFs don’t interact at all. Each customer has their own unique virtual routing table, within the physical router.

Routers 1, 2, and 3 all talk to each other via BGP to learn the prefixes in each other’s VRFs.

 

WHAT IS A ROUTE DISTINGUISHER?

Let’s focus on the two sites connected to PE1. We can see that both Customer A and Customer B are using the same private IP address in their LAN – 192.168.10.0/24.

We know already that PE1 has no problem with this: it’s aware that these two WAN interfaces are in different VRFs, and therefore these IP ranges are in different routing tables on PE1.

But when it comes to PE1 actually advertising these prefixes out via BGP, it’s a a different story.

Imagine that PE1 wants to advertise these two prefixes to PE2. If we’d stuck with the way BGP was originally designed to operate, here’s what would happen: PE1 would advertise the first 192.168.10.0/24 prefix to PE2, and PE2 would accept it. Then PE1 would advertise the 2nd 192.168.10.0/24 range to PE2 – at which point PE2 would think this was a newer version of the previous advertisement. Because of this, PE2 would actually overwrite the previous advertisement with this “newer” one!

To understand why that would have happened, you have to bear in mind that BGP was created before MPLS VPNs. The original BGP wasn’t really designed for a router to receive two of the same prefix from the same neighbor.

With that in mind, route distinguishers literally do what their name suggests – they help us to distinguish between multiple identical prefixes coming from the same neighbor, which allows a remote router to accept every single one of them. The result means that instead of receiving “192.168.10.0/24” twice, a router might instead receive these two prefixes:

  • 64512:100:192.168.10.0/24
  • 64512:700:192.168.10.0/24

The “64512:100” and “64512:700” bits are the route-distinguishers. See, I told you they did what they say on the tin!

 

LET’S LOOK AT SOME CONFIGURATION!

This is the configuration of Customer A’s VRF as it appears on PE1, the router in our ISP network that hosts Site 1. My lab is using only Juniper routers, but because I love you with all my heart, I’ll also show you the Cisco IOS equivalent as well. Let’s see how a VRF is configured on both vendors.

(By the way, if you’re coming from the Cisco IOS world, and you don’t yet understand Juniper config, I’ve got you covered in my blog post series that teaches Junos to Cisco IOS engineers. Click here to find out all about it!)

JUNOS CONFIG
 root@PE_1> show configuration routing-instances CUSTOMER_A
 instance-type vrf;
 interface ge-0/0/0.0;
 route-distinguisher 64512:100;
 vrf-target target:64512:200;
 routing-options {
     static {
         route 192.168.10.0/24 next-hop 172.31.1.1;
     }
 }
CISCO IOS EQUIVILENT
 ip vrf CUSTOMER_A
  rd 64512:100
  route-target export 64512:200
  route-target import 64512:200

 interface GigabitEthernet 0/0
 ip vrf forwarding CUSTOMER_A

 ip route vrf CUSTOMER_A 192.168.10.0 255.255.255.0 172.31.1.1

(You could also write “route-target both 64512:200” in the Cisco config, but I’ve found that folks often seem to prefer to do it the way I’ve done it above.)

The route-distinguisher itself is six bytes long (ie 48 bits). Actually, to be precise it’s really an 8 byte number (ie 64 bits), but the first two bits are reserved for the “type” field, which leaves us 6 bytes to play with. On some vendors you can write the route-distinguisher in three formats, depending on your personal preference:

  • [ 2 byte Autonomous System number ] : [ 4 byte unique number ]
  • [ 4 byte IP address ] : [ 2 byte unique number ]
  • [ 4 byte Autonomous System number ] : [ 2 byte unique number ]

In our configs above you can see that we’re using this first option: this VRF has a route-distinguisher of 64512:100. Make sure not to use this route-distinguisher in any other VRF on PE1. If two VRFs on the same PE router had the same route distinguisher, it would defeat the whole point! Luckily, Junos gives us an error message if we try to do this.

Let’s hop over to PE2 and see what it’s learning from PE1.

In Junos, 100% of the accepted VPN prefixes are stored in a special routing table called “bgp.l3vpn.0”. This is the table that stores the “raw” data, before the prefixes are then sorted into the correct VRFs. It’s a really handy place to go if you want to see exactly what’s being learned.

Behind the scenes I’ve also configured CUSTOMER_B’s VRF, which is also advertising its 192.168.10.0/24 range. With that in mind, let’s see if PE2 has learned both CUSTOMER_A and CUSTOMER_B’s prefixes:

root@PE_2> show route table bgp.l3vpn.0 192.168.10.0/24

bgp.l3vpn.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

64512:100:192.168.10.0/24
                   *[BGP/170] 00:45:14, localpref 100, from 1.1.1.1
                      AS path: I, validation-state: unverified
                    > to 10.2.4.4 via ge-0/0/0.0, Push 299840, Push 299808(top)
64512:700:192.168.10.0/24
                   *[BGP/170] 01:43:42, localpref 100, from 1.1.1.1
                      AS path: I, validation-state: unverified
                    > to 10.2.4.4 via ge-0/0/0.0, Push 299856, Push 299808(top)

We can see that the 192.168.10.0/24 prefix exists twice – and each prefix has been prepended with its own unique route-distinguisher!

When you see that the full advertised VPN prefix isn’t simply “192.168.10.0/24”, but instead is actually “64512:100:192.168.10.0/24”, it becomes much easier to see the purpose of the route distinguisher.

Below you’ll find a screenshot I did a while ago which shows you how to view the equivalent information in IOS. It’s a different prefix, from a different lab. Forgive me for not re-creating this entire network in Cisco IOS: I have a lot of chocolate to eat, which means I’m very busy.

 

AUTOMATICALLY GENERATING A ROUTE-DISTINGUISHER IN JUNOS

Let’s talk more about the three different possible ways you can write a route-distinguisher. In this config we used the first method, with a 2 byte AS number and a 4 byte number thats unique per-VRF on the box. The advantage of this format is that you have the option of using one route-distinguisher per-customer across your entire estate, which allows you to quickly identify which prefixes belong to which customer.

Just to be clear: it’s totally fine to use the same route-distinguisher for the same customer on two different routers. The thing we’re trying to distinguish is when one single PE router sends the same prefix for two different customers.

For example, imagine if a site was multi-homed to two PEs. Both PEs would advertise this customer’s prefix to the rest of the network, and both PEs could use the same route-distinguisher – because the fact that the advertisements are coming from different PEs is enough to make them unique.

Another popular way of writing route-distinguishers is to use the format where the first number is an IP address, typically the loopback address of the PE router in question. The advantage of this is that you can quickly see all the advertisements that came from one particular PE router.

Junos actually lets you automate the creation of route-distinguishers, and the automation uses this IP address format. Imagine that you don’t really care what the actual route distinguisher is: you just want Junos to automatically create a unique number per-customer, just to get things working. Let’s see how to do this.

On PE1, let’s delete the two route distinguishers I just configured, and then let’s add in a useful command where I just refer to the loopback IP of my router.

delete routing-instances CUSTOMER_A route-distinguisher 64512:100
delete routing-instances CUSTOMER_B route-distinguisher 64512:700
set routing-options route-distinguisher-id 1.1.1.1

As soon as we “commit” our changes, these VPN prefixes are withdrawn and re-advertised with new route-distinguishers. So be careful if you do this in production – you might briefly drop traffic!

Let’s see what PE2 receives after we make this change:

root@PE_2> show route table bgp.l3vpn.0 192.168.10.0/24

bgp.l3vpn.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

1.1.1.1:7:192.168.10.0/24
                   *[BGP/170] 00:00:01, localpref 100, from 1.1.1.1
                      AS path: I, validation-state: unverified
                    > to 10.2.4.4 via ge-0/0/0.0, Push 299872, Push 299808(top)
1.1.1.1:8:192.168.10.0/24
                   *[BGP/170] 00:00:01, localpref 100, from 1.1.1.1
                      AS path: I, validation-state: unverified
                    > to 10.2.4.4 via ge-0/0/0.0, Push 299888, Push 299808(top)

There we have it! The first half of the route-distinguisher is the router that advertised it, and the second half is just a random unique number. I’ve seen a lot of ISPs use this handy time-saver. The truth is, not many ISPs actually care about the route-distinguisher, and as such they’re happy to let their PE router just make a number up.

 

WHAT IS A ROUTE TARGET?

Now, what’s really interesting about this IP address format is that it introduces us to a subtle concept: if a customer’s VRF is configured on a number of PE routers, each of those PE routers could use totally different route-distinguishers. And a consequence of this is that route distinguishers don’t necessarily tell us anything about which VPN customer the prefix belongs to. Literally the only thing the route distinguisher does is make the prefix unique. Nothing more.

Something else that’s interesting is that the VRF name itself isn’t advertised. In fact, the VRF name is only locally significant to the PE router. I could go onto PE2 and rename the CUSTOMER_A VRF to something else, for example “SALLYS_ILLEGAL_WINE_EMPORIUM”, and everything would work just fine. Luckily for Sally though, I won’t do this. I don’t want to tip off the Feds. Also, she hooks me up with lots of illegal wine, and I don’t want to give the game away.

All this begs the question: if the VRF name isn’t advertised, and if the route-distinguisher is just some random number that may or may not have anything to do with the actual customer, then how does PE2 know which of the two 192.168.10.0/24 prefixes belongs to which customer?

And indeed, when we think about the fact that PE3 only hosts CUSTOMER_A, how does PE3 know which of the two advertisements to keep, and which one to ignore?

That’s where route targets come in.

In short, a route target is some extra information that’s added to an advertised prefix, to tell the receiving router exactly which VRF to import the prefix into. In fact, this “extra information” has a name: it’s an extended community.

You might have heard of communities in BGP: they’re basically just numerical tags that you can add onto a prefix. These tags don’t inherently mean anything (with a few notable and very important exceptions!), but we can configure our routers to perform a certain action based on the presence of a certain community. In other words, you yourself can give the communities meaning, and you can even make this meaning different on a router-by-router basis. (Don’t do that though. You can, but you shouldn’t.)

With route-targets, there’s a very specific action we want our router to take: “If a prefix is tagged with a certain route-target, install this prefix into VRFs that mention this route-target”.

Let’s take another look at our VRF configuration, and see this in action.

 

LET’S LOOK AT SOME CONFIGURATION!

Staying with CUSTOMER_A, let’s compare the config on PE1 and PE2. Remember, on PE1 we deleted the route-distinguishers, which is why we no longer see it in the config below. They’re just being generated automatically now.

JUNOS PE1 – CUSTOMER A’s VRF CONFIG
 root@PE_1> show configuration routing-instances CUSTOMER_A
 instance-type vrf;
 interface ge-0/0/0.0;
 vrf-target target:64512:200;
 routing-options {
     static {
         route 192.168.10.0/24 next-hop 172.31.1.1;
     }
 }
JUNOS PE2 – CUSTOMER A’s VRF CONFIG
 root@PE_2> show configuration routing-instances CUSTOMER_A
 instance-type vrf;
 interface ge-0/0/1.0;
 route-distinguisher 64512:100;
 vrf-target target:64512:200;
 routing-options {
     static {
         route 192.168.20.0/24 next-hop 172.31.1.9;
     }
 }

Let’s put the Cisco equivalent here again for comparison:

ip vrf CUSTOMER_A
 rd 64512:100
 route-target export 64512:200
 route-target import 64512:200

Notice that Cisco and Juniper write them slightly differently, but make no mistake: both configs do exactly the same thing.

The Cisco way of doing things makes it clear that route-distinguishers and route-targets are written in the same way. They’re both 8-byte numbers (ie 64 bits), where the first two bytes are reserved, giving us six bytes to play with.

Our route-distinguisher is 64512:100, but our route-target is 64512:200. Some ISPs choose to make the RD and the RT numbers the same, for consistency. But as you can see from our example config here, this is absolutely not a requirement in any way.

The Junos vrf-target line is actually performing two functions at once:

  • For prefixes in CUSTOMER_A’s VPN that PE1 advertises to PE2, the config is saying “tag the prefix with target:64512:200”.
  • For prefixes that PE1 receives from PE2, the config is saying “if the prefix is tagged with target:64512:200, put it into the CUSTOMER_A VRF”.
  • And of course, exactly the same thing is happening from the perspective of PE2, and indeed of PE3 as well.

In other words, the vrf-target line handles both importing AND exporting VPN prefixes.

Let’s hop back over to PE2 and once again look in that bgp.l3vpn.0 table – but this time, notice that I’ve deleted a lot of output so we can zoom in on one prefix in particular. I’m going to use the “detail” command to see a whole heap of information about this prefix.

root@PE_2> show route table bgp.l3vpn.0 detail

bgp.l3vpn.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
{snip}
1.1.1.1:7:192.168.10.0/24 (1 entry, 0 announced)
        *BGP    Preference: 170/-101
                Route Distinguisher: 1.1.1.1:7
                Next hop type: Indirect
                Address: 0x9758454
                Next-hop reference count: 6
                Source: 1.1.1.1
                Next hop type: Router, Next hop index: 581
                Next hop: 10.2.4.4 via ge-0/0/0.0, selected
                Label operation: Push 299872, Push 299808(top)
                Label TTL action: prop-ttl, prop-ttl(top)
                Load balance label: Label 299872: None; Label 299808: None;
                Session Id: 0x2
                Protocol next hop: 1.1.1.1
                Label operation: Push 299872
                Label TTL action: prop-ttl
                Load balance label: Label 299872: None;
                Indirect next hop: 0x9598000 1048574 INH Session ID: 0x8
                State: <Active Int Ext ProtectionPath ProtectionCand>
                Local AS: 64512 Peer AS: 64512
                Age: 19:09      Metric2: 1
                Validation State: unverified
                Task: BGP_64512.1.1.1.1+179
                AS path: I
                Communities: target:64512:200
                Import Accepted
                VPN Label: 299872
                Localpref: 100
                Router ID: 1.1.1.1
                Secondary Tables: CUSTOMER_A.inet.0
{snip}

Hey, there’s our route-target! And Junos even tells us what VRF it’s decided to import this prefix into, which saves us a bit of time. Remember, the VRF name isn’t advertised: the “Secondary Tables” you’re seeing in the output above is the result of our configuration in the VRF.

PE1, PE2 and PE3 all attach the same route-target when they advertise (ie “export”) CUSTOMER_A’s prefixes.

All three PEs also use that very same route-target to analyse the prefixes they receive, and work out which ones to import into CUSTOMER_A’s VPN.

As a result, all three PEs successfully install all of CUSTOMER_A’s prefixes that they receive from the other two PE routers – and thus, each PE router has full visibility and reachability to the prefixes at the other two PEs.

Actually, there’s a name for this type of topology: it’s called a full-mesh VPN, because each site can talk directly to every other site.

 

WAIT A SECOND…

Now, even after all that explanation, you might still be wondering: this is all very well and good, but why didn’t they just design all of this so that the RD and the RT was indeed the same number, and have one number serve both purposes? Or, why not just force the VRF name to be the same everywhere, and advertise the VRF name instead of this random RT number? All of this seems like a lot of nonsense.

And of course you’re correct, in that networking is mostly nonsense.

But to answer that question, let’s take the complexity up a notch – because it turns out that we can actually do some really clever things with route-target manipulation. For example, we can create more advanced topologies that send all site-to-site traffic via a firewall at a central hub site. Another example is that we can actually leak certain prefixes between VRFs, which means we can introduce a management VRF that has selective access to certain parts of our customer networks, giving us the ability to monitor and maintain devices we manage on their behalf!

To show you how this is possible, I’ve configured CUSTOMER_B in an unusual – but by seeing the result of this config, you’ll find it very easy to understand the concepts that follow.

 

JUNIPER ROUTING POLICIES AND CUSTOM TOPOLOGIES

Click this pic to make it bigger!

A key thing to understand is that the route-target you import and the route-target you export don’t have to be the same. In other words, you can configure your VRF to export one target, and import a totally different one!

We can see in our topology that CUSTOMER_B only has two sites, connected to PE1 and PE2. Now, let’s do an experiment.

Let’s set things up so that PE1 advertises (ie “exports”) all of CUSTOMER_B’s prefixes with target:64512:900. But, “just for fun”, let’s tell PE1 to only import prefixes into CUSTOMER_B’s VRF if they’re tagged with target:64512:800.

And on PE2, let’s do it the other way round. Let’s set PE2 to export all of CUSTOMER_B’s prefixes with target:64512:800. But, let’s tell PE2 to only import prefixes into CUSTOMER_B’s VRF if they’re tagged with target:64512:900.

How do we configure this on PE1? If PE1 were a Cisco, it would actually be quite easy:

ip vrf CUSTOMER_A
 rd 64512:100
 route-target export 64512:900
 route-target import 64512:800

I bet you can immediately see what’s going on there, right? Export everything from this VRF with a route-target of 64512:800, and import anything that’s tagged with 64512:900. All you’d have to do on PE2 is switch these numbers around, and you’re golden.

In Juniper land, it involves quite a few more lines of configuration – because if your import and export targets are different, you can’t just refer to the targets directly in your VRF config. Instead, you have to create named policies, and then refer to those policies within the VRF.

This might seem like extra config, and indeed, it is! However, the advantage of this extra configuration is that you can be really precise about what you want to import and export. You can create these detailed policies in IOS too, and they involve just as much typing. But in Junos, you have to use policies.

This might seem like a bit of a faff, but there’s a real advantage to being forced to use policies from the start: by setting things up like this in advance, it means you can easily add new “terms” to your policies in the future, to do more advanced things, and you can do it with zero downtime.

 

LET’S LOOK AT SOME CONFIG!

The first thing we need to do in Junos on PE1 is to define two target-communities and give them names, to make it easier to refer to these targets in policies. I like to use long names that are easy to read, but you can choose to give things much shorter names if you like. I like to name things in all-caps too, to make it easy to spot when I’m reading config. But again, you do you.

set policy-options community PE1_CUSTOMER_B_TARGET_EXPORT members target:64512:900
set policy-options community PE1_CUSTOMER_B_TARGET_IMPORT members target:64512:800

Now, let’s create an export policy called PE1_CUSTOMER_B_TARGET_EXPORT_POLICY. This policy has no match conditions, which in Juniper land means “match everything”. As such, when PE1 advertises prefixes in this VRF, it tags everything with our special export community of target:64512:900.

root@PE_1> show configuration policy-options policy-statement PE1_CUSTOMER_B_TARGET_EXPORT_POLICY
term ADD_TARGET {
    then {
        community add PE1_CUSTOMER_B_TARGET_EXPORT;
        accept;
    }
}

Next, a policy called PE1_CUSTOMER_B_TARGET_IMPORT_POLICY. Notice that this policy has two “terms”. The first term accepts anything and everything that contains our special import community of target:64512:800. The second term rejects everything else.

root@PE_1> show configuration policy-options policy-statement PE1_CUSTOMER_B_TARGET_IMPORT_POLICY
term ACCEPT_COMMUNITY {
    from community PE1_CUSTOMER_B_TARGET_IMPORT;
    then accept;
}
term REJECT_ALL_ELSE {
    then reject;
}

Now we’ve created these policies, we can refer to them in the VRF:

root@PE_1> show configuration routing-instances CUSTOMER_B
instance-type vrf;
interface ge-0/0/2.0;
vrf-import PE1_CUSTOMER_B_TARGET_IMPORT_POLICY;
vrf-export PE1_CUSTOMER_B_TARGET_EXPORT_POLICY;
routing-options {
    static {
        route 192.168.10.0/24 next-hop 172.31.1.5;
    }
}

The result? Let’s find out by learning some new Junos BGP commands.

First, let’s see what PE1 is exporting to PE2 (who has a loopback IP address of 2.2.2.2). Notice that this command lets me see what I’m sending to PE2 – and it lets me focus only on prefixes with target:64512:900.

root@PE_1> show route advertising-protocol bgp 2.2.2.2 community target:64512:900
CUSTOMER_A.inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
CUSTOMER_B.inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
  Prefix   Nexthop       MED     Lclpref    AS path
* 172.31.1.4/30           Self                         100        I
* 192.168.10.0/24         Self                         100        I

Now let’s see what PE1 is importing from PE2. This time, notice that I’m focusing only on prefixes with target:64512:800.

root@PE_1> show route receive-protocol bgp 2.2.2.2 community target:64512:800
{snip}
CUSTOMER_B.inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
  Prefix   Nexthop       MED     Lclpref    AS path
* 172.31.1.12/30          2.2.2.2                      100        I
* 192.168.20.0/24         2.2.2.2                      100        I
{snip}

There we have it! PE1 is sending CUSTOMER_B’s advertisements to PE2 with target:64512:900, and is receiving advertisements from PE2 with a target of target:64512:800.

We just learned two key things there. First, we learned that the target you send and the target you receive don’t have to match. Second, we learned that policies allow us to very easily match traffic and add communities to it.

Now you understand that, let’s talk about why this is useful.

 

HUB-AND-SPOKE VPNs

Click this pic to make it bigger!

Earlier on we mentioned the possibility of having a central hub site that hosts a firewall, and configuring our network so that all our sites have to talk to each other via this firewall.

The name for this topology is a hub-and-spoke VPN. One main site acts the hub, and all other spoke sites have to go via the hub if they want to talk to each other. By contrast, the full-mesh model allows each site to talk directly to every other site.

In fact, using a Junos policy, we could easily re-configure CUSTOMER_A to make this happen. We could choose Site 1 to be the hub, and Sites 2 and 3 to be the spokes. All we’d have to do is get PE1 to export a certain route-target, and get PE2 and PE3 to import that target. Similarly, we could get PE2 and PE3 to export a different route-target, and get PE1 to import it. The result is that PE2 and PE3 wouldn’t actually learn each other’s prefixes! They’d only learn whatever PE1 chooses to teach them.

How do we actually get PE1 to advertise PE2’s LAN to PE3, and vice versa? You have a few options. For example, you could get PE1 to actually re-advertise PE2’s prefixes to PE3, and vice versa, but get PE1 to rewrite the next-hop to be itself, forcing all traffic via the PE. Another option is to just advertise a default route to each site, which points to the hub.

Or, if you wanted maximum control, you could do something truly next-level. Imagine if you actually configured two VRFs at the hub PE, and had two logical WAN links going to Site 1, one link in each VRF. Imagine that one VRF was dedicated to learning prefixes from the VRFs, and another was for advertising prefixes from the VRFs. It sounds like a lot of work, but this option gives you a massive amount of control over who learns what. But wait: how does this give you more control than the other options?

As it happens, I’ve previously written a three-part series diving deep into exactly why! If you’d like to learn more about how we can use route targets to create hub-and-spoke MPLS VPNs, click here to read part 1 in my cool special series.

For now though, it’s enough to know that this kind of topology is possible by using exactly the config we just saw, where we advertise one target, but receive another.

Perhaps now it’s clear why the wizards that invented all this stuff decided to separate the functionality of the distinguisher from the functionality of the target. And indeed, perhaps it’s now clear why we don’t just use the customer’s VRF name itself in the advertisement.

 

USING ROUTE TARGETS TO CREATE A MANAGEMENT NETWORK

In fact, here’s another cool thing we could do: there’s nothing stopping me from writing a policy that imports a particular VPN prefixes into multiple VRFs on a router!

“Hang on”, I hear you thinking (I’m psychic, in case you didn’t know): “why would we do that? I thought the whole point was to keep a customer’s prefixes contained in their own VRF? Why would we ever want to leak routes between VRFs?”

In fact, there’s a very common use case for it. Imagine you, the service provider, have a piece of your network dedicated to management. Let’s say for the sake of argument that all these management boxes live inside one subnet, for example 10.254.254.0/24.

Some of the boxes in this subnet act as the monitoring platform for our estate. Other boxes act as jump boxes to log onto our customer routers. Others are operating as servers that collect logs, or perhaps they host images that you can push to our customer routers, allowing us to automate upgrades of devices that we manage on behalf of our dear and beautiful customers.

But here’s a question: if the customer routers are in their own unique VRFs, how can you log on and maintain stuff from within this Management VLAN?

GOOD QUESTION! Here’s the concept: you can create a dedicated management VRF somewhere in the network, and then configure things in such a way that extra route-targets are added to a prefix, as long as certain conditions are met, for example if the prefix is within a range dedicated to WAN links. You can then import just these WAN links into your management VRF.

Similarly, thanks to the magic of hub-and spoke, you can advertise 10.254.254.0/24 with a new dedicated target, and you can then configure your customer VRFs to accept both their target own and also your management target.

To see this in action, let’s add in this final extra bit of design to our network:

  • All our /30 WAN links will be in the range 172.31.0.0/16
  • Our management VRF will use the route-target target:64512:999

In a moment I’m going to add some lines to CUSTOMER_B’s VRF policies on PE1. This config will do two things:

  • In addition to importing prefixes tagged with this customer’s unique route targets, we will also import any prefixes tagged with the route-target of my management VRF.
  • If there is a prefix in CUSTOMER_B’s VRF that happens to be in the 172.32.0.0/16 range, advertise it as normal with the customer’s unique route-target – but in addition, also add target:64512:999 to it”.

 

ONE FINAL TIME: LET’S LOOK AT SOME CONFIG!

Considering that we’ve already got a policy on PE1 for CUSTOMER_B, let’s re-jig it.

The WAN link connecting PE1 to CUSTOMER_B_SITE_1 has an IP range of 172.31.1.4/30. With that in mind, on PE1 let’s define our new management community:

set policy-options community MANAGEMENT_VRF members target:64512:999

Next, let’s re-write our export policy to tag this WAN link with our special management community.

Notice in the config below that there’s a new term at the top of the policy. Notice as well that there’s no “accept” action in this new first term. This means that the MANAGEMENT_VRF community will be successfully added to the prefix, but the checks don’t end there; Junos carries on to the next term, and additionally checks if there’s a match there too. “Accept” really means “accept and stop checking the rest of the policy”.

root@PE_1> show configuration policy-options policy-statement PE1_CUSTOMER_B_TARGET_EXPORT_POLICY
term ADD_MANAGEMENT_VRF {
    from {
        route-filter 172.31.0.0/16 orlonger;
    }
    then {
        community add MANAGEMENT_VRF;
    }
}
term ADD_TARGET {
    then {
        community add PE1_CUSTOMER_B_TARGET_EXPORT;
        accept;
    }
}

Now, you’ll forgive me for not actually configuring a new Management VRF – this post is long enough already, and you get the idea by now. But what I want to show you is that PE2 receives 172.31.1.4/30 with two route-targets attached to it:

root@PE_2> show route table bgp.l3vpn.0 172.31.1.4/30 detail

bgp.l3vpn.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)

1.1.1.1:8:172.31.1.4/30 (1 entry, 0 announced)
        *BGP    Preference: 170/-101
                Route Distinguisher: 1.1.1.1:8
                Source: 1.1.1.1
{snip}
                Communities: target:64512:900 target:64512:999
{snip}

All we’d have to do in our special management VRF is to accept prefixes tagged with that community, and everything would work gloriously! And you can imagine how easy it would be to add a new line to our import policy that accepts any prefix with target:64512:999 on it.

What we see here is that prefixes can be tagged with multiple route-targets – and therein lies their power. You can tag prefixes multiple times, to install them in multiple places! Maybe you have a separate target for your cloud network, and another for your VoIP network, so that you can selectively install certain customer prefixes into some of your infrastructure VRFs – and indeed, to install certain infrastructure prefixes into certain customer VRFs!

Maybe you’ve even got some kind of advanced MPLS VPN that spans across multiple autonomous systems, and you’d like to use different communities in different parts of the network to quickly indicate where they came from. All these things and more are possible, thanks to the fact that we can add as many route-targets as we like.

 

THAT’S IT!

If you’re a first-time reader of this blog who comes from a Cisco background and you want to understand more about Junos, then remember to check out my blog series teaching Junos to Cisco IOS Engineers. Warning: Junos is a gateway drug. When it clicks, you won’t want to go back to IOS ever again.

Thanks so much for reading this post! If you found it useful, I’d love you to share it on your social media of choice, or maybe even email it directly to colleagues who might enjoy it. I’ve written tons of other posts on this site, many of them with a Service Provider theme, so take a read through the archives.

If you’re on Mastodon, follow me to find out when I make new posts. (2024 edit: I’m also on BlueSky nowadays too. I was once on Twitter, but I’ve given up on it, on account of… well, I don’t need to finish that sentence, do I.)

All this content is free for you, although I’ll never say no to a donation. This website is 100% a non-profit endeavour, in fact it costs me money to run. I don’t mind that one bit, but it would be cool if I could break even on the web hosting, and the licenses I buy to bring you this sweet sweet content.

Plus, enter my new competition where you could win a year’s supply of air! Simply go to your search engine of choice, and type the words “pictures of old men on skateboards” to automatically be entered. I’m currently monitoring all search engines, so don’t worry: I’ll find out when you type it. Do this every hour on the hour to increase your chances of winning. This is real. Do it. Do it now. Do it now, every hour. Do it to win a year’s supply of air. Don’t worry about where it came from. Don’t ask too many questions about this.

See you next time!

19 thoughts on “ROUTE DISTINGUISHERS vs ROUTE TARGETS: WHAT’S THE DIFFERENCE? (JUNOS FOR CISCO IOS ENGINEERS)

  • August 14, 2018 at 2:24 pm
    Permalink

    This is much helpful & full of information. I came to know that RD can be different between 2 or more routers but can hold similar routing table !!!!!! YAY !!!!!

    Reply
  • June 10, 2019 at 9:32 am
    Permalink

    Finally got it .. This is wonderful explaination RT and RD mate. Thanks

    Reply
  • August 18, 2019 at 11:59 am
    Permalink

    > The Sexy Trinity

    But really. Can we? =)

    Reply
  • January 18, 2020 at 6:41 pm
    Permalink

    Very well explained! Now if anyone from work has an issue grasping the concept – I’ll forward your blog to them.

    Reply
  • February 3, 2020 at 5:31 am
    Permalink

    Great explanations!!!

    Reply
  • March 10, 2020 at 8:54 am
    Permalink

    Amazing! I guess you could say the route target is to target which VRF the route will go into but the route distinguisher is to distinguish between the same route from different sources.

    Reply
    • April 6, 2020 at 9:47 pm
      Permalink

      Hi Jonathan!

      You’re bang on about the route TARGET. But the router DISTINGUISHER is a little bit different. It’s not to distinguish the same route from different *sources* – it’s to distinguish the multiples of the numerically-same prefix, coming from the same PE router. For example, two customers might be using 192.168.10.0/24 in different VRFs. Adding a unique route distinguisher onto these two IPs makes them unique when they’re being received from the neighbour.

      Reply
  • May 20, 2020 at 8:54 pm
    Permalink

    Amazing article, Chris. Finally, the complicated world of RD and RT makes sense. Thanks for breaking it down for us. 🙂

    Reply
    • July 6, 2020 at 12:40 pm
      Permalink

      Awesome, really glad I was able to help to make it clear. Thanks for reading Kunal!

      Reply
  • June 9, 2020 at 2:42 pm
    Permalink

    I really like the way you use lab diagrams to explain difficult topics. There are lots of articles published
    about RD and RT but this is by far the best explanation I have read through. Looking forward to read
    more 🙂

    Reply
    • July 6, 2020 at 12:26 pm
      Permalink

      That’s really kind of you to say, thank you so much! 🙂

      Reply
  • July 10, 2020 at 2:49 pm
    Permalink

    One more time I needed to review RD & RT and clarify a few things I knew that you had something.
    Your article it is really detailed and with really good scenarios.

    Thanks !!! 🙂

    Reply
    • August 1, 2020 at 4:02 pm
      Permalink

      My guy Manolis! Sorry for the slow reply. Super glad you found it helpful. I hope everything is good and well with you!

      Reply
  • December 27, 2021 at 11:36 am
    Permalink

    Nice article. Thank you for sharing you understanding of RD and RTs.

    Reply
  • February 14, 2022 at 3:10 pm
    Permalink

    I found this article very informative, actually I already have a background about this topic, but your article looks me to the next level, I really appreciate your efforts, thanks a lot

    Reply
  • December 16, 2022 at 4:06 pm
    Permalink

    This really helped me click some of the missing pieces. Thanks a lot!

    Reply
  • December 24, 2022 at 9:57 pm
    Permalink

    Hi Chris!

    I truly appreciate you writing these awesome articles. I’m trying to catch up with MPLS and they are pure gold.
    However, I can’t wrap my head around what defines a VPN. Is VRF the same as VPN when It comes to MPLS? Hosts in one VRF can’t communicate with hosts in other VRFs unless we import they RTs.

    Regards

    Reply
    • January 3, 2023 at 7:50 pm
      Permalink

      Hi there Jesús!

      The term “VPN” is a very broad one. You can use it to describe any time that two or more private networks are joined together over another network. That could be a site-to-site VPN using firewalls and encrypted tunnels, or it could be a VPN connection from your work laptop into a corporate LAN, or it could be an MPLS VPN that connects lots of sites together over a service provider backbone.

      In that respect, a VRF isn’t exactly the same as VPN – but it is fair to say that it’s one of the main building blocks in creating an MPLS Layer 3 VPN. The VRF is the name for the customer’s special routing table on the device, which is separate from the “main” routing table, and also separate from other customer routing tables.

      Reply

Leave a Reply to Chris Cancel reply

Your email address will not be published. Required fields are marked *