JUNOS ROUTING POLICY, BGP COMMUNITIES, AND MPLS VPNs

I fixed a ticket recently that I wanted to share with you, because it hits on three big topics at once: BGP communities, MPLS VPNs, and Junos routing policy.

If you fancy learning a little bit more about these three fun topics, then you’ve come to the right place! Pull up a chair, take your shoes off, grab a coffee, put your shoes back on, take them off again, grab yourself a cup of hot cocoa, put your shoes on, sit down, take your shoes off, put your shoes on, and get ready to learn some Deeply Cool Facts. Then, get a cup of tea, and take your shoes off. Why are you wearing shoes indoors? You monster. You absolute monster. Get off my website.

This is a blog post in two parts. First of all, we’re going to learn about BGP communities, and using multiple route targets in a Junos MPLS VPN. Then, in part 2, we’ll learn about a gotcha of Junos routing policy. If you’re comfortable with Communities and Route Targets already, click here to skip straight to Part 2. For the rest of us, let’s learning!!

 

WHAT IS A BGP COMMUNITY?

Most routing protocols allow you to add some kind of tag to an advertised route. This tag, which is essentially just a number, tells the router to take a certain action on the newly-learned prefix. These numbers tend to not have an industry-standard meaning; instead, we’re free to give these tags whatever meaning we like. You know: like how it’s up to us to work out whether there’s any meaning to our lives on this hell earth!!

So, for example, if a router learns 192.168.10.0/24 via OSPF, and the router sees the prefix is tagged with 100, you could say “This prefix has been redistributed into OSPF, from IS-IS. Therefore, don’t advertise it back into IS-IS.” If a route has tag 200, you could tell your beautiful, handsome router that it’s a high priority route, and to install it into the routing table first. This could be useful in a network with a lot of prefixes, where it’s important for certain essential parts of the network to be accessible as soon as possible.

One final example: if a route was tagged with 69, you could tell the router that the prefix was “nice”.

BGP also uses tags, but they’re a bit more complicated: in BGP, a tag is called a Community.

Communities works in a very similar way to tags: they’re numbers added to an advertised prefix, and you can make a router take action on a prefix based on the community it’s tagged with. Perhaps you could tell the router to increase the local preference. Perhaps you might perform some AS-path prepending. Or, if a prefix is tagged with your favourite number, maybe – just maybe – you’ll marry it. The only limit to the possibilities is your imagination! Well, your imagination, and also RFCs 1997 and 4360.

There’s actually two different types of community. Shall we take a look? Spoiler alert: yes!

 

COMMUNITIES vs EXTENDED COMMUNITIES

A regular community is 32 bits long, just like an IP address – though as we’ve seen already, it’s actually written a little differently to an IP address. The format is xxxx:yyyy. For example, 12345:32, or 6969:420. The first number is almost always the AS number (autonomous system) of the organisation, while the second number is just a unique number. 

Hey, here’s a fun exercise: how many unique numbers can you think of? Try to think of as many different numbers as you can. 15? 27? 193? There’s plenty more where that came from!

Most of the time these communities only have meaning to the admins of the autonomous system, though there are a few “well-known” communities. For example, if a prefix is tagged with community 65535:65281, routers recognise this as “NO-EXPORT”, and won’t advertise the prefix to any peer outside of the router’s AS. Thankfully, you don’t have to memorise the well-known numbers: routers will let you refer to them by name.

(That graphic above comes from a presentation called BGP Techniques for Network Operators, by a chap called Philip Smith. It’s well worth your time!)

Extended communities are 64 bits long – eight octets. The pic on the right is from the RFC. Notice that the first two octets have names: Type High and Type Low. They define the “type” of extended community, and whole bunch of types have been defined to allow you to do even more with our dearly beloved BGP. For example, there’s an Origin type, which allows you to define the router, or set of routers, that a prefix originated from. There’s a Bandwidth community, which allows you to inject link-bandwidth information into BGP, perhaps for route-decision making, perhaps for unequal-cost load-balancing.

The list of types is very long, and luckily you rarely actually need to know the numbers for these types, because all good routers allow you to just refer to the type by name. Scroll down this page to see some of the types available to you in Junos.

Let’s talk about the Target type. Why? Because it’s beautiful. It’s beautiful, and romantic, and poetic. Also: because we use it to make MPLS VPNs work!

 

HOW DO BGP COMMUNITIES MAKE MPLS VPNs WORK?

So, you may know that MPLS VPNs use a thing called a Route Target. They’re basically extended communities that you can attach to a prefix when it’s advertised by BGP, to identify that prefix as belonging to a particular customer’s VRF. You can then tell your other PE routers to import any prefixes they receive, that contain that community, into the correct VRF.

For example, in the config below, our friendly Juniper router will add any prefixes learned by BGP into this VRF, as long as they’re tagged with the community “target:6969:12345”. Also, when our router advertises out any prefixes in this VRF it will make sure to tag them with this community, so that other routers can add the prefix into the VRF at their end as well.

(By the way, if you’re curious about the actual number that’s used for the “target” type, there’s a good blog post here that describes how it all works.)

funtimes@Router5> show configuration routing-instances
     BARRYS_COOKIE_SHACK {
     instance-type vrf;
     interface em3.0;
     route-distinguisher 5.5.5.5:12345;
     vrf-target target:6969:12345;
     vrf-table-label;
     routing-options {
          static {
               route 172.16.2.0/24 next-hop 192.168.1.6;
               }
          }
     }

You can see the communities attached to a prefix by using the extensive knob, as in the output below. I’ve deleted some uninteresting lines of output.

funtimes@Router1> show route table BARRYS_COOKIE_SHACK.inet.0 172.16.2.0/24 extensive

BARRYS_COOKIE_SHACK.inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
172.16.2.0/24 (1 entry, 1 announced)
TSI:
KRT in-kernel 172.16.2.0/24 -> {indirect(131071)}
     *BGP Preference: 170/-101
     Route Distinguisher: 5.5.5.5:12345
     Next hop type: Indirect
     Source: 5.5.5.5
     Next hop: 10.10.12.2 via em0.0, selected
     Label operation: Push 16, Push 299808(top)
[snip]
     Communities: target:6969:12345
[and more snip]

“Cool!”

In the above config, we’re importing prefixes into our VRF that match a single community. We’re also attaching that community to every prefix we advertise out of the VRF. This is okay, but if you’re an ISP, and you also manage your customer’s CPE routers, the management of those CPEs can be a bit tricky. If the routers all have their own private IPs, and are in their own private VPN, how do you log on to them? How do you monitor them?

Well, how about this: as an ISP, what if we imported not just the customer’s prefixes into the customer’s VRF, but also routes to our management and monitoring platforms? That would be pretty cool: it would mean that even though the customer is using private IPs, in a private VPN, we could still connect to any customer’s CPE router, from one central platform. We’d need to make sure that our monitoring platform knew how to get to the CPEs as well, of course. And for sure, we’d want to make sure that our customers could get to the management IPs, but not to other customer routers.

That’s a lot of requirements. But guess what, bozo: with a little config magic, you can do just that!

 

USING MULTIPLE COMMUNITIES, FOR MANAGEMENT PURPOSES

Let’s say that our ISP has reserved the range 192.168.0.0/17 mostly for customer point-to-points. Let’s also imagine that 192.168.252.0/23 is reserved for the ISP’s management platforms, and 192.168.254.0/23 is reserved for the management point-to-points.

Let’s make up a new community, target:6969:1, aka MANAGEMENT. We’re going to make sure that any point-to-point links in any customer VRFs get tagged with this MANAGEMENT community, as well as being tagged with the community for the customer’s VRF itself. That’s right: two tags, on one prefix! For free! What a great offer! By doing this, we can create a new management VRF somewhere, and import all the WAN link IP addresses tagged with the MANAGEMENT community

We’re also going to import our two management IPs ranges into our customer VRF – but again, only if the prefix is tagged with the MANAGEMENT community. We need to filter by just that prefix, because if we didn’t then we’d accidentally import every single customer’s point-to-point network would be in every other customer’s VRF! That would be what security experts would call a “bit of an oopsie”.

(In fact, in the real world there’s even more security steps you’d want to take to make sure that customers can’t get onto other customer’s routers. But this is a single blog post, not a book, so to keep the config simple, just to demonstrate how the concept works, for now we’ll do it like this.)

Here’s what our new import and community statements look like on Routers 1 and 5:

community BARRYS_COOKIE_SHACK_VRF members target:6969:12345;
community MANAGEMENT members target:6969:1;

policy-statement BARRYS_COOKIE_SHACK_IMPORT {
     term 1 {
          from {
               community MANAGEMENT;
               route-filter 192.168.252.0/22 orlonger;
          }
          then accept;
     }
     term 2 {
          from community BARRYS_COOKIE_SHACK_VRF;
          then accept;
     }
}

You can  also have two separate policies, and refer to them both in the VRF config. Let’s see how it looks when you do it that way:

policy-statement MANAGEMENT_EXPORT {
     term 1 {
          from {
               route-filter 192.168.0.0/17 orlonger;
          }
          then {
               community add MANAGEMENT;
          }
     }
}

policy-statement BARRYS_COOKIE_SHACK_EXPORT {
     then {
          community add BARRYS_COOKIE_SHACK_VRF;
          accept;
     }
}

Now we’ve got some import and export statements defined, let’s alter the config of Barry’s VRF slightly:

funtimes@Router1> show configuration routing-instances | match "import|export"
     vrf-import BARRYS_COOKIE_SHACK_IMPORT;
     vrf-export [ MANAGEMENT_EXPORT BARRYS_COOKIE_SHACK_EXPORT ];

Then finally, on Router 4 in our lab I’m going to create my MANAGEMENT_VRF, and import everything tagged with the MANAGEMENT community.

funtimes@Router4> show configuration routing-instances
    MANAGEMENT_VRF {
    instance-type vrf;
    interface em3.0;
    route-distinguisher 4.4.4.4:1;
    vrf-target target:6969:1;
    vrf-table-label;
    routing-options {
         static {
              route 192.168.252.0/23 next-hop 192.168.254.2;
              }
        }
    }
}

A quick look in the routing table shows that Router4 has successfully only learned all our point-to-point links, and none of our customer LANs! That means that if we had a computer in the range 192.168.252.0/22, we could use it as a jumpbox onto any of our customer’s CPEs.

MANAGEMENT_VRF.inet.0: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.1.0/30      *[BGP/170] 00:24:34, localpref 100, from 1.1.1.1 < customer /30 from R1
                    AS path: I
                     > to 10.10.14.1 via em0.0, Push 16
192.168.1.4/30      *[BGP/170] 00:24:17, localpref 100, from 5.5.5.5 < customer /30 from R5
                    AS path: I
                     > to 10.10.45.5 via em1.0, Push 16
192.168.252.0/23    *[Static/5] 02:18:33
                     > to 192.168.254.2 via Static via em3.0
192.168.254.0/30    *[Direct/0] 02:19:38
                     > via em3.0
192.168.254.1/32    *[Local/0] 02:19:40
                       Local via em3.0

NOW YOU KNOW THE THEORY, LET’S SEE A PROBLEM!

So, you’re ready to find out what problem we had, and how to fix it? Well, friend, click here to read Part 2 of this blog post, where we’re going to learn about exactly that!

If you’re on Mastodon, follow me to find out when I make new posts. My posts are 80% excellent, and so good that you can forgive the 20% of bad ones. (2024 edit: I’m also on BlueSky nowadays too. I was once on Twitter, but I’ve given up on it, on account of… well, I don’t need to finish that sentence, do I.)

And if you like, you’d make my world if you shared this post on your favourite social media of choice!

And if you fancy some more learning, take a look through my other posts. I’ve got plenty of cool new networking knowledge for you on this website, especially covering Juniper tech and service provider goodness.

It’s all free for you, although I’ll never say no to a donation. This website is 100% a non-profit endeavour, in fact it costs me money to run. I don’t mind that one bit, but it would be cool if I could break even on the web hosting, and the licenses I buy to bring you this sweet sweet content.