This content is for Patreon subscribers of the j2 blog. Please consider becoming a Patreon subscriber for as little as $1 a month. This helps to provide higher quality content, more podcasts, and other goodies on this blog.
This was originally published at https://www.mtin.net/blog/internet-routing-registries/
It has been updated form grammar, but I am working on an updated version of this,
Routing Registries are a mysterious underpinning of the peering and BGP world. To many, they are arcane and complicated. If you have found this article you are at least investigating the use of a registry. Either that or you have run out of fluffy kittens to watch on YouTube. Either way, one of the first questions is “Why use a routing registry”.
As many of us know BGP is a very fragile ecosystem. Many providers edit access lists in order to only announce prefixes they have manually verified someone has the authority to advertise. This is a manual process for many opportunities for error. Any time a config file is edited errors can occur. Either typos, misconfiguration, or software bugs.
Routing registries attempt to solve two major issues. The first is automating the process of knowing who has the authority to advertise what. The second is allowing a central repository of this data.
So what is a routing Registry?
From Wikipedia: An Internet Routing Registry (IRR) is a database of Internet route objects for determining, and sharing route and related information used for configuring routers, with a view to avoiding problematic issues between Internet service providers.
The Internet routing registry works by providing an interlinked hierarchy of objects designed to facilitate the organization of IP routing between organizations, and also to provide data in an appropriate format for automatic programming of routers. Network engineers from participating organizations are authorized to modify the Routing Policy Specification Language (RPSL) objects, in the registry, for their own networks. Then, any network engineer, or member of the public, is able to query the route registry for particular information of interest.
What are the downsides of a RR?
Not everyone uses routing registries. So if you only allowed routes from RR’s you would get a very incomplete view of the Internet and not be able to reach a good amount of it.
Okay, so if everyone doesn’t use it why should i go to the trouble?
If you are at a formal Internet Exchange (IX) you are most likely required to use one. Some large upstream providers highly encourage you to use one to automate their process.
What are these objects and attributes?
In order to participate you have to define objects. The first one you create is the maintainer object. This is what the rest of the objects are referenced to and based on. Think of this as setting up your details in the registry.
From this point you setup “object types”. Object types include:
If you want to learn more about each of these as well as templates visit this ARIN site.
So what do I need to do to get started?
The first thing you need to do is set up your mntner object in the registry. I will use ARIN as our example. You can read all about it here:https://www.arin.net/resources/routing/.
You will need a couple of things before setting this up
1.Your ARIN ORGID
2.Your ADMIN POC for that ORGID
3.Your TECH POC for that ORGID
Once you have these you can fill out a basic template and submit to ARIN.
descr: Example, Inc.
auth: MD5-PW $1$ucVwrzQH$zyamFnmJ3XsWEnrKn2eQS/
changed: firstname.lastname@example.org 20150202
The templates is very specific on what to fill out. The mnt-by and referral-by are key to following instructions. MD5 is another sticking point. The process is documented just in a couple of places. In order to generate your MD5-PW follow these instructions.
1. Go to https://apps.db.ripe.net/crypt/ Enter in a password. Make sure you keep this cleartext password as you will need it when sending future requests to ARIN’s Routing Registry.
2. Submit the password to get the md5 crypt password. Keep this password for your records, as you may need it when interacting with ARIN’s IRR in the future.
3. Add the following line to your mntner object template in the text editor.
Our example above has a MD5 password already generated.
Once this is done and created you can add objects. The most commonly added objects are your ASN and IP space.
Create your ASN object using the as-num template
descr: Example, Inc.
descr: 114 Pine Circle
descr: ANYWHERE, IN 12345
import: from AS65535 accept ANY
import: from AS65533 accept AS65534
export: to AS65533 announce ANY
export: to AS65535 announce AS2 AS65533
changed: email@example.com 20150202
The things to know about the above template are the import and export attributes.
Now on to adding IP space
Suppose you have IP space of 192.0.2.0/24 Your template would look like:
inetnum: 192.0.2.0 – 192.0.2.255
descr: Example, Inc.
descr: 115 Oak Circle
descr: ANYWHERE, IN 12345
changed: firstname.lastname@example.org 20150202
The password attribute is the cleartext password for your MD5 key.
Using RPSL in practice
Lately, there has been some discussion about the pricing of Internet Exchanges (IXes) and the reasons behind free peering ports and paid peering ports. In this article, I want to go over some of the benefits and pitfalls of both.
Everyone loves free. Sure, we all do. While most free peering has costs associated, those costs are usually in the form of cross-connect charges. In some data centers, this can be a one-time fee to many hundreds of dollars a month for a typical cross-connect. Herein lies the rub. Any entity providing services to others needs to have money for equipment upgrades, labor, accounting expenses, and costs of operation. This money can come in the form of donations from members or outside sources. A non-profit can be the vehicle for this, but you still need funding to support the entity. If the labor is volunteer, what happens if the most active volunteers get pulled away due to personal or business-related obligations? Do things still run? Is there a plan in place to fix any issues? I am not saying it can’t be done, but life happens. When it comes down to it, most people will choose their family’s welfare over a volunteer gig.
On the flip side, we have IXes, which charge port fees and other fees for peering. These fees go equipment and operational costs and sometimes salaries. Paid peering costs….well money.
The biggest question to ask yourself is how stable the company is. It does not matter if they are giving away ports or charging. What matters is are they going to be able to provide you service when things get busy, outages happen, or things get busy. When peering becomes more and more critical to a health network and your company’s bottom line, these questions also become essential. It does not matter if it is free or paid if the IX isn’t able to provide service. This scenario can happen on an IX, which charges no money or charges money for a port.
Had a Hurricane peer without any filtering or the like reach over 800,000 routes today. As some of you may know providers do their own aggregation and such so your mileage may vary.
Normally on a peering exchange, all connected parties will establish bilateral peering relationships with each other customer connected to the exchange. As the number of connected parties increases, it becomes increasingly more difficult to manage peering relationships with customers of the exchange.
However, by using route servers for peering relationships, the number of BGP sessions per router stays at two, if the IX has deployed redundant servers.
Imagine this scenario. Outside your house, the most awesome superhighway has been built. It has a speed limit of 120 Mile Per Hour. You calculate at those speeds you can get to and from work 20 minutes earlier. Life is good. Monday morning comes, you hop in your 600 horsepower Nissan GT-R, put on some new leather driving gloves, and crank up some good driving music. Your pull onto the dedicated on-ramp from your house and are quickly cruising at 120 Miles an hour. You make it into work before most anyone else. Life is good.
Near the end of the week, you notice more and more of your neighbors and co-workers using this new highway. Things are still fast, but you can’t get up to speed like you could earlier in the week. As you ponder why you notice you are coming up on the off-ramp to your work. Traffic is backed up. Everyone is trying to get to the same place. As you are waiting in the line to get off the superhighway, you notice folks passing you by going on down the road at high rates of speed. You surmise your off-ramp must be congested because it is getting used more now.
Speedtest servers work the same way. A speedtest server is a destination on the information super-highway. Man, there is an oldie term. To understand how these servers work we need a quick understanding of how the Internet works. The internet is basically a bunch of virtual cities connected together. Your local ISP delivers a signal to you via Wireless, Fiber, or some sort of media. When it leaves your house it travels to the ISP’s equipment and is aggregated with your neighbors and sent over faster lines to larger cities. It’s just like a road system. You may get access via a gravel road, which turns into a 2 lane blacktop, which then may turn into a 4 lane highway, and finally a super-highway. The roads you take depend on where you are going. Your ISP may not have much control over how the traffic flows once it leaves their network.
Bottlenecks can happen anywhere. Anything from fiber optic cuts, oversold capacity, routing issues, and plain old unexpected usage. Why are these important? All of these can affect your results and can be totally out of control of your ISP and you. They can also be totally your ISP’s fault.
They can also be your fault, just like your car can be. An underpowered router can be struggling to keep up with your connection.Much like a moped on the above super-highway can’t keep up with a 600 horsepower car, your router might not be able to keep up either. Other things can cause issues such as computer viruses, and low performing components.
Just about any network can become a speedtest.net node or a node with some of the other speedtest sites. These networks have to meet minimum requirements, but there is no indicator of how utilized these servers are. A network could put up one and it’s 100 percent utilized when you go running a test. This doesn’t mean your ISP is slow, just the off-ramp to that server is slow.
The final thing we want to talk about is the utilization of your internet pipe from your ISP. This is something most don’t take into consideration. Let’s go back to our on-ramp analogy. Your ISP is selling you a connection to the information super-highway. Say they are selling you a 10 meg download connection. If you have a device in your house streaming an HD Netflix stream, which is typically 5 megs or so, that means you only have 5 megs available for a capacity test while that HD stream is happening. Speedtest only tests your current available capacity. Many folks think a speedtest somehow stops all the traffic on your network, runs the test, and starts the traffic. It doesn’t work that way. Your available capacity is only tested at that point in time. The same is true for any point between you and the speedtest server. Remember our earlier analogy about slowing down when you got to work because there were so many people trying to get there. They exceeded the capacity of that destination. However, that does not mean your connection is necessarily slow because people were zooming past you on their way to less congested destinations.
This is why results to a server should be taken with a grain of salt. They are a useful tool, but not an absolute. The speedtest server is just a destination. That destination can have bottlenecks, but others don’t. Even after this long article, there are many other factors which can affect Internet speed. Things we didn’t touch on like Peering, the technology used, speed limits, and other things can also affect your internet speed to destinations.
Imagine this scenario. You have bought an IP or DIA circuit from someone that is going to provide your network with bandwidth. Typically this company will make the connection, IP wise, over a /30 or even a /29 of IP space. I have called this the “glue address” for many years. This is the IP address that binds (the glue reference) you to the other provider’s network. They can route you IP blocks over that glue address or you can establish BGP across it, but it is the static address which binds the two networks together.
Some network folks call this a peering address. This isn’t wrong but can infer you are doing BGP peering across the address. You aren’t always doing BGP across the glue address.
Explainer video about what an Internet exchange point (IXP) is and how peering works. Making the internet better through peering. https://www.fd-ix.com #keeptrafficlocql #routinglight #peering #ixp
Martin J. Levy from Cloudflare did a presentation about remote peering possibly being a bad thing. In this presentation, he brings up several valid points.
Some thoughts of my own.
Yes, remote peering is happening. One thing touched upon is the layer3 vs layer2 traffic. We at MidWest-IX only allow remote peering at a layer2 level unless it is groups like routeviews.org or other non-customer traffic situations.
Many providers are overselling their backbone and transit links. This oversubscription means access to content networks in places that do not have an exchange or places that do have the content locally can suffer through no fault of the ISP or the content provider. We have situations with content folks like Netflix who do not join for-profit IXes at the moment, keeping the content further away from customers. These customers are reaching Netflix through the same transit connections many other providers are. The can result in congested ports and poor quality for the customer. The ISP is left trying to find creative ways to offload that traffic. An Internet Exchange is ideal for these companies because cross-connect charges within data centers are on the rise.
When we first turned up MidWest-IX, now known as FD-IX, in Indianapolis we used a layer2 connection to Chicago to bring some of the most needed peers down to our members. This connection allowed us to kick-start our IX. We had one member, who after peering with their top talkers, actually saw an increase in bandwidth. The data gained told the member that their upstream providers were having a bottleneck issue. They had suspected this for a while, but this confirmed it. Either the upstream provider had a congested link, or their peering ports were getting full.
As content makes it way closer remote peering becomes less and less of an issue. There are many rural broadband companies just now getting layer2 transport back to carrier hotels. These links may stretch a hundred miles or more to reach the data center. The rural broadband provider will probably never get a carrier hotel close to them. As they grow, they might be able to afford to host caching boxes. The additional cost and pipe size to fill the caches is also a determining factor. The tradeoff of hosting and filling multiple cache boxes outweighs the latency of a layer2 circuit back to a carrier hotel.
I think remote peering is necessary to by-pass full links which give the ISP more control over their bandwidth. In today’s race to cut corners to improve the bottom line having more control over your own network is a good thing. By doing a layer2 remote peer you might actually cut down on your latency, even if your upstream ISP is peered or has cache boxes.