This content is for Patreon subscribers of the j2 blog. Please consider becoming a Patreon subscriber for as little as $1 a month. This helps to provide higher quality content, more podcasts, and other goodies on this blog.
So a question I get on a fairly regular basis is how do I assign IPs to my customers out of this /29 block my ISP gave me. As many of you know, IPV4 public addresses, commonly known as globally routable IPV4 addresses, are in short supply and obtaining more is usually very expensive. You have three options to obtain IP addresses.
Option #1 is getting them from whoever is supplying your bandwidth. These IPs are known as Provider Assigned IP Space or PI space for short. Typically these IP addresses are leased to you for a price or just allocated to you because you are a customer. The key here they are tied to the provider. If you leave for someone else, your IPs are typically not portable.
Option#2 is going to the open market to buy or lease IP addresses. There are several companies out there which sell or rent IP addresses. For business continuity buying your addresses is the best way to go. As of this writing, IPV4 addresses in the ARIN region at going for $17-21 an Ip address. This pricing means the smallest block you can get costs you around $5,600.
Option #3 is asking ARIN or whatever Regional Internet Registry (RIR) you fall under and getting put on a waiting list. This wait can take quite some time and you may be waiting for a very long time.
Most startup ISPs get a small block of addresses from their provider. The block can be a /29 ,or sometimes they can get a /28 or larger. The upstream ISP routes the block(s) to the startup. Here is where the problem comes up. If an ISP gets a /29 this means they have 1 block of 6 IPs to use or two blocks of two IPs they can use in the form of two /30’s. For those of you not familiar with subnetting, each time you “break down” a block of IP addresses you always lose 2 to the network and broadcast addresses.
if you want to learn about subnetting the following Youtube Videos will be helpful. Be warned there is a lot of numbers and math involved, but it’s necessary to learn about subnetting and proper IP planning.
Imagine an ISP has two customers on different towers that want a public IP address. These addresses are for cameras, VPNs, or any number of uses. In order to assign IPs to each of those customers out of your /29 you have to either break it apart into two /30’s and route that to the appropriate router of yours the customer is attached to. With small blocks assigned to you, this becomes very inefficient. Even with large blocks, it can be inefficient. The below diagram shows how we can properly subnet this out given the limited IP block you have.
We broke out 126.96.36.199/29 into the following subnets.
We still have the same number of IP addresses we had before, but due to the way subnetting works we lost a total of 4 addresses to the network and broadcast addresses. Think of these as the beginning and end of the subnet. We can’t use them to assign to devices, but we need them as part of the subnet. We can note these as the following
188.8.131.52/30 – Network Address
184.108.40.206/30 – FIrst assignable address
220.127.116.11/30 – Second Assignable address
18.104.22.168/30 – Broadcast address
The next subnet would be 4,5,6,7 in the same order. The rules of subnetting say we can not assign the network and broadcast addresses to devices or else we start breaking things.
Now if both of these customers were on the same tower, behind the same ISP router we would not necessarily have to subnetted the /29 out. We could have routed the /29 to the router and customer 1 has the .2 IP address and customer 2 has the .3 IP address. the subnet would have IPs 4-6 available, but only for customers on the same tower.
The other way to save on public IP addresses is to do a 1:1 nat. Natting involves doing port forwarding from the public IP to a private IP address at the customer location. If you want to learn about 1:1 Nat here are some links:
So what is the real problem? If you have a /29 you can only break that down twice into the two /30s for customers going through different routers. Your network is routed instead of bridged right?
So how does the startup WISP overcome this without spending lots of money on IP addresses? You can do the 1:1 nat for each customer as explained above. In my experience, this works 98% of the time. It is also something most I.T. folks are not familiar with. I have had to spend hours on the phone with customers over the years explaining how a 1:1 nat works and they do have a public address to connect to, even though they have a private IP address on their router. Customers who do not get the 1:1 nat concept seem to be very leary of it and often blame the 1:1 nat when things go wrong. Now if you are an ISP that provides a managed router, the customer does not see this layer.
Your other option is an MPLS network with VPLS tunnels. MPLS with VPLS allows you to bridge your sites together to share an IP address pool. However, bridging is bad is what you hear. MPLS has things like split-horizon built into the protocol to make bridging things together not so bad of a thing.
Another option is PPPoE. PPPoE creates a tunnel utilizing a /32 (one ip address). Many providers are using a shared pool to hand out IP addresses to customers who are given Public Ip addresses.
But my provider also gave me a /30 that is between my router and their’s. Can I use this?
The short answer is this is the “Glue” between you and the upstream provider. The only real use for this is to Nat traffic out the Ip assigned to your side. Doing this is not scalable though.
I have briefly touched on things like PPPoE, MPLS, and other things in this article. These are such large topics there was no way this single post could do them justice. I wanted to present the problem, a solution, and other methods to research to solve it. I will go into these in later blog posts.
Recently, there has been much talk about the Cloudflare BGP blunder and others. The Network Collective even did a video about such things. But did you know there was one involving the entire /12 of IPV6 space? Airtel AS9498 announced the entire IPv6 block 2400::/12 for a week and no-one noticed. Someone typed a /12 instead of a /127.
So why did no one notice? I think part of it is due to the low usage of v6 space. Sure, all kinds of people claim stats on IPV6 usage. They talk about X number of traffic is v6, etc. There is a difference between users and connections. A connection may not actually represent unique users.
Secondly, people are used to IPV6 being buggy. I know many ISPs who disabled v6 as part of their troubleshooting steps.
I know there will be several folks who jump all over me about IPV6 being the wave of the future and we all should be using it. Yes, we should, but there is no huge hurry when it comes to business cases.
Spectrum use Article
The wireless industry has always had to deal with regular (and alarming) pronouncements that we’re somehow running out of radio spectrum. We’re not. But the misconception regardless gives many IT and network managers pause.
It does not mention WiSPs, but is a perspective nonetheless.
How many of you are focusing on Internet of Everything (IoT)? I have posted some links to how healthcare and others are using IoT to further their business. As a service provider, you should be coming up with an IoT strategy.
Microsoft and IPV6
I have written about IPV6 lately and Microsoft has published a post where they are moving their internal network to an IPV6 only network.
Carriers and telcos are also interested in adopting Kubernetes — AT&T is using Kubernetes as the basis of the Airship project it will use to run 5G and public safety network services — but they’ve already deployed significant amounts of IPv6, especially for mobile networks. Around 90 percent of T-Mobile USA and Verizon Wireless traffic already goes over IPv6; for Comcast and AT&T it’s about 70 percent.
This post is designed to give you a lot of information on implementing IPV6 into your network. Much of this has is aimed at the service provider market, but there are resources for the enterprise market.
One of the mindsets of this article is you are treating your customers as one of two ways. The first is a typical ISP end user. This can be John Smith on your wireless network or the small branch office down the street. These can vary in size and I will talk about how you can deal with these. The second type is the Enterprise or BGP peer. These are folks who should have their own IPV6 allocations and ASN.
Anyone who has followed me for a while knows I talk a lot of network philosophy. This article has bits of philosophy mixed in this article. Some of it is my own while others are the debates on certain aspects of IPV6 implementation. This is not a comprehensive guide to IPV6. rather, this is designed to fill in some blanks.
Packet Pushers has an excellent IPV6 planning podcast
Kevin Myers over at Stubarea51 has a great article on implementing IPV6 in a service provider market using Mikrotik
Typical service providers will are assigned a /36 or a /32 from their Regional Routing Registry (RIR). Don’t think in terms of IP addresses, but think in terms of subnets. I will emphasize this throughout the article. Also, another rule of thumb is when you are subnetting everything has to be done in multiples of 4 when it comes to the subnet mask. As with anything, there are exceptions but they might not be best practices.
IPV6 and Point-to-Point links
There is much debate on whether you should use a /64, a /124,126, or a /127 for point-to-point links. These would be the equivalent of /30’s in the IPV4 world. One of my favorite articles on this topic https://tools.ietf.org/id/draft-palet-v6ops-p2p-links-00.html
I typically use a /124 or /126, which seems to be what most of the larger upstream providers and content networks are using for point-to-point links. If you are using a /64 you have to contend with neighbor exhaustion attacks as well as ping pong attacks. One of the hybrid approaches is to use a /124,/126 or a /127 pulled from a single /64. I have yet to have a major upstream or content network hand me a /127.
Point-to-point links are one of the most highly talked about IPV6 methodologies. There are many right answers in this debate, as you can tell my the ietf article above. My advice is to pick one and go with it. The majority of the BGP peers I work with use either a /124 or a /126. I pull these out of a single /64. Some ISPs will allocate a /56 or /60, which we will talk about next, and use the first /64 and pull their point-to-point from that.
IPV6 and customer allocations
For years the standard allocation to a customer was a /56. In recent years this has been shrunk to a /60 with most of the major providers, such as Comcast. When we break down allocations in a service provider network, we have the standard accepted school of thought.
- Individual pops are assigned a /48 prefix.
- Enterprise customers are assigned a /56 prefix. That gives them 256 /64 subnets.
- Small customers are assigned a /60 prefix, allowing 16 /64 subnets.
One of the critical errors in thinking I see network engineers and architects do in IPV6 thinking is concentrating too much on IP addresses and not on the subnets. Just like in IPV4 subnetting it is all about the math. Assigning a customer a /64 directly is bad practice. The biggest reason is it does not allow the customer the ability to subnet the block into smaller chunks.
Some engineers think a customer will never use 18,446,744,073,709,551,616 and they would be correct. That is 18 quintillion IP addresses. 99.99 percent of us have never dealt with numbers that high. Many of us have only heard about numbers that high in science fiction such as Star Trek. However, a /64 is the smallest subnet assigned without breaking core functionality of IPV6. Remember earlier when we talked about point-to-point links? When we subnet out the /64 we are taking away sing a subnet prefix length other than a /64 will break many features of IPv6, amongst other things Neighbor Discovery (ND), Secure Neighborship Discovery (SEND) [RFC3971], privacy extensions [RFC4941], parts of Mobile IPv6 [RFC4866], PIM-SM with Embedded-RP [RFC3956], and SHIM6 [RFC5533]. Many new IPV6 developments also are relying on /64s being in place.
Back to our point-to-point addressing, we do not need neighbor discovery because we are only talking to one host. We don’t need privacy because we know the one host we are talking with, and so on.
But 18 quintillion IP addresses is still too much. Yes, but all the features relying on at least a /64 are worth the tradeoff. The key is to think in terms of subnets, not IP addresses.
So why not assign the customer a /62 to save on space..err..I mean subnets? Remember, when subnetting the math has to be in multiple of 4. So you can assign a /48, a /52, a /56, or a /60 to the customer and be safe. We talked about why assigning a /64 is bad practice even though it fits the multiple of 4.
Provider assigned IP (PI) space
There are very few cases for assigning Provider space to a customer who has an ASN. If the customer has an ASN they should be going to their Regional Internet Registry (RIR) and getting an allocation. In my case, most of the requests I do go through ARIN. Arin has a very good document on getting your first IPV6 allocation. https://www.arin.net/resources/guide/ipv6/first_request/
Anyone else I want to assign them a /56 or /60 as stated above. These are customer not participating in BGP. One of the lessons learned in the IPV4 world is how many started with provider assigned IP space and then went through the pain of renumbering when they received allocations from the RIR. In IPV6 we would have to re-subnet, but still the same pain.
What gets assigned IPV6 space?
One of my philosophies is I treat IPV6 like public IP addresses in the IPV4 world. I am not assigning Internet routable IP addresses to my switches, and APs unless they are doing routing. I can still use private Ip addresses for my infrastructure such as 10.x.x.x./8
Philosophies rounded up
-Point-to-point links. Should you use a /64,124,126,127, or even something else. I have seen Cogent hand me a /112 in several places
-Customer Allocations. Should you assign the customer a /56 or bigger? I believe any customer that needs more than a /56 should have their own ASN and their own IPV6 allocation. If necessary a large customer could have a /52 and still be within the bounds.
IPv6 is very much in flux with some conflicting methods of doing things. However, a good solid plan you stick to and follow can save you lots of headaches and allow your network to scale. Implementing IPV6 to some degree on your network should be a priority. There are many benefits to IPV6, which are not IP space-related which I will talk about in a later article
Other articles I have done on V6
#routingrf #routinglight #bendinglight