j2sw Blog

As some of you may have heard Mikrotik has added in some VXLAN support in the latest RouterOS7 beta.  What is VXLAN and how would service providers use it? Let’s start out with some broad information about VXLAN

Where does TRILL and VXLAN fit in to your network strategy?

The always interesting RFC read
https://tools.ietf.org/html/rfc7348

This document describes Virtual eXtensible Local Area Network
   (VXLAN), which is used to address the need for overlay networks
   within virtualized data centers accommodating multiple tenants.  The
   scheme and the related protocols can be used in networks for cloud
   service providers and enterprise data centers

Boil it down for me. What is vxlan?
In short, VXLAN allows you to create a layer2 network on top of a layer3 network. It allows you to bind separate layer2 domains and make them look like one. If you are thinking this looks like a GRE tunnel, you are correct except the layer2 domains are still separate with tunnels. VXLAN is mainly touted as a way to interconnect data centers. If you are having to use spanning-tree then VLXLAN is an answer.

Okay, but why not use tunnels or MPLS?
VXLAN allows you to accomplish what GRE does without having to change the network design. By using VXLAN you are also able to have standalone layer2 domains that talk to each other. With the tunnel approach, you have to do a lot of manual configuration.

Is this just a data center thing?
VXLAN was designed to solve many of the edge computing and hyper-scale computing issues. Imagine having compute nodes in different parts of a data center or even in different data centers.  You want all of those nodes on the same VLAN.  With GRE you could extend that VLAN, but with VXLAN you can have two standalone layer2 VLANs that are merged together. VXLAN also solves the 4096 VLAN issue.  This is important in hyper-scale cloud computing.

VXLAN benefits in a nutshell

• increases layer2 segments to 16 million
• Centralize control
• Standards-based
• Scalable

VXLAN downsides in a nutshell

• Multicast must be available
• more overhead to layer2 packet
• no built-in encryption
• Slow adoption of ipv6 support by open source

What about the service provider? How can I use this?
In a service-provider network, you have things like broadcast issues. Basically, bridging is bad. Your layer2 networks need to be contained. Imagine you are a service provider who is providing LTE services. You may have an LTE VLAN on your network.  Historically you would have to extend your VLAN across the network in order to do management and access your LTE core. Now you have this large broadcast domain across your entire network.  Or worse yet, you have tunnels to other cities or locations you don’t have physically connected to your network.  Now you have tunnels a part of your LTE VLAN.  MTU issues and other things are now a part of your life.

With VXLAN each LTE node can have its own layer2 VLAN but still talk to the others. This prevents the broadcast storms which can occur.

Another use for VXLAN is a way to allow managed service providers to deploy large scale networks over the 4000 limits of VLANs.  You could literally deploy thousands of layer2 segments to tenants

Why I should or should not care about VXLAN as a service provider?
If you just have a couple of layer2 networks to extend across your network VXLAN is not for you. However, VXLAN does allow for multipath routing and other protocols to be extended to remote networks.

VXLAN adds 50+ bytes of overhead to the layer2 frame. In many service provider networks, this is not an issue due to MTU being raised for MPLS, etc.   IP multicast must be extended across the entire network. Mac addresses are used in creating a distribution network across all of the routed layer2 domains.

Large service providers have started looking at segment routing to solve many of the issues I talk about. This causing them to gravitate toward EVPN. EVPN allows for BGP for the control plane and MPLS for the data plane. More on this coming soon.

In closing, VXLAN is an ultra-cool technology and has use cases for service providers.  Other methods also exist to solve these issues in the service provider world. For those of you looking to learn all you can, I will be posting a list of links for my Patreon folks.