VXLAN terminology and what it means

What are some of the various terminology used in VXLAN?

VNI/VNID: Virtual Network Identifier or VXLAN Network Identifier

Layer2 VNI carried in VXLAN bridged packets. VNI is configured per VLAN.

Layer3 VNI carried in VXLAN routed packets across VTEPs. One L3 VNI per tenant (VRF) Note: Tenant, VRF, L3VNI are sometimes used interchangeably

VTEP: VXLAN Tunnel End Point

Performs VXLAN encapsulation/decapsulation

NVE: Network Virtualization Edge

Logical representation of the VTEP

VXLAN Gateway

Device that forwards traffic between VXLANs

It could be both Layer 2 and Layer 3 forwarding.

Anycast Gateway

All VTEPs are configured with the same IP and MAC on a host facing SVI

Underlay Network: Provides the transport for VXLAN

VXLAN an us OSPF,ISIS,EIGRP, BGP, or Multicast routing

VXLAN and why you should care as a service provider

As some of you may have heard Mikrotik has added in some VXLAN support in the latest RouterOS7 beta.  What is VXLAN and how would service providers use it? Let’s start out with some broad information about VXLAN

Where does TRILL and VXLAN fit in to your network strategy?

The always interesting RFC read

This document describes Virtual eXtensible Local Area Network
   (VXLAN), which is used to address the need for overlay networks
   within virtualized data centers accommodating multiple tenants.  The
   scheme and the related protocols can be used in networks for cloud
   service providers and enterprise data centers

Boil it down for me. What is vxlan?
In short, VXLAN allows you to create a layer2 network on top of a layer3 network. It allows you to bind separate layer2 domains and make them look like one. If you are thinking this looks like a GRE tunnel, you are correct except the layer2 domains are still separate with tunnels. VXLAN is mainly touted as a way to interconnect data centers. If you are having to use spanning-tree then VLXLAN is an answer.

Okay, but why not use tunnels or MPLS?
VXLAN allows you to accomplish what GRE does without having to change the network design. By using VXLAN you are also able to have standalone layer2 domains that talk to each other. With the tunnel approach, you have to do a lot of manual configuration.

Is this just a data center thing?
VXLAN was designed to solve many of the edge computing and hyper-scale computing issues. Imagine having compute nodes in different parts of a data center or even in different data centers.  You want all of those nodes on the same VLAN.  With GRE you could extend that VLAN, but with VXLAN you can have two standalone layer2 VLANs that are merged together. VXLAN also solves the 4096 VLAN issue.  This is important in hyper-scale cloud computing.

VXLAN benefits in a nutshell

  • increases layer2 segments to 16 million
  • Centralize control
  • Standards-based
  • Scalable

VXLAN downsides in a nutshell

  • Multicast must be available
  • more overhead to layer2 packet
  • no built-in encryption
  • Slow adoption of ipv6 support by open source

What about the service provider? How can I use this?
In a service-provider network, you have things like broadcast issues. Basically, bridging is bad. Your layer2 networks need to be contained. Imagine you are a service provider who is providing LTE services. You may have an LTE VLAN on your network.  Historically you would have to extend your VLAN across the network in order to do management and access your LTE core. Now you have this large broadcast domain across your entire network.  Or worse yet, you have tunnels to other cities or locations you don’t have physically connected to your network.  Now you have tunnels a part of your LTE VLAN.  MTU issues and other things are now a part of your life.

With VXLAN each LTE node can have its own layer2 VLAN but still talk to the others. This prevents the broadcast storms which can occur.

Another use for VXLAN is a way to allow managed service providers to deploy large scale networks over the 4000 limits of VLANs.  You could literally deploy thousands of layer2 segments to tenants

Why I should or should not care about VXLAN as a service provider?
If you just have a couple of layer2 networks to extend across your network VXLAN is not for you. However, VXLAN does allow for multipath routing and other protocols to be extended to remote networks.

VXLAN adds 50+ bytes of overhead to the layer2 frame. In many service provider networks, this is not an issue due to MTU being raised for MPLS, etc.   IP multicast must be extended across the entire network. Mac addresses are used in creating a distribution network across all of the routed layer2 domains.

Large service providers have started looking at segment routing to solve many of the issues I talk about. This causing them to gravitate toward EVPN. EVPN allows for BGP for the control plane and MPLS for the data plane. More on this coming soon.

In closing, VXLAN is an ultra-cool technology and has use cases for service providers.  Other methods also exist to solve these issues in the service provider world. For those of you looking to learn all you can, I will be posting a list of links for my Patreon folks.

OTV and VXLAN differences

A great article on explaining what OTV is and how it compares to VXLAN.

OTV(Overlay Transport Virtualization) is a technology that provide layer2 extension capabilities between different data centers. In its most simplest form OTV is a new DCI (Data Center Interconnect) technology that routes MAC-based information by encapsulating traffic in normal IP packets for transit”


EVPNs: The answer to your MPLS issues

I had a good discussion with my Buddy JJ tonight on kind of the next step of network evolution for provider networks.  Many providers have evolved to MPLS networks with VPLS.  There are some inherent issues with this when it comes to things like bonding, MLAG, among other issues. Nothing is perfect, right?

So as we dive into What is EVPN I want you to know I am approaching this from a service provider standpoint. I also am no EVPN expert, but I am seeing it more and more as a solution to solve specific issues.  As a result, EVPN is sliding into a natural progression of the service provider network.

So what is EVPN?
There are folks much more versed on EVPN than I am. As a result, I will lean on some already written articles.


Components of EVPN
Now that you have a high-level overview of EVPN, what are some of the major components and features you should know? Let’s dive into that

Unified control plane.  EVPN can be used throughout your network.  You don’t have to use one stack for data center, one for metro to the data center, and yet another for connectivity between data centers. You can bring it all under one control roof so to speak.

EVPN, through BGP, marries the Layer 2 and Layer 3 layers together.  With MPLS everything is controlled at the layer3 level.  Now with EVPN Mac addresses become much more important. For example, Each EVPN MAC route announces the customer MAC address and the Ethernet segment associated with the port where the MAC was learned from and is associated MPLS label. This EVPN MPLS label is used later by remote PEs when sending traffic destined to the advertised MAC address. Pretty cool huh?

Image result for evpn service provider

As networks grow network engineers learn about things such as north-south traffic and east-west traffic.  Microsoft has a great article which explains this concept. https://blogs.technet.microsoft.com/tip_of_the_day/2016/06/29/tip-of-the-day-demystifying-software-defined-networking-terms-the-cloud-compass-sdn-data-flows/

East-West – East-West refers to traffic flows that occur between devices within a datacenter. During convergence for example, routers exchange table information to ensure they have the same information about the internetwork in which they operate. Another example are switches, which can exchange spanning-tree information to prevent network loops.

North | South – North- South refers to traffic flows into and out of the datacenter. Traffic entering the datacenter through perimeter network devices is said to be southbound. Traffic exiting via the perimeter network devices is said to be northbound.

So, if you are a growing Service provider look at EVPN.  In some upcoming articles, I will talk more about various components of EVPN and such.


Where does TRILL and VXLAN fit in to your network strategy?

As networking trends yo-yo between layer-3 and layer-2 centric different protocols have emerged. Protocols such as Transparent Interconnection of Lots of Links (TRILL), Shortest Path Bridging (SPB), and Virtual Extensible LAN (VXLAN) have emerged to address the need of scalability at Layer2.   Cloud scalability, spanning tree bridging issues, and big broadcast networks start to become a problem in large data center or cloud environments.

To figure out if things like TRILL is a solution for you, you must understand the problem that is being addressed by TRILL. The same goes for the rest of the mentioned protocols. When it boils down to it the reason for looking at such protocols is you want high switching capacity, low latency, and redundancy.  The current de facto standard of Spanning Tree Protocol (STP) simply is unable to meet the needs of modern layer2 networks.  TRILL addresses the problem of STP’s ability to only allow one network path between switches or ports.  STP prevents loops by managing active layer -2 paths.   TRILL applies Intermediate System-to-Intermediate System protocol (IS-IS), which is a layer3 routing protocol translated to Layer 2 devices.

For those who say TRILL is not the answer things like SPB also known as 802.1aq, and VXLAN are the alternative. A presentation at NANOG 50 in 2010 addressed some of the SPB vs TRILL debate. This presentation goes into great detail on the differences between the two.

The problem, which is one most folks overlook, is that you can only make a layer 2 network so flat.  The trend for a while, especially in data centers, is to flatten out the network. Is TRILL better? Is SPB better? The problem isn’t what is the better solution to use.  What needs to be addressed is the design philosophy behind why you need to use such things.   Having large Layer2 networks is generally a bad idea. Scaling issues can almost always be solved by Layer-3.

So, and this is where the philosophy starts, is TRILL, SPB, or even VXLAN for you? Yes, but with a very big asterisk. TRILL is one of those stop gap measures or one of those targeted things to use in specific instances. TRILL reduces complexity and makes layer-2 more robust when compared to MLAG. Where would you use such things? One common decision of whether to use TRILL or not comes in a virtualized environment such as VSPHERE.

Many vendors such as Juniper, have developed their own solutions to such things.  Juniper and their Virtual Chassis solution does away with spanning tree issues, which is what TRILL addresses.   Cisco has FabricPath, which is Cisco’s proprietary TRILL based solution. Keep in mind, this is still TRILL.   If you want to learn some more about Fabric Path this article by Joel Knight gets to the heart of Fabric path.

Many networks see VXLAN as their upgrade path.  VXLAN allows layer 2 to be stretched across layer 3 boundaries. If you are a “Microsoft person” you probably hear an awful lot about Network Virtualization using Generic Routing Encapsulation (NVGRE) which can encapsulate a layer two frame into IP.

The last thing to consider in this entire debate is how does Software Defined Networking (SDN) play into this. Many folks think controllers will make ECMP and MLAG easy to create and maintain. If centralized controllers have a complete view of the network there is no longer a need to run protocols such as TRILL.   The individual switch no longer makes the decision, the controller does.

Should you use Trill, VXLAN, or any of the others mentioned? If you have a large Layer-2 virtualized environment it might be something to consider.  Are you an ISP, there is very little case for running TRILL in anything other than your data center. Things such as Carrier Ethernet and MPLS are the way to go.