Switch VLAN configuration de-mystified

There seems to be a great deal of confusion when it comes to VLAN configuration across different platforms. In this article, I am going to clarify some often misunderstood terms and bad configurations I see.

Let’s start with some terms

Tagged Frame – A tagged frame includes a VLAN ID in the header.

Untagged Frame – A frame that does not include and VLAN ID in the header.

If you are unfamiliar with ethernet frame here is a good definition https://www.ionos.com/digitalguide/server/know-how/ethernet-frame/ or https://en.wikipedia.org/wiki/Ethernet_frame

These are often misunderstood and intermixed with ports which can be referred to as tagged or untagged. This is a term that is not entirely accurate. using the term “tagged port” is one of those misleading terms.

Switch ports are identified in the following ways. These definitions have to do with how the port handles frames. To understand how frames get handled, we must explain the two camps. The first is the cisco-like way and how almost everyone else does it. We will go over both philosophies.

Cisco Terms

Cisco has two port definitions.
Access Port
The first is an access port. If the port is in access mode, any frames coming into it will get marked with whatever VLAN the port is a member of. This configuration is reflected in the following configuration

switchport mode access
switchport access vlan 10

Another way to put this is the switch dumps all traffic coming into this port into the VLAN. In our example above, this would be VLAN 10.

Trunk Port
A trunk port is a port that only accepts tagged frames. If you forgot above, these frames have a VLAN id in the header. A trunk port can allow specific VLANs, which are referred to as pruning or allowing all VLANs. The following configuration puts the port in trunking mode and allows all tagged frames with a VLAN id to pass.

switchport mode trunk
switchport trunk allowed vlan all


So what does a Cisco port do when it gets an untagged frame? By default, it drops it. A cisco port will not pass untagged traffic by default. Learn this. Remember this! So what do you do if you need to pass untagged traffic across a trunk port? Cisco has what is called a native VLAN. Any untagged frame get dumped into this native VLAN.

switchport mode trunk
switchport trunk allowed vlan all
switchport trunk native vlan 10

The above code puts any untagged traffic into VLAN10. This configuration is a trunk with a native VLAN. There are a few things to remember about this.

  1. All untagged frame are turned into tagged frame on VLAN 10
  2. Only one VLAN can be the native VLAN.

Just about everyone else

The majority of the switch manufacturers are where folks start to get fuzzy. Many switch manufacturers have “hybrid” ports that can pass untagged and tagged traffic without a native VLAN. There are several caveats to this\ which we will discuss throughout this article.

A port that is a member of just one VLAN (aka access port)
This is normally done with the following command

switchport 0/1
pvid 10

This command can vary based upon the switch manufacturer, but most are similar. When you think of configuring these types of switches, consider whether the VLAN is tagged or untagged on the particular port. This port configuration is where the misnomer of a tagged or untagged port can confuse people. It’s not the port; it’s how the VLAN is addressed. the following is the same result but from a different switch platform requiring the PVID command.

switchport 0/1
switchport add vlan add 10 untagged
pvid 10

PVID Notes.
On some switches, if you set a port as untagged on VLAN 10 and excluded from all others, the switch will automatically tag the untagged incoming frames with the same VLAN id without requiring you to set the PVID. Other platforms require you to set the PVID. Some of the switches that don’t need you to set the PVID will allow you to set the PVID even if you don’t have to. This loose use of the pVID is where confusion in example configs can be an issue. It highly depends on the switch platform.

Port that passes multiple tagge VLANS (aka Trunk)

interface 0/15
switchport allowed vlan add 101, 102, 310 tagged

The example above allows the port to pass multiple tagged VLANs. This method is similar to a Cisco Trunk port. Only the VLANs defined are allowed to pass as tagged over the port.

Next example

interface 0/15
switchport allowed vlan add 101,102,310 tagged

In the above example, two things are going on. The first is that VLANS 101,102,and310 are allowed to pass through the port as tagged. Second, All other tagged traffic gets dropped. This dropping of non-specified VLANS is referred to as VLAN pruning.

An important thing to step and discuss is the VLAN database. In order for a switch to pass a tagged VLAN that VLAN must be understood by the switch. This can be referred to as adding it to the VLAN database or creating the VLAN. Even if the VLAN is not used on any ports on the switch, for the switch to pass it through the VLAN must be added. Creating the VLAN varies from switch to switch.

Tagged and Untagged commands on the same port

Up to now, this is all been pretty simple. We have done trunk ports, allowed VLANs on those trunk ports, done access ports, and done a single VLAN per port (access port/pvid). Let us get onto where I see people go wrong. Let’s take an example configuration and pick it apart

interface 0/15
switchport allowed vlan 101,102,310 tagged
switchport allowed vlan 10 untagged

So what is going on here? Are we allowing VLAN10 as untagged and VLANs 101,102, and 310 as tagged? No. There is no way to know what frames are VLAN 10 because those frames do not have a VLAN id in them. Otherwise, they would be tagged frames. With me? The confusing part is in the “allowed” command. You cannot allow multiple VLANs as untagged because there are no such things. Remember, an untagged frames does not carry a VLAN header. So you can’t pass VLAN 10,20, and 30 untagged because they do not exist as untagged VLANs. If they did, they would have a VLAN tag. We can pass untagged traffic on some platforms, but again, that has no vlan ID. This is the number one misconception I see.

The critical thing to remember is each port can only have ONE untagged VLAN. Untagging is almost the same as setting a native VLAN (more on this shortly). Any good switch manufacturer will only let you attach ONE VLAN as untagged to a port. If you have a Netonix, Dell, FS, Netgear, or any other switch which is configured the non-Cisco way, it will not let you assign more than one untagged VLAN to a port. This is the second thing I see in thinking. In practical use, the switch manufacturers, will not let you configure more than one untagged VLAN per port. Some will allow you to pass untagged traffic, but this is not the same. Why? No VLAN info in the frames header to distinguish VLANS.

Configurations can go wrong when an untagged vlan is defined as well as a PVID. This gets worse if these are separate IDs.

On some platforms this is configured as

interface 0/15
vlan particpation include 101,102,310
vlan tagging 101,102,310

Look familiar? “VLAN participation” means you allow VLANs 101,102, and 310 to pass through the port. VLAN tagging means it understands frames with tags of 101,102, 310, and no others. There is a subtle difference here that can confuse some. The port does not generate the tag; it understands it (tagging) and allows it to pass (include). I mention this because it is a subtle difference in how platforms change the syntax to do the same thing.

Even confusing more is how some switch platforms treat the following. This was explained under the PIVID Notes above

interface 0/15
switchport allowed vlan 101,102,310 tagged
switchport allowed vlan 10 untagged
pvid 10

PVID vs Native VLAN vs untagged
A native VLAN is not the same as a PVID, but it’s close. A PVID is the assigned VLAN of an access port. A native VLAN is configured in a trunk, if needed. In theory, when you would connect a trunk port from one switch to an access port with a defined PVID of another, communication for the native VLAN would be possible. In such a scenario, the native VLAN-ID doesn’t have to match the PVID. Native VLAN is mainly a cisco thing and not always compatible with other platforms. Native VLANs are also not mentioned in the 802.1q standard from what I read, only tagged vs. untagged.

To further complicate things, on some platforms, PVID has to do with ingress to a port while the untagged command has to do with egress. If an untagged frame comes in, on these platforms, the PVID puts it into the appropriate VLAN. If the frame leaves the interface the untagged command puts it into that VLAN. On other platforms, ingress and egress are not an issue.

Where configurations go wrong.

Here is a scenario I see a lot. The customer has a network of tagged and untagged traffic. Say they have a management VLAN of 10 like in our previous examples. they may have a port configured as the following

interface 0/15
switchport allowed vlan 101,102,310 tagged
switchport allowed vlan 10 untagged
pvid 10
1

In the above example, on most platforms, the PVID 101 command will cancel out the switch port allowed VLAN 10 and cancel out the 101 tag. This configuration will result in undesired behavior. All untagged traffic will be dumped into VLAN 101. On other switch platforms, the behavior will be different. The key takeaway here is don’t do your configurations like this.

The problem is compounded when the exit port on the switch looks like

interface 0/1
switchport allowed vlan 310 tagged
switchport allowed vlan 10 untagged
pvid 10

So what happens to our traffic tagged with VLAN 101 on interface 0/15? On some switches, it gets dropped by 0/1 because there is no allowed statement referencing 101. However, on other switches, the tag is not understood, but the software sees there is a PVID. it then strips the 101 VLAN tag and replaces it with 10. By doing this we are rewriting the VLAN IDs on the frames. This rewriting can cause looping and other weird things within the switch. Suddenly, we switched up which ports have which VLANs due to misconfiguration. Traffic may enter ports it is not supposed to.

Configuration Examples

Scenario 1
We have customers on a wireless access point. I want to pass a management VLAN of 10 to the customer CPE/SM, but I want to put customers into an untagged VLAN of 11. The router to the internet is in port 1. The AP, the customer is connected to is in port 2 of my switch. The following is my config.

Cisco

interface ethernet 0/1
switchport mode trunk
switchport trunk allowed vlan 10
switchport trunk native vlan 11

interfacce ethernet 0/2
switchport mode trunk
switchport trunk allowed vlan 10
switchport trunk native vlan 11

Non Cisco

interface 0/1
switchport allowed vlan 10 tagged
switchport allowed vlan 11 untagged
pvid 11

#(note pvid 11 could or could not be required due to your platform

interface 0/2
switchport allowed vlan 10 tagged
switchport allowed vlan 11 untagged
pvid 11

#(note pvid 11 could or could not be required due to your platform

Closing notes

Most WISPs will want any switch ports facing the customers to be a truck with tagged VLAN(s) and an untagged/native VLAN. This is so you can have a management VLAN that is tagged for your CPE/SMs/ONTS and an untagged for the customer router. The untagged VLAN means the customer can plug in a router without any special configuration.

A bridge is not the same as a switch. I see these used in the same context. A bridge does not understand mac addresses, VLANs, or Ip addresses. A switch understands all of this. bridging ports together is different than adding VLANs to a switch.

One of the reasons I am a Cisco fan is the port is either access or a trunk. You can’t add a configuration that will make it ambitious. This can happen on other platforms where you add a tagged command, an untagged command, and a PVID. Now you have conflicting configurations producing undesired results.

References

https://www.ieee802.org/1/pages/802.1ad.html

https://en.wikipedia.org/wiki/VLAN

https://en.wikipedia.org/wiki/IEEE_802.1Q

https://www.expertnetworkconsultant.com/configuring/understanding-vlans-for-ccnp-switch/


Auto mapping your Cisco Network

This content is for Patreon subscribers of the j2 blog. Please consider becoming a Patreon subscriber for as little as $1 a month. This helps to provide higher quality content, more podcasts, and other goodies on this blog.
To view this content, you must be a member of Justin Wilson's Patreon
Already a qualifying Patreon member? Refresh to access this content.

Running Docker inside Cisco Catalyst 9000

https://blogs.cisco.com/networking/application-hosting-on-catalyst-9000-series-switches

Catalyst 9000 series of switches, Cisco’s flagship enterprise switching portfolio delivering Intent Based Networking (IBN) runs a modern, modular and model-driven operating system stack, Cisco IOS® XE. Powered by Intel x86 CPU, Catalyst 9000 series now supports secure Docker™ container based application hosting environment, starting with the Catalyst 9300 switches. Users now have the option to either build their own apps or host any off-the-shelf apps to enable network monitoring/troubleshooting, security or IoT related outcomes.

BGP, a single /24 and two diverse non-connected exit points

I am starting to see the following scenario more and more as IPv4 space is hard to get, but isn’t.

With ARIN it is still possible to get an IPv4 allotment. Many smaller ISPs qualify for a /24 and can get one if they wait long enough on the ARIN waiting list. a /24 of IPv4 space is the smallest block that 99% of the Internet allows to be advertised on the Capital I Internet. There are filter rules in place that drop smaller prefixes because that is the agreed upon norm.

So what happens if you are an ISP and you have a shiny new /24 but you have two networks which are not connected. Let’s look at our scenario.

The above network have no connectivity between the two of them on the internal side. These could be half way across the world or next door. If they were half way across the world it would make sense to try and get another /24. Maybe they are either side of a big mountain or one is down in a valley and there is no way to get a decent link between the two networks.

So what is a way you can use this /24 and still be able to assign IP addresses to both sides of the network? One way is to use a tunnel between your two edge routers.

Without the tunnel the scenario is traffic could come into network1, but if the IP is assigned on network 2 it will come back as unreachable. BGP is all about networks finding the shortest path to other networks. You don’t have much control over how networks find your public IP space if you have two providers advertising the same information. Some of the Internet will come in Network2 and some will come in Network1.

By running a tunnel between the two you can now subnet out that /24 into two eqal /25s and assign one /25 Network1 and one /25 to Network2 or however you want to. You can make the tunnel a GRE, EOIP, or other tunnel type. If I am using Mikrotik I prefer to use EOIP. If it’s another vendor I tend to use GRE.

Once the tunnel is established you can use static routing, OSPF, or your favorite IGP (interior Gateway Protocol) to “tell” one side about the routes on the other side. Let’s look at a fictional use.

In the above example our fictional ISP has an IPv4 block of 1.2.3.0/24. They have two networks separated by a tall mountain range in the center. It’s too cost prohibitive to run fiber or a wireless backhaul between the two networks so they have two different upstream providers. The ISP is advertising this /24 via BGP to Upstream1 from the Network 1 router. Network 2 router is also advertising the same /24 via BGP to Upstream 2.

We now create a Tunnel between the Mikrotiks. As mentioned before this can be EOIP, GRE, etc. We won’t go into the details of the tunnel but let’s assume the ISP is using Mikrotik. We create an EOIP tunnel (tons of tutorials out there) between Network 1 router and Network 2 router. Once this is established we will use 172.16.200.0/30 as our “Glue” on our tunnel interfaces at each side. Network 1 router gets 172.16.200.1/30. Network 2 router gets 172.16.200.2/30

To keep it simple we have a static route statement on the Network 1 Mikrotik router that looks like this:

/ip route add dst-address=1.2.3.129/25 gateway=172.16.200.2

This statement routes any traffic that comes in for 1.2.3.128/25 via ISP 1 to network1 across the tunnel to the Network 2 router. The Network 2 router then send it to the destination inside that side of the network.

Conversely, we have a similar statement in the Network 2 Mikrotik router

/ip route add dst-address=1.2.3.0/25 gateway=172.16.200.1

This statement routes any traffic that comes in for 1.2.3.0/25 via ISP 2 to network2 across the tunnel to the Network 2 router. The Network 2 router then send it to the destination inside that side of the network.

It’s as simple as that. You can apply this to any other vendor such as Cisco, Juniper, PFSense, etc. You also do not have to split the network into even /25’s like I did. You can choose to have os of the ips available on one side and route a /29 or something to the other side.

The major drawback of this scenario is you will takef a speed hit because if the traffic comes in one side and has to route across the tunnel it will have to go back out to the public internet and over to the other ISP.

#packetsdownrange

Preseem and Switches in switch centric design

Anyone who follows me knows I am a big fan of switch centric designs. This usually involves a router on a stick paired with a high port count switch. Recently I had a client that installed a Preseem appliance in their network.

Equipment used in this setup
-Dell R710 with a 4 Port SFP+ card running Preseem
-Cisco 3064-X 48 Port switch
-Maxxwave Vengeance router with dual QSF+ card and 4 Port SFP+ card

A visio diagram of how this looks

We have two transport links coming into the switch on the left. These are dumped into VLANs 506 and 507. We then come out of the switch into the Preseem box via 2 SFP+ ports, one for each VLAN. In this case, we just used DAC cables In the future, we can turn these into trunk ports to pass more VLANS through.

The data then leaves the Preseem box over dual SFP fibers directly into the router’s SFP+ ports. If the Preseem appliance fails we have a secondary OSPF/IBGP path from the router’s 40 GIG QSFP down to the switch. This is a bypass in case the Preseem appliance hardware fails.

If you start flowing more than 10 Gigs through a single link you can upgrade to more SFP+ ports into your appliance and a 40 Gig QSFP+ card. You then link the appliance to the spare QSFP port on your router.

Learning, Certifications and the WISP

One of the most asked questions which come up in the xISP world is “How do I learn this stuff?”.   Depending on who you ask this could be a lengthy answer or a simple one-sentence answer.  Before we answer the question, let’s dive into why the answer is complicated.

In many enterprise environments, there is usually pretty standard deployment of networking hardware.  Typically this is from a certain vendor.  There are many factors involved. in why this is.  The first is the total Cost of Ownership (TCO).  It almost always costs less to support one product than to support multiples.  Things like staff training are usually a big factor.  If you are running Cisco it’s cheaper to train and keep updated on just Cisco rather than Cisco and another vendor.

Another factor involved is the economies of scale.  Buying all your gear from a certain vendor allows you to leverage buying power. Quantity discounts in other words.  You can commit to buying a product over time or all at once.

So, to answer this question in simple terms.  If your network runs Mikrotik, go to a Mikrotik training course.  If you run Ubiquiti go to a Ubiquiti training class.

Now that the simple question has been answered, let’s move on to the complicated, and typically the real world answer and scenario.  Many of our xISP clients have gear from several vendors deployed.  They may have several different kinds of Wireless systems, a switch solution, a router solution, and different pieces in-between.  So where does a person start?

I recommend the following path. You can tweak this a little based on your learning style, skill level, and the gear you want to learn.

1. Start with the Cisco Certified Network Associate (CCNA) certification in Routing and Switching (R&S).  There are a ton of ways to study for this certification.   There are Bootcamps (not a huge fan of these for learning), iPhone and Android Apps (again these are more focused on getting the cert), online, books, and even youtube videos. Through the process of

studying for this certification, you will learn many things that will carry over to any vendor.  Things like subnetting, differences between broadcast and collision domains, and even some IPV6 in the newest tracks.  During the course of studying you will learn, and then reinforce that through practice tests and such.  Don’t necessarily focus on the goal of passing the test, focus on the content of the material.  I used to work with a guy who went into every test with the goal of passing at 100%.  This meant he had to know the material. CompTIA is a side path to the Cisco CCNA.  For reasons explained later, COMPTIA Network+ doesn’t necessarily work into my plan, especially when it comes to #3. I would recommend COMPTIA if you have never taken a certification test before.

2. Once you have the CCNA under your belt, take a course in a vendor you will be working

the most with.  At the end of this article, I am going to add links to some of the popular vendor certifications and then 3rd party folks who teach classes. One of the advantages of a 3rd party teacher is they are able to apply this to your real-world needs. If you are running Mikrotik, take a class in that. Let the certification be a by-product of that class.

3.Once you have completed #1 and #2 under your belt go back to Cisco for their Cisco Certifed Design Associate (CCDA). This is a very crucial step those on a learning path overlook.  Think of your networking knowledge as your end goal is to be able to build a house.  Steps one and two have given you general knowledge, you can now use tools, do some basic configuration.  But you can’t build a house without knowing what is involved in designing foundations,  what materials you need to use, how to compact the soil, etc.  Network design is no different. These are not things you can read in a manual on how to use the tool.  They also are not tool-specific.   Some of the things in the Cisco CCDA will be specific to Cisco, but overall it is a general learning track.  Just follow my philosophy in relationship to #1. Focus on the material.

Once you have all of this under your belt look into pulling in pieces of other knowledge. Understanding what is going on is key to your success.  If you understand what goes on with an IP packet, learning tools like Wireshark will be easier.  As you progress let things grow organically from this point.  Adding equipment in from a Vendor? Update your knowledge or press the new vendor for training options.  Branch out into some other areas , such as security, to add to your overall understanding.

WISP Based Traning Folks.
These companies and individuals provide WISP based training. Some of it is vendor focused. Some are not.  My advice is to ask questions. See if they are a fit for what your goals are.
-Connectivity Engineer
Butch Evans
Dennis Burgess
Rick Frey
Steve Discher
Baltic Networks

Vendor Certification Pages
Ubiquiti
Mikrotik
Cisco
Juniper
CWNA
CompTIA

If you provide training let me know and I will add you to this list.

Cisco and Verizon demonstrate multi-haul

As Internet traffic grows and becomes more dynamic, optical transport networks for sub-sea, terrestrial long haul and metro need more capacity. The ability to deploy capacity quickly is equally important to handle the increasingly dynamic nature of the traffic. The concept of a multi-haul transport platform, as introduced by Andrew Schmitt of Cignal AI, becomes very appealing for achieving this ability to scale with speed while maintaining operational simplicity – a single platform for all requirements. A critical element of the multi-haul optical platform is the flexibility of the coherent optics to be tuned to fine granularity in order to meet the reach-capacity target of any given network.

https://ciscocentral.blogspot.com/2019/02/cisco-and-verizon-to-demonstrate.html