Equipment

Equipment manufactures, gear,and other topics

Tech Tip: New features Apple Mail for macOS Ventura

For those of you running Apple Mall under the new macOS Ventura (13) below are some of the new features.

Undo send. After you send an email, you have a few seconds to unsend it to make corrections or other changes. See Recall email with Undo Send.

Schedule with Send Later. You now have the option to schedule a time to send an email. See Write and send emails.

Remind me. If you don’t have time to respond to an email right away, you can set a time and date to receive a reminder and have the email moved back to the top of your inbox. See Use Remind Me to come back to emails later.

Follow-up. You can receive a notification to follow up on emails you’ve sent that have yet to receive a response. See Reply to, forward, or follow up on emails.

Missing recipients. If you forget to include recipients that you mention in an email, you receive a message before you send it asking if you want to add them.

Improved search. Smart search suggestions and corrections improve your searches and offer better content when you search in Mail. See Search for emails.

Filter your inbox with Focus. Specify Mail accounts that you can receive messages from when a Focus is on. See Use Focus filters in Mail.

The ever-evolving service provider

Some of you may have noticed a subtle difference in the title. In many of my previous posts and articles, wISP was written as WISP. Note the capitalization of W. This represents a shift from Wireless Internet Service Providers (WISPs) becoming a more hybrid approach to service delivery. What does this mean? Are Wireless providers going away? Read on, dear reader.

Mainly due to government definitions of what broadband is, speeds being delivered by service providers are being increased. In 2015 the FCC defined broadband as a download of 25 megs and an upload of 3 megs. Several wireless equipment manufacturers were able to come out with new point-to-multipoint radios in unlicensed and 3.65 frequencies to meet this demand.

Fast forward to the “Covid years.” Demand for broadband increased. Working from home has become more mainstream than it ever has. This is the time many WISPs shined. These operators could service up new areas and increase bandwidth in existing coverage areas quickly. As with many governmental dealings,

What does this mean for the wISP? wISPs will be transforming into an overall service provider to satisfy several needs and regulations. Delivery speed will be the number one focus on all new build-outs. Many politicians and government regulators are already suggesting a 100 meg service tier.

So how are wISPs evolving? Let’s jump into it.

Automation
Automation saves money. Saving money allows for more devliery devices, whether they be Access Points, OLTs, or switches). Automation makes customer interaction faster and more efficient. From signups to support ques, automation is becoming the key to optimizing these interactions. Companies like SBR Consulting LLC provide automation. Other companies like RemoteWinbox can automate managing your large Mikrotik network.

Add on services and vertical markets.
Most government grants require you to provide Voice as part of your offerings. Why not let companies like Atheral take this load off you. You can then concentrate on acquiring customers. Video services like Realchoice can make sense if your network supports their unique data demands.

Network Quality of Service
Modern access customers are demanding. Technology is always evolving, and so are data flows. Preseem and Cambium QOE are two companies that can apply policies to flows and data endpoints as well as other traffic manipulation. This allows you to optimize your network. Latency sensitive items such as VOIP can benefit from a QOS/QOE soluiion.

High-Speed multipoint wireless
Fiber takes time to build and is costly. Wireless makes sense in so many places. 802.11-based systems are still a viable option for rural and less dense areas. However, MU-MIMO systems from the likes of Tarana and Cambium Medusa are the next generation of fixed wireless systems delivering higher bandwidth. LTE players such as Nokia have solutions for the growing wISP.

Hybrid Networks
As customer demands increase, there have to be bigger pipes pushing bandwidth to the towers and aggregation points. Licensed links are pushing more and more bandwidth. Fiber-fed towers are also becoming a thing. This means the wISP is possibly building their own fiber infrastructure to support the gigabit and muti-gigabit clusters being installed on towers. One natural progression is , as the provider passes homes, it makes sense to use strands to provide some sort of Fiber To The Home (FTTH). This is an easier cost to absorb as it can be both revenue generating and business supporting at the same time.

Peering and Interconneciton
Internet customers tend to consume much of the same data repeatedly. Pulling this from geographically close locations speeds up the customer experience while reducing latency. Internet Exchange Points (IXPSs) bring more resiliency to a fragile Internet. Companies like FD-IX and Ohio-IX are independent exchange points.

these are just a few things the xISP (wISP, fISP, etc.) can look into to migrate their networks to the next level. Many of them mentioned are at WISPAPALOOZA 2022 in Las Vegas this week.

ISP news and notables

The Commerce Department can’t allocate $42.5 billion in grants until the FCC publishes better coverage data.
https://www.engadget.com/bipartisan-infrastructure-law-broadband-funding-delayed-180710457.html

Walmart and Streaming Services
https://techcrunch.com/2022/08/09/walmart-streaming-nyt-report/

Mikrotik August 2022 Newsletter
https://mt.lv/News107

Unifi Version 7.0
https://blog.ui.com/2022/03/01/unifi-network-version-7-0-introduces-revamped-settings-to-simplify-system-configuration/

Switch VLAN configuration de-mystified

There seems to be a great deal of confusion when it comes to VLAN configuration across different platforms. In this article, I am going to clarify some often misunderstood terms and bad configurations I see.

Let’s start with some terms

Tagged Frame – A tagged frame includes a VLAN ID in the header.

Untagged Frame – A frame that does not include and VLAN ID in the header.

If you are unfamiliar with ethernet frame here is a good definition https://www.ionos.com/digitalguide/server/know-how/ethernet-frame/ or https://en.wikipedia.org/wiki/Ethernet_frame

These are often misunderstood and intermixed with ports which can be referred to as tagged or untagged. This is a term that is not entirely accurate. using the term “tagged port” is one of those misleading terms.

Switch ports are identified in the following ways. These definitions have to do with how the port handles frames. To understand how frames get handled, we must explain the two camps. The first is the cisco-like way and how almost everyone else does it. We will go over both philosophies.

Cisco Terms

Cisco has two port definitions.
Access Port
The first is an access port. If the port is in access mode, any frames coming into it will get marked with whatever VLAN the port is a member of. This configuration is reflected in the following configuration

switchport mode access
switchport access vlan 10

Another way to put this is the switch dumps all traffic coming into this port into the VLAN. In our example above, this would be VLAN 10.

Trunk Port
A trunk port is a port that only accepts tagged frames. If you forgot above, these frames have a VLAN id in the header. A trunk port can allow specific VLANs, which are referred to as pruning or allowing all VLANs. The following configuration puts the port in trunking mode and allows all tagged frames with a VLAN id to pass.

switchport mode trunk
switchport trunk allowed vlan all


So what does a Cisco port do when it gets an untagged frame? By default, it drops it. A cisco port will not pass untagged traffic by default. Learn this. Remember this! So what do you do if you need to pass untagged traffic across a trunk port? Cisco has what is called a native VLAN. Any untagged frame get dumped into this native VLAN.

switchport mode trunk
switchport trunk allowed vlan all
switchport trunk native vlan 10

The above code puts any untagged traffic into VLAN10. This configuration is a trunk with a native VLAN. There are a few things to remember about this.

  1. All untagged frame are turned into tagged frame on VLAN 10
  2. Only one VLAN can be the native VLAN.

Just about everyone else

The majority of the switch manufacturers are where folks start to get fuzzy. Many switch manufacturers have “hybrid” ports that can pass untagged and tagged traffic without a native VLAN. There are several caveats to this\ which we will discuss throughout this article.

A port that is a member of just one VLAN (aka access port)
This is normally done with the following command

switchport 0/1
pvid 10

This command can vary based upon the switch manufacturer, but most are similar. When you think of configuring these types of switches, consider whether the VLAN is tagged or untagged on the particular port. This port configuration is where the misnomer of a tagged or untagged port can confuse people. It’s not the port; it’s how the VLAN is addressed. the following is the same result but from a different switch platform requiring the PVID command.

switchport 0/1
switchport add vlan add 10 untagged
pvid 10

PVID Notes.
On some switches, if you set a port as untagged on VLAN 10 and excluded from all others, the switch will automatically tag the untagged incoming frames with the same VLAN id without requiring you to set the PVID. Other platforms require you to set the PVID. Some of the switches that don’t need you to set the PVID will allow you to set the PVID even if you don’t have to. This loose use of the pVID is where confusion in example configs can be an issue. It highly depends on the switch platform.

Port that passes multiple tagge VLANS (aka Trunk)

interface 0/15
switchport allowed vlan add 101, 102, 310 tagged

The example above allows the port to pass multiple tagged VLANs. This method is similar to a Cisco Trunk port. Only the VLANs defined are allowed to pass as tagged over the port.

Next example

interface 0/15
switchport allowed vlan add 101,102,310 tagged

In the above example, two things are going on. The first is that VLANS 101,102,and310 are allowed to pass through the port as tagged. Second, All other tagged traffic gets dropped. This dropping of non-specified VLANS is referred to as VLAN pruning.

An important thing to step and discuss is the VLAN database. In order for a switch to pass a tagged VLAN that VLAN must be understood by the switch. This can be referred to as adding it to the VLAN database or creating the VLAN. Even if the VLAN is not used on any ports on the switch, for the switch to pass it through the VLAN must be added. Creating the VLAN varies from switch to switch.

Tagged and Untagged commands on the same port

Up to now, this is all been pretty simple. We have done trunk ports, allowed VLANs on those trunk ports, done access ports, and done a single VLAN per port (access port/pvid). Let us get onto where I see people go wrong. Let’s take an example configuration and pick it apart

interface 0/15
switchport allowed vlan 101,102,310 tagged
switchport allowed vlan 10 untagged

So what is going on here? Are we allowing VLAN10 as untagged and VLANs 101,102, and 310 as tagged? No. There is no way to know what frames are VLAN 10 because those frames do not have a VLAN id in them. Otherwise, they would be tagged frames. With me? The confusing part is in the “allowed” command. You cannot allow multiple VLANs as untagged because there are no such things. Remember, an untagged frames does not carry a VLAN header. So you can’t pass VLAN 10,20, and 30 untagged because they do not exist as untagged VLANs. If they did, they would have a VLAN tag. We can pass untagged traffic on some platforms, but again, that has no vlan ID. This is the number one misconception I see.

The critical thing to remember is each port can only have ONE untagged VLAN. Untagging is almost the same as setting a native VLAN (more on this shortly). Any good switch manufacturer will only let you attach ONE VLAN as untagged to a port. If you have a Netonix, Dell, FS, Netgear, or any other switch which is configured the non-Cisco way, it will not let you assign more than one untagged VLAN to a port. This is the second thing I see in thinking. In practical use, the switch manufacturers, will not let you configure more than one untagged VLAN per port. Some will allow you to pass untagged traffic, but this is not the same. Why? No VLAN info in the frames header to distinguish VLANS.

Configurations can go wrong when an untagged vlan is defined as well as a PVID. This gets worse if these are separate IDs.

On some platforms this is configured as

interface 0/15
vlan particpation include 101,102,310
vlan tagging 101,102,310

Look familiar? “VLAN participation” means you allow VLANs 101,102, and 310 to pass through the port. VLAN tagging means it understands frames with tags of 101,102, 310, and no others. There is a subtle difference here that can confuse some. The port does not generate the tag; it understands it (tagging) and allows it to pass (include). I mention this because it is a subtle difference in how platforms change the syntax to do the same thing.

Even confusing more is how some switch platforms treat the following. This was explained under the PIVID Notes above

interface 0/15
switchport allowed vlan 101,102,310 tagged
switchport allowed vlan 10 untagged
pvid 10

PVID vs Native VLAN vs untagged
A native VLAN is not the same as a PVID, but it’s close. A PVID is the assigned VLAN of an access port. A native VLAN is configured in a trunk, if needed. In theory, when you would connect a trunk port from one switch to an access port with a defined PVID of another, communication for the native VLAN would be possible. In such a scenario, the native VLAN-ID doesn’t have to match the PVID. Native VLAN is mainly a cisco thing and not always compatible with other platforms. Native VLANs are also not mentioned in the 802.1q standard from what I read, only tagged vs. untagged.

To further complicate things, on some platforms, PVID has to do with ingress to a port while the untagged command has to do with egress. If an untagged frame comes in, on these platforms, the PVID puts it into the appropriate VLAN. If the frame leaves the interface the untagged command puts it into that VLAN. On other platforms, ingress and egress are not an issue.

Where configurations go wrong.

Here is a scenario I see a lot. The customer has a network of tagged and untagged traffic. Say they have a management VLAN of 10 like in our previous examples. they may have a port configured as the following

interface 0/15
switchport allowed vlan 101,102,310 tagged
switchport allowed vlan 10 untagged
pvid 10
1

In the above example, on most platforms, the PVID 101 command will cancel out the switch port allowed VLAN 10 and cancel out the 101 tag. This configuration will result in undesired behavior. All untagged traffic will be dumped into VLAN 101. On other switch platforms, the behavior will be different. The key takeaway here is don’t do your configurations like this.

The problem is compounded when the exit port on the switch looks like

interface 0/1
switchport allowed vlan 310 tagged
switchport allowed vlan 10 untagged
pvid 10

So what happens to our traffic tagged with VLAN 101 on interface 0/15? On some switches, it gets dropped by 0/1 because there is no allowed statement referencing 101. However, on other switches, the tag is not understood, but the software sees there is a PVID. it then strips the 101 VLAN tag and replaces it with 10. By doing this we are rewriting the VLAN IDs on the frames. This rewriting can cause looping and other weird things within the switch. Suddenly, we switched up which ports have which VLANs due to misconfiguration. Traffic may enter ports it is not supposed to.

Configuration Examples

Scenario 1
We have customers on a wireless access point. I want to pass a management VLAN of 10 to the customer CPE/SM, but I want to put customers into an untagged VLAN of 11. The router to the internet is in port 1. The AP, the customer is connected to is in port 2 of my switch. The following is my config.

Cisco

interface ethernet 0/1
switchport mode trunk
switchport trunk allowed vlan 10
switchport trunk native vlan 11

interfacce ethernet 0/2
switchport mode trunk
switchport trunk allowed vlan 10
switchport trunk native vlan 11

Non Cisco

interface 0/1
switchport allowed vlan 10 tagged
switchport allowed vlan 11 untagged
pvid 11

#(note pvid 11 could or could not be required due to your platform

interface 0/2
switchport allowed vlan 10 tagged
switchport allowed vlan 11 untagged
pvid 11

#(note pvid 11 could or could not be required due to your platform

Closing notes

Most WISPs will want any switch ports facing the customers to be a truck with tagged VLAN(s) and an untagged/native VLAN. This is so you can have a management VLAN that is tagged for your CPE/SMs/ONTS and an untagged for the customer router. The untagged VLAN means the customer can plug in a router without any special configuration.

A bridge is not the same as a switch. I see these used in the same context. A bridge does not understand mac addresses, VLANs, or Ip addresses. A switch understands all of this. bridging ports together is different than adding VLANs to a switch.

One of the reasons I am a Cisco fan is the port is either access or a trunk. You can’t add a configuration that will make it ambitious. This can happen on other platforms where you add a tagged command, an untagged command, and a PVID. Now you have conflicting configurations producing undesired results.

References

https://www.ieee802.org/1/pages/802.1ad.html

https://en.wikipedia.org/wiki/VLAN

https://en.wikipedia.org/wiki/IEEE_802.1Q

https://www.expertnetworkconsultant.com/configuring/understanding-vlans-for-ccnp-switch/


Cambium QOE DPI

As an ISP you probably have an inkling of where your customers are pulling data from. With the new Cambium QOE software, you can know for sure.

What do you notice about this?
-Netflix
-Roku
-Youtube
-Amazon Video
-Hulu
All streaming services. Your customers are consuming more and more video content.

DNS in ISP networks and why you should care

If you are not familiar with how DNS works, please go and read this article and watch the accompanying video.

In this article we will talk about the different types of DNS servers an ISP will encounter and the design considerations of implementing them into your network. Let’s jump into the three types.

Cache servers
local DNS (LDNS) servers
authoritative DNS (ADNS) servers, of which the Root and Top Level Domain (gTLD) servers are special cases

Let’s go into detail on what each of these is.  We will start at the top and work our way down.

Authoritative Name Servers

An authoritative name server, just like the name says, is the authority on the website or resource it is responsible for. Without authoritative name servers, the DNS hierarchy would not function. You would have no way of telling an incorrect answer from a real answer.

A special kind of authoritative name server is the root nameservers, which are responsible for the Top Level Domains (TLD).  These are better known as .com,.net,.org, and so on.

Local DNS resolvers
These are servers run by Internet Service Providers (ISPs) and other entities such as Google (8.8.8.8) and Cloudflare (1.1.1.1). These are machines running BIND, Tiny DNS, or other software.  It is best practice to not run your authoritative name server on the same instance as your resolver.

Local DNS resolvers sit on the ISPs Internet backbone on fast connections to the Internet.

Caching servers
Caching servers are more typical in a larger ISP network with large concentrations of customers n certain geographic areas. .  Since DNS is such an important function, ISPs want their clients to have the quickest access to DNS lookups possible. In larger networks, this means putting caching servers as close to customers as possible. DNS lookups do not take up much bandwidth, but a slow DNS lookup can affect the user experience. If you have ever entered an address or clicked a link and experienced a “Looking up xyz.com” message for more than a second or two you are experiencing a slow DNS.

So why is this important in an ISP network?
As mentioned above the quicker you can get DNS lookups done and over with the better user experience your customers will have. You do not want your clients waiting 3-4 seconds for each lookup.  While you might have the fastest Internet on the planet, your customers will always be waiting on these requests, which will cause them to think things are slow.  And technically they would be right.

To speed up a large ISP network you will need to design DNS resolvers/caches into your design.  These can be at major traffic points on your network, or at each POP location. Part of this depends on your IP design, while some of it depends on the overall design of your infrastructure.  As networks flatten out due to things like VPLS and EVPN the need for lots of resolvers in an ISP network fades away. A general rule of thumb is to deploy DNS resolvers at each customer gateway.  This can be the wireless tower they receive their signal from, the local DSLAM, or a fiber node at the edge of your network.  By deploying resolvers at the customer gateway you are giving them fast access to cached DNS lookups. This can also lead to more complex troubleshooting should a cache server fail

What do I mean by cached?
A DNS resolver has two major functions in life.  It takes a request for a website, in our case j2sw.com, to look up the appropriate IP address and returns that information to the client.  The whole www stuff is for us, dumb humans. DNS translates www.j2sw.com to 209.209.44.4. This is what the computer and routers understand.

Secondly, a DNS resolver caches previous lookups.  This can be configured for minutes, hours, or days.  This speeds up any subsequent lookups because the resolver can serve it up locally without having to go clear across the internet to look it up and wait for the return. Once it has done one lookup, it will cache that lookup for any client who requests it after.

So, how does an ISP implement all of these?

Many modern routers have the ability to cache lookups and act as a DNS resolver.  Mikrotik, Cisco, and others all have this ability.   Even a Raspberry PI has enough horsepower to be a DNS resolver.  These can be deployed at each POP site and customers can point to them as their primary DNS servers.  This means their routers or PCs will look to the closest DNS resolver first.  If the resolver has the answer in its cache, it will serve it up right away.  If not, the resolver can go on to the next step up.

The next step is local resolvers the ISP typically runs.  There are lots of arguments for why an ISP should run their own instead of pointing to Google (8.8.8.8) or someone else. We won’t go into these here.  Let’s just say it is beneficial for the ISP to run their own resolvers with root hints.

It’s all about the trickle-down or trickle-up effect. The Local resolvers cache what they can for their subset of customers.  These resolvers then look to the name server resolvers, which have larger caches, and those name server resolvers finally look up the information directly.