Learning, Certifications and the WISP

One of the most asked questions which come up in the xISP world is “How do I learn this stuff?”.   Depending on who you ask this could be a lengthy answer or a simple one-sentence answer.  Before we answer the question, let’s dive into why the answer is complicated.

In many enterprise environments, there is usually pretty standard deployment of networking hardware.  Typically this is from a certain vendor.  There are many factors involved. in why this is.  The first is the total Cost of Ownership (TCO).  It almost always costs less to support one product than to support multiples.  Things like staff training are usually a big factor.  If you are running Cisco it’s cheaper to train and keep updated on just Cisco rather than Cisco and another vendor.

Another factor involved is the economies of scale.  Buying all your gear from a certain vendor allows you to leverage buying power. Quantity discounts in other words.  You can commit to buying a product over time or all at once.

So, to answer this question in simple terms.  If your network runs Mikrotik, go to a Mikrotik training course.  If you run Ubiquiti go to a Ubiquiti training class.

Now that the simple question has been answered, let’s move on to the complicated, and typically the real world answer and scenario.  Many of our xISP clients have gear from several vendors deployed.  They may have several different kinds of Wireless systems, a switch solution, a router solution, and different pieces in-between.  So where does a person start?

I recommend the following path. You can tweak this a little based on your learning style, skill level, and the gear you want to learn.

1. Start with the Cisco Certified Network Associate (CCNA) certification in Routing and Switching (R&S).  There are a ton of ways to study for this certification.   There are Bootcamps (not a huge fan of these for learning), iPhone and Android Apps (again these are more focused on getting the cert), online, books, and even youtube videos. Through the process of

studying for this certification, you will learn many things that will carry over to any vendor.  Things like subnetting, differences between broadcast and collision domains, and even some IPV6 in the newest tracks.  During the course of studying you will learn, and then reinforce that through practice tests and such.  Don’t necessarily focus on the goal of passing the test, focus on the content of the material.  I used to work with a guy who went into every test with the goal of passing at 100%.  This meant he had to know the material. CompTIA is a side path to the Cisco CCNA.  For reasons explained later, COMPTIA Network+ doesn’t necessarily work into my plan, especially when it comes to #3. I would recommend COMPTIA if you have never taken a certification test before.

2. Once you have the CCNA under your belt, take a course in a vendor you will be working

the most with.  At the end of this article, I am going to add links to some of the popular vendor certifications and then 3rd party folks who teach classes. One of the advantages of a 3rd party teacher is they are able to apply this to your real-world needs. If you are running Mikrotik, take a class in that. Let the certification be a by-product of that class.

3.Once you have completed #1 and #2 under your belt go back to Cisco for their Cisco Certifed Design Associate (CCDA). This is a very crucial step those on a learning path overlook.  Think of your networking knowledge as your end goal is to be able to build a house.  Steps one and two have given you general knowledge, you can now use tools, do some basic configuration.  But you can’t build a house without knowing what is involved in designing foundations,  what materials you need to use, how to compact the soil, etc.  Network design is no different. These are not things you can read in a manual on how to use the tool.  They also are not tool-specific.   Some of the things in the Cisco CCDA will be specific to Cisco, but overall it is a general learning track.  Just follow my philosophy in relationship to #1. Focus on the material.

Once you have all of this under your belt look into pulling in pieces of other knowledge. Understanding what is going on is key to your success.  If you understand what goes on with an IP packet, learning tools like Wireshark will be easier.  As you progress let things grow organically from this point.  Adding equipment in from a Vendor? Update your knowledge or press the new vendor for training options.  Branch out into some other areas , such as security, to add to your overall understanding.

WISP Based Traning Folks.
These companies and individuals provide WISP based training. Some of it is vendor focused. Some are not.  My advice is to ask questions. See if they are a fit for what your goals are.
-Connectivity Engineer
Butch Evans
Dennis Burgess
Rickey Frey
Steve Discher
Baltic Networks

Vendor Certification Pages
Ubiquiti
Mikrotik
Cisco
Juniper
CWNA
CompTIA

If you provide training let me know and I will add you to this list.

Mikrotik Router OS 6.46.2 is out

Notables from the changelog of Mikrotik RouterOS 6.46.2

*) console – prevent “flash” directory from being removed (introduced in v6.46);
*) crs305 – disable optical SFP/SFP+ module Tx power after disabling SFP+ interface;
*) defconf – fixed default configuration loading on RBwAPG-60adkit (introduced in v6.46);
*) lora – fixed packet sending when using “antenna-gain” higher than 5dB;
*) lte – fixed “cell-monitor” on R11e-LTE in 3G mode;
*) lte – fixed “earfcn” reporting on R11e-LTE6 in UMTS and GSM modes;
*) lte – report only valid info parameters on R11e-LTE6;
*) qsfp – do not report bogus monitoring readouts on modules without DDMI support;
*) qsfp – improved module monitoring readouts for DAC and break-out cables;
*) security – fixed vulnerability for routers with default password (limited to Wireless Wire), admin could login on startup with empty password before default configuration script was fully loaded;
*) system – fixed “*.auto.rsc” file execution (introduced in v6.46);
*) traffic-generator – improved memory handling on CHR;
*) winbox – fixed “Default Route Distance” default value when creating new LTE APN;

Full changelog at
https://mikrotik.com/download

Mikrotik mAP for the WISP installer

One of the problems installers run into on a few networks we manage is having the right tools to properly test a new install. Sure, an installer can run a test to speedtest.net to verify customers are getting their speed.  Anyone who has done this long enough knows speedtest.net can be unreliable and produce inconsistent results. So, what then? Or what happens if you need to by-pass customer equipment easily? Most installers break out their laptop, spend a few minutes messing with settings and then authenticating themselves onto the network. Sometimes this can be easy, other times it can be challenging.

mAP with extenral battery pack

In steps the Mikrotik mAP.
What you are about to read is based on a MUM presentation by Lorenzo Busatti from http://routing.wireless.academy/ with my own spin on it. You can read his entire presentation on the mAP in PDF at : https://mum.mikrotik.com//presentations/US16/presentation_3371_1462179397.pdf . The meat of what we are talking about in this article starts on Page 50. If you want to watch the video you can do so at https://www.youtube.com/watch?v=VeZetH9uX_Y . The focus of this article starts around 21:00.

I have taken Lorenzo’s idea and have several different versions based upon the network.  In most of our scenarios, the ethernet ports are what plug into the CPE or the customer’s equipment, and the technician connects to the mAP over wifi.  This post covers using the mAP as an installer tool, not a traveling router. Lorenzo covers the travel option quite well in his presentation.

In this post, we focus on networks which use PPPoE. PPPoE networks usually are the ones who take much time to set up to diagnose.   What we have done is set up an uncapped user profile that is available on every tower.  Authentication can be done with local secrets or via radius.  Depending on your IP design the user can get the same IP across the network, or have an IP that assigned to this user on each tower/routed segment. We could do an entire article on IP design.

On our Mikrotik, we setup ether1 to have a PPPoE client running on it.  When the installer plugs this into the customers CPE the mAP will automatically “dial-out” and authenticate using the technician user we talked about earlier.  Once this connection has is established, the mAP is set to turn on the red “PoE out” light on the mAP using the following code.

/system leds
add interface=pppoe-out1 leds=user-led type=interface-status

Note. Our PPPoE interface is the default “pppoe-out1″ name. If you modify this, you will need to modify the led setup as well to match.

The red light gives the technician a visual indicator they have authenticated and should have internet. At the very least their mAP has authenticated with PPPoE. There are netwatch scripts mentioned in the above presentation which can kick on another LED indicating true internet reachability or other functions.  In our case, we can assume if the unit authenticates with the tower, then internet to the tower is up.  While this isn’t always the case if the Internet is down to the tower you quickly know or the NOC quickly knows.  At least you hope so. We chose the PoE out led because we are not using POE on this setup and a red light is noticeable.

Once the technician has a connection they can connect to an SSID set aside for testing.  In our case, we have set aside a “COMPANY_TECH” SSID. The tech connects to this on their laptop, and they are online.  Since this is a static profile, you can set it up just like a typical customer, or you can give the tech user access to routers, APs or other devices.  Our philosophy is you set up this SSID to mimic what a customer account experiences as closely as possible.  It goes through the same firewall rules and ques just like a typical customer.

To further enhance our tool we can set up a VPN.  This VPN can is accessible from the laptop with a second SSID named “COMPANY_VPN”. Once the technician switches over to this SSID they have access, over a preconfigured VPN on the mAP, to the network, from where they can access things customers can not, or at least should not be able to access. Many modern networks put APs, and infrastructure on separate VLANs not reachable from customer subnets.  The VPN comes in handy here. You can access these things without changing security. If you plan on using this router internally, the type of VPN you choose is not as important as if you plan to modify the config so you can travel as is the case with the above MUM presentation. If you plan to travel an SSTP VPN is the most compatible.  If it’s just inside your network, I would suggest an l2tp connection with IPsec.

Our third configuration on this is to set up the second ethernet port to be a DHCP client.  This setup is handy for plugging into the customer router for testing or for places where DHCP is the method of access, for example, behind a Baicells UE.  If your network does not use PPPoE, you could have one ethernet be a DHCP client, and the other be a DHCP server. We have found having the technicians connect wirelessly makes their lives easier.  They can plug the unit in and not have to worry about cables being too short, or getting behind a desk several times to plug and unplug things.

So why go through all this trouble?
One of the first things you learn in troubleshooting is to eliminate as many variables as you can. By plugging this into your CPE, you have a known baseline to do testing. You eliminate things such as customer routers, customer PCs, and premise wiring.  The mAP is plugged directly in CPE, whether it be wired or wireless. Experience has shown us many of the troubles customers experience are traced back to their router. Even if you provide the router, this can eliminate or point to that router as being a source of the problem if a technician needs to visit the customer.

Secondly, the mAP allows us to see and do more than your typical router. From the mAP we can run the Mikrotik bandwidth test tool from it to the closest router, to the next router inlines, all the way out to the internet. A while back I did an article titled “The Problem with Speedteststs“.  This article explains many of the issues testing just using speedtest.net or other sites.  Being able to do these kinds of tests is invaluable.  If there are four Mikrotik routers between the customer and the edge of your network all four of them can be tested independently. If you have a known good host outside your network, such as the one we provide to our clients, then you can also test against that. 

Having a Mikrotik test tool like this also allows you access to better logging and diagnostics.   You can easily see if the ethernet is negotiating at 100 meg or a Gig.  You can do wireless scans to see how noisy or busy 2.4GHZ is.  You have easy to understand ping and traceroute tools.  You also have a remote diagnostic tool which engineers can remote into easily to perform tests and capture readings.

Thirdly, the mAP allows the installer to establish a good known baseline at the time of install.  You are not reliant on just a CPE to AP test, or a speedtest.net test.

How do we make this portable?
You may have noticed in my above pictures I have an external battery pack hooked up to my mAP.   I am a fan of the Anker battery packs

Distributors such as ISP Supplies and CTIconnect have the mAP.

Finally, you will need a USB to MicroUSB cable

If you want you can add some double sided tape to hold the mAP to the battery pack for a neat package. I like the shorter cable referenced above in order to have a neat and manageable setup.

No matter what gear you use for delivering Internet to your customers, the mAP can be an invaluable troubleshooting tool for your field staff. I will be posting configs for Patreon and subscribers to download and configure their mAPs for this type of setup, as well as a road warrior setup. In the meantime, we do offer a setup service for $200, which includes the mAP, battery, USB cable and customized configuration for you.

Cleaning Dude Database in Version 4

This content is for Patreon subscribers of the j2 blog. Please consider becoming a Patreon subscriber for as little as $1 a month. This helps to provide higher quality content, more podcasts, and other goodies on this blog.
To view this content, you must be a member of Justin Wilson's Patreon at $0.01 or more
Already a qualifying Patreon member? Refresh to access this content.

Updated Mikrotik firewall script from Rick Frey

Our friend Rick Frey has updated his Mikrotik Firewall script.  You can find it here
http://rickfreyconsulting.com/rfc-mikrotik-firewall-6-0-for-ipv4-free-version/

You will need a fairly beefy router to run all of this.  If you are an enterprise this will be very handy for protecting your corporate network.  If you are an ISP I would pick and choose some of the parts which apply to you.  Your infrastructure should already be on non accessible IP space so the need for this big of a firewall should not be necessary

Mikrotik RouterOS 6.46 is out

What’s new in 6.46 (2019-Dec-02 11:16):

Lots of fixes in this.  Many LTE, WInbox, SNMP fixes.  CRS fixes as well.
Notable Changes (not all but ones I think are worth pulling out). Full changelog can be foudn at https://www.mikrotik.com/download

*) backup – fixed automatic backup file generation when configuration reset by button;
*) backup – store automatically created backup file in “flash” directory;
*) bonding – correctly remove HW offloaded bonding with ARP monitoring;
*) bonding – properly handle MAC addresses when bonding WLAN interfaces;
*) bridge – disable/enable bridge port when setting bpdu-guard;
*) bridge – do not add bridge as untagged VLAN member when frame-types=admit-only-vlan-tagged;
*) bridge – do not add dynamically VLAN entry when changing “pvid” property for non-vlan aware bridge;
*) bridge – include whole VLAN-id in DHCP Option 82 message;
*) ccr – improved general system stability;
*) crs1xx/2xx – allow to set trunk port as mirroring target;
*) crs3xx – correctly handle L2MTU change;
*) crs3xx – do not send pause frames when ethernet “tx-flow-control” is disabled on CRS326/CRS328/CRS305 devices;
*) crs3xx – improved interface initialization;
*) crs3xx – improved switch-chip resource allocation on CRS317-1G-16S+, CRS309-1G-8S+, CRS312-4C+8XG, CRS326-24S+2Q+ devices;
*) crs3xx – improved system stability on CRS309-1G-8S+, CRS312-4C+8XG, CRS326-24S+2Q+ devices;
*) defconf – require “policy” permission to print default configuration;
*) dhcpv6-client – fixed timeout when doing rebind;
*) dhcpv6-client – properly update bind time when unused prefix received from the server;
*) dhcpv6-client – properly update IPv6 address on rebind;
*) dhcpv6-server – fixed logged error message when using “address-pool=static-only”;
*) dhcpv6-server – ignore prefix-hint from client’s DHCPDISCOVER if static prefix received from RADIUS;
*) dhcpv6-server – include “User-Name” parameter in accounting requests;
*) dhcpv6-server – made “calling-station-id” contain MAC address if DUID contains it;
*) dot1x – added “reject-vlan-id” server parameter (CLI only);
*) dot1x – added support for dynamic switch rules from RADIUS;
*) dot1x – added support for “mac-auth” authentication type (CLI only);
*) ethernet – automatically detect interface when using IP address for power-cycle-ping;
*) ethernet – do not enable interface after reboot that is already disabled;
*) ipsec – fixed DNS resolving when domain has only AAAA entries;
*) ipsec – fixed policy “sa-src-address” detection from “local-address” (introduced in v6.45);
*) ipv6 – changed “advertise-dns” default value to “yes”;
*) route – fixed area range summary route installation in VRF;
*) sniffer – allow filtering by packet size;
*) usb – general USB modem stability improvements;

IPV6 NAT-PT to communicate to an ipv4 only device

This content is for Patreon subscribers of the j2 blog. Please consider becoming a Patreon subscriber for as little as $1 a month. This helps to provide higher quality content, more podcasts, and other goodies on this blog.
To view this content, you must be a member of Justin Wilson's Patreon at $0.01 or more
Already a qualifying Patreon member? Refresh to access this content.