The importance of phone numbers in a WISP

One of the things I see startup wisps do wrong is their use of phone numbers.  This is one of those details that is often overlooked but is critical. It’s critical not only for tracking but also for the sanity of everyone involved.  Let’s identify where many WISPs go wrong.

The typical startup wisp is a type A go-getter. This is what Entrepreneurs are by default.  Once they have a plan they jump head over heels in. Many may start with a simple phone number, but when they call a customer if they are on their way to do an install or something they end up using their phone number.  The problem is customers keep this cell phone.  If the office is closed they start texting or calling any number they have.  Some customers will be respectful of boundaries, but many will not.  If they are getting packet loss at 3 am they are calling and texting.  This problem compounds as you grow and you have multiple installers involved. You want customer issues tracked in some sort of ticket/CRM system. You also don’t want your employees ahev to answer customer texts or calls after hours if they aren’t being paid.  It’s one of the quickest ways for employees to get burnt out or say the incorrect things.

So how do you solve this? The simple buzzword answer is unified communications.  One of the easiest and cheapest is Google Voice. With Google Voice and others, you have a primary number. This is the number you give out to clients. They call this and it rings another phone or phones.  This can be an extension on the VOIP system it is a part of, another number, and/or cell phones.  Depending on the level of sophistication it can ring all the programmed numbers at once, or ring one, and move on to the next one. If no one answers it drops the caller into voice mail. With Google voice, the programmed numbers are all rang at once.

The inbound ringing is pretty standard.  The “trick” for the WISP is the outgoing calling. You want to be able to call a customer and have it come up as the main number’s caller ID, not your cell phone. Most PBX systems can be set up to do this with the extensions attached to them.  Cell phone calls are a little more complicated.  The way Google Voice solves this is through the use of forwarding numbers, You bring up the app, enter a number and it actually calls a different number.  Behind the scenes, it is using this forwarding number to “spoof” your number to the person you are calling.   Your phone is not calling the other party directly. Your phone calls this forwarding number behind the scenes and works it all out on the backend.

Other vendors have Apps which do similar functions. Asterisk has their DISA function.  Once you have these functions setup it boils down to training and processes.  Your installers need to remember to use the app or the function when calling customers.  As the company grows, a way to help this situation is for employees to not use personal cell phones.  If a company provides a cell phone the employee can customize voicemail, or even forward no answers to the help desk should a customer get the cell phone.

Hope this helps one of the glaring issues a startup faces.

Why WPA is not encrypting your traffic

There was a Facebook discussion that popped up tonight about how a WISP answers the question “Is your network secure?” There were many good answers and the notion of WEP vs WPA was brought up.

In today’s society, you need end-to-end encryption for data to be secure. An ISP has no control over where the customer traffic is going. Thus, by default, the ISP has no control over customer traffic being secure.  “But Justin, I run WPA on all my aps and backhauls, so my network is secure.”  Again, think about end-to-end connectivity. Every one of your access points can be encrypted, and every one of your backhauls can be encrypted, but what happens when an attacker breaks into your wiring closet and installs a sniffer on a router or switch port?What most people forget is that WPA key encryption is only going on between the router/ap and the user device.  “But I lock down all my ports.” you say.  Okay, what about your upstream? Who is to say your upstream provider doesn’t have a port mirror running that dumps all your customer traffic somewhere.  “Okay, I will just run encrypted tunnels across my entire network!. Ha! let’s see you tear down that argument!”. Again, what happens when it leaves your network?  The encryption stops at the endpoint, which is the edge of your network.

Another thing everyone hears about is hotspots. Every so often the news runs a fear piece on unsecured hotspots.  This is the same concept.  If you connect to an unsecured hotspot, it is not much different than connecting to a hotspot where the WPA2 key is on a sign behind the cashier at the local coffee shop. The only difference is the “hacker” has an easier time grabbing any unsecured traffic you are sending. Notice I said unsecured.  If you are using SSL to connect to a bank site that session is sent over an encrypted session.  No sniffing going on there.  If you have an encrypted VPN the possibility of traffic being sniffed is next to none. I say next to none because certain types of VPNs are more secure than others. Does that mean the ISP providing the Internet to feed that hotspot is insecure? There is no feasible way for the ISP to provide end to end security of user traffic on the open Internet.

These arguments are why things like SSL and VPNs exist. Google Chrome is now expecting all websites to be SSL enabled to be marked as secure. VPNs can ensure end-to-end security, but only between two points.  Eventually, you will have to leave the safety and venture out into the wild west of the internet.  Things like Intranets exist so users can have access to information but still be protected. Even most of that is over encrypted SSL these days so someone can’t install a sniffer in the basement.

So what is a WISP supposed to say about security? The WISP is no more secure than any other ISP, nor are then any less secure.  The real security comes from the customer. Things like making sure their devices are up-to-date on security patches.  This includes the often forgotten router. Things like secure passwords, paying attention to browser warnings, e-mail awareness, and other things are where the real user security lies. VPN connections to work. Using SSL ports on e-mail. Using SSH and Secure RDP for network admins. Firewalls can help, but they don’t encrypt the traffic. Does all traffic need encrypted? no.

Cambium ePMP 3000 features

An older post but still relative. Figured I would re-post this.

Cambium and CTIconnecxt put on a webinar about ePMP 3000 today.  This should be available online at one point.  Look for it in the Cambium forums.

Some notes I took
-ePMP 3000 offers Simultaneous MIMO downlink transmission
-You will be able to use the beamsteering antenna with the 3000.  Cambium is working on the software to make this work.
-3000 has a dedicated receiver chip.  This allows you to run the spectrum analyzer in realtime. Also has “edetect on steroids” which shows more information than the current edetect.
-Sector is a 4X4 90 degree sector with beamforming. Achieves and extra 3db in the downlink.

Beamforming vs Beamsteering
Beamsteering is for dealing with interference.
Beamforming is for downlink gain.

-Cambium mentioned the concept of Azimuth Delta.  This is groups of SMs in terms of how the AP talks to groups.  The gave an example on a google earth plot.  In a nutshell, when you have gain in one direction it takes advantage of the null in different directions. More to this, but that is for another post.

-“Sounding” -Sends a special packet and gets feedback from the subscriber. Determines how the phase shift works and other things.

-Elevated clients beta is coming to make the elevated clients work with the 3000.

I hope distributors work out a smaller cold shrink for the sma connectors on the ePMP Ap radios.  Weatherproofing these properly will be an issue due to the close proximity of the connectors.  I have not seen the connectors on a sector to see how those will be.  This is where folks could take a page from the coldshring that comes with the Baicells gear or the cables with the integrated boot some distributors sell.

 

Antenna patterns and interference

This content is for Patreon subscribers of the j2 blog. Please consider becoming a Patreon subscriber for as little as $1 a month. This helps to provide higher quality content, more podcasts, and other goodies on this blog.
To view this content, you must be a member of Justin Wilson's Patreon at $0.01 or more
Already a qualifying Patreon member? Refresh to access this content.

Mikrotik mAP for the WISP installer

One of the problems installers run into on a few networks we manage is having the right tools to properly test a new install. Sure, an installer can run a test to speedtest.net to verify customers are getting their speed.  Anyone who has done this long enough knows speedtest.net can be unreliable and produce inconsistent results. So, what then? Or what happens if you need to by-pass customer equipment easily? Most installers break out their laptop, spend a few minutes messing with settings and then authenticating themselves onto the network. Sometimes this can be easy, other times it can be challenging.

mAP with extenral battery pack

In steps the Mikrotik mAP.
What you are about to read is based on a MUM presentation by Lorenzo Busatti from http://routing.wireless.academy/ with my own spin on it. You can read his entire presentation on the mAP in PDF at : https://mum.mikrotik.com//presentations/US16/presentation_3371_1462179397.pdf . The meat of what we are talking about in this article starts on Page 50. If you want to watch the video you can do so at https://www.youtube.com/watch?v=VeZetH9uX_Y . The focus of this article starts around 21:00.

I have taken Lorenzo’s idea and have several different versions based upon the network.  In most of our scenarios, the ethernet ports are what plug into the CPE or the customer’s equipment, and the technician connects to the mAP over wifi.  This post covers using the mAP as an installer tool, not a traveling router. Lorenzo covers the travel option quite well in his presentation.

In this post, we focus on networks which use PPPoE. PPPoE networks usually are the ones who take much time to set up to diagnose.   What we have done is set up an uncapped user profile that is available on every tower.  Authentication can be done with local secrets or via radius.  Depending on your IP design the user can get the same IP across the network, or have an IP that assigned to this user on each tower/routed segment. We could do an entire article on IP design.

On our Mikrotik, we setup ether1 to have a PPPoE client running on it.  When the installer plugs this into the customers CPE the mAP will automatically “dial-out” and authenticate using the technician user we talked about earlier.  Once this connection has is established, the mAP is set to turn on the red “PoE out” light on the mAP using the following code.

/system leds
add interface=pppoe-out1 leds=user-led type=interface-status

Note. Our PPPoE interface is the default “pppoe-out1″ name. If you modify this, you will need to modify the led setup as well to match.

The red light gives the technician a visual indicator they have authenticated and should have internet. At the very least their mAP has authenticated with PPPoE. There are netwatch scripts mentioned in the above presentation which can kick on another LED indicating true internet reachability or other functions.  In our case, we can assume if the unit authenticates with the tower, then internet to the tower is up.  While this isn’t always the case if the Internet is down to the tower you quickly know or the NOC quickly knows.  At least you hope so. We chose the PoE out led because we are not using POE on this setup and a red light is noticeable.

Once the technician has a connection they can connect to an SSID set aside for testing.  In our case, we have set aside a “COMPANY_TECH” SSID. The tech connects to this on their laptop, and they are online.  Since this is a static profile, you can set it up just like a typical customer, or you can give the tech user access to routers, APs or other devices.  Our philosophy is you set up this SSID to mimic what a customer account experiences as closely as possible.  It goes through the same firewall rules and ques just like a typical customer.

To further enhance our tool we can set up a VPN.  This VPN can is accessible from the laptop with a second SSID named “COMPANY_VPN”. Once the technician switches over to this SSID they have access, over a preconfigured VPN on the mAP, to the network, from where they can access things customers can not, or at least should not be able to access. Many modern networks put APs, and infrastructure on separate VLANs not reachable from customer subnets.  The VPN comes in handy here. You can access these things without changing security. If you plan on using this router internally, the type of VPN you choose is not as important as if you plan to modify the config so you can travel as is the case with the above MUM presentation. If you plan to travel an SSTP VPN is the most compatible.  If it’s just inside your network, I would suggest an l2tp connection with IPsec.

Our third configuration on this is to set up the second ethernet port to be a DHCP client.  This setup is handy for plugging into the customer router for testing or for places where DHCP is the method of access, for example, behind a Baicells UE.  If your network does not use PPPoE, you could have one ethernet be a DHCP client, and the other be a DHCP server. We have found having the technicians connect wirelessly makes their lives easier.  They can plug the unit in and not have to worry about cables being too short, or getting behind a desk several times to plug and unplug things.

So why go through all this trouble?
One of the first things you learn in troubleshooting is to eliminate as many variables as you can. By plugging this into your CPE, you have a known baseline to do testing. You eliminate things such as customer routers, customer PCs, and premise wiring.  The mAP is plugged directly in CPE, whether it be wired or wireless. Experience has shown us many of the troubles customers experience are traced back to their router. Even if you provide the router, this can eliminate or point to that router as being a source of the problem if a technician needs to visit the customer.

Secondly, the mAP allows us to see and do more than your typical router. From the mAP we can run the Mikrotik bandwidth test tool from it to the closest router, to the next router inlines, all the way out to the internet. A while back I did an article titled “The Problem with Speedteststs“.  This article explains many of the issues testing just using speedtest.net or other sites.  Being able to do these kinds of tests is invaluable.  If there are four Mikrotik routers between the customer and the edge of your network all four of them can be tested independently. If you have a known good host outside your network, such as the one we provide to our clients, then you can also test against that. 

Having a Mikrotik test tool like this also allows you access to better logging and diagnostics.   You can easily see if the ethernet is negotiating at 100 meg or a Gig.  You can do wireless scans to see how noisy or busy 2.4GHZ is.  You have easy to understand ping and traceroute tools.  You also have a remote diagnostic tool which engineers can remote into easily to perform tests and capture readings.

Thirdly, the mAP allows the installer to establish a good known baseline at the time of install.  You are not reliant on just a CPE to AP test, or a speedtest.net test.

How do we make this portable?
You may have noticed in my above pictures I have an external battery pack hooked up to my mAP.   I am a fan of the Anker battery packs

Distributors such as ISP Supplies and CTIconnect have the mAP.

Finally, you will need a USB to MicroUSB cable

If you want you can add some double sided tape to hold the mAP to the battery pack for a neat package. I like the shorter cable referenced above in order to have a neat and manageable setup.

No matter what gear you use for delivering Internet to your customers, the mAP can be an invaluable troubleshooting tool for your field staff. I will be posting configs for Patreon and subscribers to download and configure their mAPs for this type of setup, as well as a road warrior setup. In the meantime, we do offer a setup service for $200, which includes the mAP, battery, USB cable and customized configuration for you.

Some WISP CPE grounding

A discussion which comes up over and over in the WISP space is grounding and proper installation of customer CPE. The folks at perfect-10 (https://www.perfect-10.tv/) were a vendor at #WISPAPALOOZA2018.  One of the best things I have seen them in a long time is the below photo they created. This is a great illustration of how a proper CPE goes.

Importance of PIM in WISP LTE deployments

PIM sweeps are a common thing in the Cellular field.   One of the first questions folks often ask is what is a PIM sweep? If you think of PIM testing as a passive test and line sweeping as an active test that is a good start.  PIM testing looks for problems with things like connectors, cables, and other “layer 1” items.  A PIM test is not a line sweep. Line sweeping measures the signal losses and reflections of the transmission system. this is typically VSWR.  A line sweep is an active test. It can not detect the same things a PIM test can.  Many HAM radio folks are familiar with a line sweep where the reflected power is measure in an antenna system. In a line sweep you deal with reflected power and all that.

What does a PIM test do?

When you do a PIM test typical two high power signals are injected into the antenna line.  You can actually pass a sweep test but not a PIM test.

I won’t go into PIM tests very much because you need high dollar units such as those from Anritsu and Kaelus. These cost 10’s of thousands of dollars new.  Sometimes you can find these used.  However, the next thing you will run into is understanding the output of such a device.  Cell crews go to week long certification classes to become a PIM certified tech from Anritsu and others.

What causes a PIM test to fail?

According to Kaelus the most common problems are:

• Contaminated surfaces or contacts due to dirt, dust, moisture or oxidation.
• Loose mechanical junctions due to inadequate torque, poor alignment or poorly prepared contact surfaces.
• Loose mechanical junctions caused transportation shock or vibration .
• Metal flakes or shavings inside RF connections.
• Poorly prepared RF connections
•Trapped dielectric materials (adhesives, foam, etc.)
•Cracks or distortions at the end of the outer conductor of coaxial cables caused by over tightening the back nut during installation.
• Solid inner conductors distorted in the preparation process causing these to be out of round or tapered over the mating length.
• Hollow inner conductors excessively enlarged or made oval during the preparation process.

Why does cable matter?

Cables do not typically cause PIM, but poorly terminated or damaged cables can and do cause problems.

Cables with Seams can cause issues.  The seam can corrode.  Plated copper, found in cheaper cables, can break away from the aluminum core. This actually allows small amounts of flaking to happen between the connector and the core of the cable.  This will cause PIM issues and is very hard to diagnose. Imagine little flakes inside a connector. You don’t see them until you break open the connector, and even then they may be pretty little flakes.

Cables can change their physical configuration as temperature varies. For instance, sunshine can warm cables, changing their electrical length. A cable that happens to be the right length to cancel out PIM when cool may show strong PIM after changing its length on a warm day, or, it can work the other way around, good when hot and bad when cold. In addition, the physical change in length can make a formerly good connection into a poor one, also generating PIM. Other environmental factors such as water in the connector or cable can be an issue, as with any RF setup.

I think I have PIM issues. What are some indications?

PIM often shows up as poor statistics from the affected antenna. One of the first and most direct indications of PIM can be seen in cells with two receive paths. If the noise floor is not equal between the two paths, the cause is likely PIM generated inside the noisy receive path.

How Do I prevent PIM issues?

Cable quality and connector quality are one of the biggest factors in the PIM quality of a LTE system.  Many WISPs are used to making their own LMR cables and putting on their own connectors.  There is a difference between a low PIM LMR-400 cable and normal LMR-400.  Same for connectors.  One of the recommendations today was to use 1/2” superflex heliax.

The easy recommendation is to buy pre-made cables that have already been PIM certified.  In a typical WISP setup, you do not have lots and lot of components in your setup. Buy already certified components from your distributors that are “Low PIM rated”.

ALG 5GHZ backhaul before and after

If you are looking for a U.S. stocking distributor of ALG products check out ISP Supplies
https://www.ispsupplies.com/search?keywords=alg&order=relevance:asc

The following are results from a series of tests of AGLcom’s parabolic dish antennas on an existing link that is 5.7 miles long. The link typically passes 80-90Mbs with a TX capacity of 140 Mbs and radios used are Ubiquiti AF5X operating at 5218 Mhz.  A full PDF with better Readability can be downloaded here..

The tests were taken in stages:

  1. 1)  The normal performance of the link was recorded.
  2. 2)  The 2′ dish at one end, B, was replaced with the AGLcom, C, dish and the link reestablished.The link performance was recorded.
  3. 3)  The 2′ dish at the other end, A, was replaced with the AGLcom, D, dish and the link reestablished. The link performance was recorded.
  4. 4)  The setting on the AF5xs were adjusted to optimize the link performance with data recorded.
  5. 5)  The 2′ dish, B was put back in the link and the performance was recorded.
  6. 6)  The ACLcom C was put back into place.

The tables below do not follow the test order as the third line of data was actually the last test performed.

Antennas:

A-Jirous JRC-29EX MIMO
B-Jirous JRC-29EX MIMO C-AGLcom – PS-6100-30-06-DP D-AGLcom – PS-6100-29-06-DP-UHP

Results:

Table 1 is the signal strength results of the various dishes on the link. The first line, A-B, is the original Jirous to Jirous. A is the first two columns of the link and are the A side and the last two columns are the B side on the link. What is of interest is that exchanging B to C in the second line brought the signal deviation between the channels to only 1db and 0 db as seen in Table 2. The third line was a result of replacing the horn on the A dish and optimizing the setting on the AF5X radios. This changed the signal by around 7db and improved the link capacity, Table 3. Clearly, the A dish had a problem with the original horn.

In the fourth line, D-B, the signal strength improved as well at the signal deviation on the two channels, Table 2 first two columns. This link was not optimized. The fifth line, D-C is both AGLcom dishes which improved the bandwidth, Table 3, and the signal deviations, Table 2. The final line, D-C, was the previous line optimized. The signal strengths moved closer together and the bandwidth improved.

Link Ch0 Ch1 Ch0 Ch1

  1. A-B  -73 -76
  2. A-C  -73 -74

A*-C -64 -66

  1. D-B  -63 -62
  2. D-C  -62 -62

D*-C -60 -60

-70 -74 -71 -71 -65 -66 -59 -59 -58 -58 -61 -61

Signal Strength (* optimized data) Table 1

Table 2 has four data columns, the first two being the measured results and the latter two being the measured difference from theory. The Jirous and AF5X calculators were used for the theory signals. Clearly the signal approached the theoritical limit with the optimization and with the change of dishes. The optimization improved the signal by ~9db for the link that we replaced the horn on the Jirous and by ~2db for the AGLcom link.

Link dSig dSig A-B 3 4 A-C 1 0 A*-C 2 1 D-B -1 0 D-C 0 0 D*-C 0 0

dSig dSig -16.5 -17.4 -17.0 -15.0 -8.0 -9.0 -13.3 -5.3 -7.0 -4.3 -5.0 -6.0

Signal strength variation from theory Table 2

The band width improvement was more obvious, Table 3, from 22 Mbs to 39 Mbs for the RX and 144 Mbs to 141 Mbs TX for the link with the horn replacement. The bandwidth improvement for the optimization of the AGLcom link was from 61Mbs to 66Mbs RX and from 211Mbs to 267Mbs for TX.

The bandwidth improvement from the original, optimized link to the AGLcom link is from 61Mbs RX to 67Mbs and from 210Mbs TX to 267Mbs. There is a clear improvement for the AGLcom link over the Jirous link.

Link BW-RX

  1. A-B  22.5
  2. A-C  39.0

A*-C 60.9

  1. D-B  61.4
  2. D-C  60.6

D*-C 66.6

BW-TX 144.6 141.4 210.0 211.0 215.0 267.6

Table 3

Conclusions:

The data supports a measurable improvement in both signal strength and bandwidth with the use of the AGLcom dishes. However, it is difficult to quantify the improvement. The Jirous dishes were identical whereas the AGLcom dishes were not. One of the jirous dishes was under performing initially but was repaired for the last tests. Additional testing is needed to provide accurate data analysis and performance comparison. The best performance tests would involve identical AGLcom dishes, ideally two links, one each of both types of dishes.