Hurricane Electric now requires IRR and RPKI

If you are a Hurricane Electric customer you may be receiving e-mails like the following:

Dear AS394356,

Routing Security Report for ASXXX

Hurricane Electric cares about your routing security.  We filter all BGP sessions using prefix filters based on IRR and RPKI.

This report is being sent to help you identify prefixes which may need either their IRR or RPKI information created or updated 
and to also help you identify possibly hijacked routes you may be accepting and reannouncing.  

Routes with RPKI status INVALID_ASN strongly indicate a serious problem.

IPv4 SUMMARY

Routes accepted: 3
Routes rejected: 3
Routes with RPKI status VALID: 0
Routes with RPKI status INVALID: 0

IPv6 SUMMARY

Routes accepted: 1
Routes rejected: 0
Routes with RPKI status VALID: 0
Routes with RPKI status INVALID: 0

We currently do not have a valid as-set name for your network.  Please add an export line to your aut-num ASXXXX 
that references your as-set name.  For example,

export: to AS-ANY announce your-as-set-name

If you do not currently have an as-set, we recommend you create one named ASXXXX:AS-ALL

Your as-set should contain just your ASN and your customers' ASNs and/or as-sets (not your peers or upstream providers).

What does this mean for you as a service provider? If you use Hurricane Electric as transit or peer with them on an exchange you will need to enable RPKI and have routing registry objects. I did a tutorial based upon Arin which can be found at: https://blog.j2sw.com/networking/routing-registries-and-you/

In short you need to do the following:

  • Create a mntner object (equivalent of a user account) to give you the ability to create IRR objects in your selected IRR database
  • Create an aut-num to represent your autonomous system and describe its contact information (admin and technical) and your routing policy
  • Create an as-set to describe which autonous system numbers your peers should expect to see from you (namely your own and your transit customers)
  • Create a route/route6 object for every prefix originated from your network
  • Update your peeringdb profile to include your IRR peering policy
  • Generate RPKI https://www.arin.net/resources/manage/rpki/roa_request/#creating-a-roa-in-arin-online



New Speed Test server for Patreons

This content is for Patreon subscribers of the j2 blog. Please consider becoming a Patreon subscriber for as little as $1 a month. This helps to provide higher quality content, more podcasts, and other goodies on this blog.
To view this content, you must be a member of Justin Wilson's Patreon at $4 or more
Already a qualifying Patreon member? Refresh to access this content.

Some WordPress tips

If you are wanting to force non SSL to SSL. Add the following to your site’s .htaccess file

RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]

Set proper file permissions Script from https://www.ryadel.com/en/set-file-system-permissions-wordpress-web-site-centos-7-chmod/

#
# This script configures WordPress file permissions based on recommendations
# from http://codex.wordpress.org/Hardening_WordPress#File_permissions
#
# execute it with the following command:
# bash set-wordpress-permissions.sh /var/www/<site_folder>
#
OWNER=apache # <-- wordpress owner
GROUP=www # <-- wordpress group
ROOT=$1 # <-- wordpress root directory
 
# reset to safe defaults
find ${ROOT} -exec chown ${OWNER}:${GROUP} {} \;
find ${ROOT} -type d -exec chmod 755 {} \;
find ${ROOT} -type f -exec chmod 644 {} \;
 
# allow wordpress to manage wp-config.php (but prevent world access)
chgrp ${GROUP} ${ROOT}/wp-config.php
chmod 660 ${ROOT}/wp-config.php
 
# allow wordpress to manage wp-content
find ${ROOT}/wp-content -exec chgrp ${GROUP} {} \;
find ${ROOT}/wp-content -type d -exec chmod 775 {} \;
find ${ROOT}/wp-content -type f -exec chmod 664 {} \;

CCR1016 BGP route pull down

This morning I had a Mikrotik CCR1016 where I had to change the router ID, which caused all the sessions to reset. The following is a screenshot of the time it took to re-learn all of the peers. Obviously, the smaller prefixes were learned pretty quickly. It took about 10 minutes to learn two full IPv4 route tables and about 5 minutes to learn the IPv6 routing tables.

This is why I always get full routes plus a default from the upstream when it warrants full routes. This way I can have slow convergence time like this and still have traffic flowing.

OpenVPN, rooter project, and Mikrotik

Over the past couple of weeks, I have been fighting with getting an LTE device running The Rooter Project to establish an OpenVPN connection with a Mikrotik router. Apparently, OPENVPN is the only option when it comes to VPNs on The Rooter Project. For the purpose of this article, I am going to refer to the software as “the rooter”. This is just to denote the device running The Rooter Project software. In my case, this is a GL.iNET GL-X750 LTE device.

There are two parts to this setup. The OpenVPN setup on the Mikrotik and the setup on the rooter.

Mikrotik Setup

The Mikrotik setup is pretty straight forward. There are some great tutorials out there for a more in-depth setup. The RouterOS version I used for this setup is 6.47.

Creating Certificates
You will need to create 3 certificates on the Mikrotik.
1. cert_export_ca-certificate.crt
2.cert_export_client-certificate.crt
3.cert_export_client-certificate.key

/certificate
add name=ca-template common-name=example.com days-valid=3650 key-size=2048 key-usage=crl-sign,key-cert-sign
add name=server-template common-name=*.example.com days-valid=3650 key-size=2048 key-usage=digital-signature,key-encipherment,tls-server
add name=client-template common-name=client.example.com days-valid=3650 key-size=2048 key-usage=tls-client

Signing Certificates
Once you have created the above certificates you will need to sign them with the following

/certificate
sign ca-template name=ca-certificate
sign server-template name=server-certificate ca=ca-certificate
sign client-template name=client-certificate ca=ca-certificate

Exporting Certificates
Run the following commands to add a passphrase to your key certificate and export them to files

/certificate
export-certificate ca-certificate export-passphrase=""
export-certificate client-certificate export-passphrase=j2sw123com

This will give you three files: cert_export_ca-certificate.crtcert_export_client-certificate.crt, and cert_export_client-certificate.key. Download these out of “files” from the Mikrotik to the same computer you have access to the rooter on. I like to rename them to ca.crtclient.crt, and client.key so I can keep track of what is what.



Rooter Client Setup

Caveats
I could not find out how to make the operating system read a config file I would edit by hand. Even after a reboot, the config file would not be read. I am not sure if there is a command to read it into the running-config. If someone knows, let me know and that will make this process much easier.

client
dev tun
proto tcp
remote example.com 1194
resolv-retry infinite
nobind
persist-key
persist-tun
ca ca.crt
cert client.crt
key client.key
remote-cert-tls server
cipher AES-128-CBC
auth SHA1
auth-user-pass
redirect-gateway def1
verb 3

In my rooter, the config is in /var/etc. I would cat this occasionally to make sure I did not have any extra options turned on. Since I could not make my edits the file stick, I would make the below changes in the GUI and verify they matched up to my above file.

If your OpenVPN is using a username and password create a file named passowrd.txt and put the username on the first line and the password on the second.

You will need that file along with the three files you generate on the Mikrotik above.

Log in to the router and create you an open VPN instance. In my case, I named it Nexstream because this is who I was working for on this project. You can name it anything you want.

Click on edit and you will be brought to the following screens. Fill them out as shown.

When you get to the bottom this is where you upload your password.text and your cert and key files. If you see anything missing go to the bottom and select the field and click add.

Make sure to hit save and apply before proceeding. Click on “switch to advanced configuration”. Match up your configuration with the following screenshots, which match up with the above config file. You are just basically making the proper checkboxes to match the plain text config I posted above. Again, if anyone knows how to get OpenVPN. on the rooter to read the config in let me know.

Once you have the GUI part done and the certs uploaded to the rooter you will need to deal with the keyphrase via the command line. Simply SSH to the rooter. The below code is a generic code for changing the client.key to not ask for a passphrase anymore.

cd /etc/luci-uploads/
openssl.exe rsa -in client.key -out client.key
Enter pass phrase for client.key: j2sw123com
writing RSA key

Couple of things to note about the process.
1. Your location may vary. You must either be inside the directory with your keys or provide the path to the keys in the OpenSSL command

2.when I uploaded the keys it changed them to cbid.openvpn.FRIENDLYNAME.key.

what my actual code looked like to change the passphrase

cd /etc/luci-uploads/
openssl.exe rsa -in cbid.openvpn.vpnout.key -out cbid.openvpn.vpnout.key
Enter pass phrase for client.key: j2sw123com
writing RSA key

If everything goes well you will be rewarded with the following screen on your OpenVPN main page. If, for some reason, it does not start the system log is actually pretty informative on what is going on.

Briefing June 26th 2020 – API,5g,interconnection

Bots are awesome. They really are. So are APIs; both boost productivity by advancing automation, the exchange of business data and support decision making. If only everything was so perfect…. Unfortunately, 81% of organizations have reported attacks against their APIs, and 75% suffered bot attacks in a 12 month period.
Data centers will become even more pivotal to the digital economy over the next five years, which will see a meteoric rise in the volume of data traffic flowing through network intersections. This existing trend takes on new urgency with the COVID-19 pandemic, which has driven a massive shift to online services.

https://www.businessinsider.com/exclusive-massive-spying-on-users-of-googles-chrome-shows-new-security-weakness-2020-6
A newly discovered spyware effort attacked users through 32 million downloads of extensions to Google’s market-leading Chrome web browser, researchers at Awake Security told Reuters, highlighting the tech industry’s failure to protect browsers as they are used more for email, payroll and other sensitive functions.

The fifth generation of wireless connectivity, commonly referred to as 5G, is revolutionizing our digital lives by enabling unprecedented speed, bandwidth, processing and capacity at an industrial scale. For years now, we’ve been hearing about the almost science fiction-esque advancements 5G will bestow on our society, such as driverless carstelemedicine, factory automation, and smart cities.

Justin’s I.T. maintenance tip #7

When you are scheduling late maintenance schedule it anytime after 12:01AM so there is no confusion on the day. It’s easier to clarify 12:01AM on Friday the 10th than midnight on Friday. Folks tend to get confused when you say midnight. Is that midnight Friday into Saturday or Midnight Thursday into Friday?

If you want to do midnight do 12:01AM.

Takeaways for the customer from Tech Support

I have been in the ISP industry for close to thirty years now. One thing that has changed very little is the customer side of technical support. What I mean by this is, no matter if it is dial-up or fiber optic, customers will still have issues. They will still need to occasionally call their ISP for support in one fashion or another. This article is to hammer home how you the customer should always have something positive they walk away with, even if you can not solve their problem right away.

Duh! you say. This is common sense, you say. After seeing many posts from technicians, I have come to realize technical support folks fall into a rut. Let’s face, being on the front line of technical support operations is brutal. From the irate customers to the ones who can’t turn on their computer, it is stressful. It takes a unique mindset to be able to do technical support. When I managed a team of customer representatives back in the dial-up days, I had one very strict rule.

At the end of every interaction with the customer give them something positive to walk away with

You hope each time the positive is you fixed their issue(s). That is always the goal. If you can’t give them a clear resolution you should always give the customer a time for the next contact and something you are going to be doing for them in the meantime. Let’s go into some scenarios below.

You determine the customer had a bad CPE/NID, etc.
What you want to be able to do is give the customer a timeframe at the very least of when a technician can be at their place, or a new unit mailed to them.
“Let’s schedule a time we can get someone out there”
“Scheduling will be calling you in the next couple of hours to schedule a service call”
“I will get a new unit in the mail to you in Today’s/Tomorrow’s mail”

Each of the above gives the customer something concrete they can expect. It might not be the answer they want to hear, but it gives them something that is a step toward resolving their issues.

The customer has an issue on their end (bad computer, bad router that isn’t yours, etc)
Reinforce with the customer you are not abandoning them, just have exhausted your avenues of resolution.
“I think it might be your computer. I can give you the names of some computer repair shops in your area. Once they take a look we are here to make sure you get back online.
“I think your router may be bad. We bypassed it and things are working. I would suggest a new router. We have a guide or can help you once you have the new one.

In each of these cases, the key is to let the customer you are not pushing off the problem. It is something you can not fix and is impeding you from helping them.

One of the phrases I heard on a message board from an ISP was “Don’t call us again if you have this problem.” I know sometimes this can be said in jest, and to the right customer, you are okay. But normally, this is something you should never say to a customer. It gives them the impression you do not want to help them anymore. I often see this used in the content of a recurring problem, usually at the customer’s own doing.

What you want to say is. If this happens again here are the steps you need to do on your own. This will save you time as you won’t have to call us. If you still need to call us after you have done this feel free.

So remember, when talking to customer’s give them something they can either look forward to or something they can do if you can’t fix it in one interaction. Give them a positive takeaway with defined goals or time.


Mikrotik Routeros 7.0beta7

What’s new in 7.0beta7 (2020-Jun-3 16:31):

!) added Layer3 hardware offloading support for CRS317-1G-16S+RM more info here: https://wiki.mikrotik.com/wiki/Manual:CRS3xx_series_switches#L3_Hardware_Offloading
!) enabled BGP support with multicore peer processing (CLI only);
!) enabled RPKI support (CLI only);
!) ported features and fixes introduced in v6.47;
!) routing updates, complete status report: https://help.mikrotik.com/docs/display/ROS/v7+Routing+Protocol+Status
!) system kernel has been updated to version 5.6.3;
*) other minor fixes and improvements;