Source: cybersecurityventures.com
There is an attention getting headline if I ever saw one! Here is another little tidbit of info that a quick stroll with my old friend Google brought to my attention:
This is a drum I have been beating with increasing urgency for several years and it has slowly been coming to a head (drummer pun intended). This is something that affects all of us – radio stations, private individuals, corporations, everybody. Ultimately, it is not a matter of if, it is when – and how hard is it to recover. Doing our best to mitigate risk and having processes in place to allow a recovery are the topics for the day – and remember, I am NOT an IT professional by any stretch… Greg, our IT Manager, will tell you that I am the antithesis of an IT professional… I am more the reason why we NEED IT professionals!
So, all that said, why am I going down this particular rabbit hole? That’s easy – you can blame a small station in Lake Geneva, Wisconsin. Their GM, Nancy Douglass, did a great presentation at the Wisconsin Broadcasters Association Media Training Institute this past June and it really got my attention. WLKG got hit by ransomware one Friday night. They were doing everything right – backing things up, keeping backups isolated from the network, all the right things… until one night when somebody didn’t pull the backup media before leaving for the night. That is all it took – and Nancy is not shy about talking about the stress that resulted, or the cost to the station. Kudos to her for sharing that story, because, even though I’ve been preaching IT security for a while, learning that it can happen to a small single station in an unrated market between Milwaukee, WI and Rockford, IL was a bit of a shock. In the trades, you read about the big groups getting hit and just assume, “oh, we are too small to bother with”… but that is not the case.
The second reason I am bringing this up is quite simple – as I said, I do present on this on a fairly regular basis, three or four times a year, in various venues. As part of the research for these presentations, I quite frequently use an IoT search engine to see what is visible out there – because the biggest part of security is visibility… the less of it you have, the less of a target you will be. Thus, I get on the internet and see who I can see. Fortunately, it is quite rare for me to find more than half a dozen Nautel products, which is reassuring… although finding a demo transmitter from one of our product managers using a default password was good for a chuckle… and now he has a preset to remember me by (I wish I were able to use red text, but hello Matt!)
However, when I look at other brands, the results are not nearly so reassuring:
Here we see two popular audio codecs and an EAS unit… which brings me to the third reason for the subject of this article. On October 27th, the FCC released an NPRM (Notice of Proposed Rulemaking) specific to the security of the Emergency Alerting System. Among other things, this NPRM proposes requirements for stations to notify the FCC if they become aware their IT security (specifically relating to the EAS unit) has been compromised. It goes so far as to specify the time from when they’re compromised to when they must report it – or when they “should have known that an incident has occurred”. That’s a pretty big ball of potential liability right there. The NPRM goes further to suggest that there may be need for a Cybersecurity Risk Management plan. (download the FCC PDF here)
Understand that I am neither for nor against this – I think it could be potentially onerous for a small station with limited resources, but I do support the need for a plan. You need a plan – for backing up all critical systems, for limiting access to your networks and for restoring it all when things go sideways. So, how do we accomplish that?
As I said, I am not an IT expert, so this will be couched in very general terms and if I mess up a detail, I will own up to it in advance. However, I do get to speak to a lot of IT experts on a very regular basis, so I do hope I have absorbed a thing or two in these discussions!
BACKUPS:
First, a backup plan. How you accommodate this will depend on your own systems – if you have a music library, for example, that is in a digital format, it should be backed up. Traffic, billing, accounting, all should be backed up. If you are in an area where you need to save as played logs for auditing purposes, those also should be backed up. The one thing we all know about backups is that they take time – and the more you are backing up, the longer they take.
Fortunately, there are different levels of backups: a full backup is what the name implies, a complete backup of the files in question. It uses the most resources in terms of both time and storage space and the first backup you do, if you have not been doing them, will need to be a full backup, so be prepared for it to take a while. After that, you can choose between incremental or differential backups.
An incremental backup stores data that was created after the previous backup. So, if I do a backup on November 10, for example, then any files created after that would go in the next incremental backup. Any files that are added after that backup would be saved in the following one. In a system where you are rarely changing existing data, and mostly adding new data (say a music library, or as played logs), an incremental backup is a good solution.
Differential backups store any data that has changed – this might include files that were stored on the full backup, but have been edited since… billing, accounting, that sort of thing. You can find a pretty good explanation of the types of backups here.
Now, how often to perform the backups? Well, again, that is up to you – and it is a balancing act of risk vs. reward. As I said, backups take time and storage space. Fortunately, storage is fairly inexpensive these days, so we just invest the time. However, understand that if you do get hit by ransomware, or an unrecoverable virus of some sort, you will essentially lose everything that you have done from the day it happens to the last backup… so the time frame will be determined in large part by how much you are willing to lose. Again, those are decisions that will be specific to your operation – but they are decisions that must be made.
LIMITING ACCESS:
IT experts understand that security happens in the various layers of the OSI model (OSI = Open Systems Interconnection). OSI has seven layers that computer systems use to communicate over a network, starting with the physical (layer 1) and going to the application (layer 7). There are things that can be done in each layer, but that’s a series of articles on its own – for this missive, we will focus on only a few…
Source: bmc.com
Layer 1 – physical. This one seems self-explanatory – if you have a computer connected to the network and the hacker can get to it, they have access. So limit this access – require passwords and enforce lock screens, so that computers taken out on remote are not accessible if the host ends up leaving them on a table. Use biometrics on more portable devices (phones and tablets).
Layer 2 – Data link. This one is typically less prone to intervention for our purposes, but if you have a network switch in another office, or in another portion of the facility that’s more accessible, it is still a potential risk.
Layer 3 – this is the risky one for a lot of us who spend time on the road. This is the unsecured coffee shop wifi – you know the one, where you go in, get your latte, sit down, open your laptop, select the wifi and you are instantly connected… along with everybody else in range. This is where we risk “man in the middle” attacks… where the person at the table beside you is simulating the wifi on their own machine, you connect to that instead of the legitimate wifi, and now every bit you send and receive is stored on their device. Your best defense here is a VPN, because at least then, there is encryption involved that should render your data much less useful. For personal use, there are a lot of free or nearly free VPNs out there… I keep one on my personal devices for just this reason. Tech Radar does an annual report of their favourites. You can find the most recent report right here.
Skipping layers 4 through 6, the last note is layer 7… people, PLEASE stop using default user names and passwords! Even if you are protected by a VPN, it is just poor security… because there may come that day when you are not protected by the VPN.
One other note on limiting access – although I fought it in the beginning, I have become a fan of zero trust. Essentially, nobody gets access to anything until they prove they need it. There is no question that this can be an inconvenience. If I am on the road and somebody suggests a tool that can help me do my job, having to reach out to IT with a service ticket and schedule a time so that they can take over my laptop, download the tool, vet it, declare it safe and then allow me to use it, well, that is frustrating to say the least. But it also means that there is much greater chance that I won’t end up compromising the network – which would be vastly more frustrating!
RESTORING:
This is the big one… you got the malware/ransomware/trojan/virus, you know you have an air gapped (disconnected) backup, now what? This is where you really do need to engage a proper IT professional. Things need to be done in a sequence in order to ensure that the corrupted media is either removed and replaced or safely reformatted, that the necessary operating systems are reloaded, then the full backup restored, and the incremental/differential backups added in the right order. This is not the time to cut corners – but it is easy to get that smug, “we got this covered” feeling, hit the wrong button and end up corrupting your backup… and now you are back to square one.
Last note on this topic… do you remember the first clipping I attached way back at the beginning? The one about 25% of breaches being caused by social engineering? That is huge – that means that you are probably more likely to have your data compromised by somebody clicking on a pdf attached to an email that says, “Here is your Invoice”, than by the wifi at your local coffee shop. Spend some time teaching your staff that it is okay to question everything – if an email has a link or an attachment and you were not specifically told that you would be receiving it, take a few seconds to call the sender to verify. If it is not somebody you know, forward it to your IT professional (or the station engineer – sorry folks!). More and more, the hackers are getting better – the emails have good grammar now, proper spelling and in many cases look totally legitimate. Zero trust extends to this, also… make sure the folks in the station are all aware. Same with bringing in external media (a friend gave me a song on this USB stick)… treat it all as if it is compromised until you know for certain that it isn’t.
Those are my thoughts on IT security – the monologue ended up taking longer than I expected… but that is not much of a surprise!
One other cool thing happened recently – the VX Series beta units started going out! I am sure you have read about them elsewhere, but I had the pleasure of installing the first one at a little station I help out with on occasion, and it was really nice to see 1400W of RF coming out of a box smaller than a VS300! As I am writing this, that transmitter has been making power for 12 hours – here is looking at several thousand more!
On that note, folks, let’s all take care out there and I’ll see you next time – until then, be safe and happy engineering!
Jeff
Jeff Welton, has worked with Nautel for 30+ years. He is currently the Nautel Sales Manager for U.S. Central Region but previously he spent 16.5 years as a Nautel Customer Service Technician. A regular speaker and contributor on broadcast engineering, Jeff has been recognized with the following awards: 2020 NAB Radio Engineering Achievement Award; 2019 APRE Engineering Achievement Award; and 2018 SBE Educator of the Year Award.
Submissions for this Tips ‘n Tricks column are encouraged and if published you’ll receive a Nautel T-shirt. Submissions should be typed and emailed, with high resolution photos, to [email protected] using the subject line Tips ‘n Tricks.
