Is your customer's building automation system secure?
How can you make sure your BAS projects are cyber secure?
How do you know where to start?
In this week's episode, we discuss what cybersecurity is and what you need to know in order to securely execute BAS work.
Click here to download or listen to this episode now.
Resources mentioned in this episode
- https://courses.smartbuildingsacademy.com/products/it-for-bas-professionals
- ICS Cert
- CVE Index
- NIST 800 Series
Transcript
Phil Zito 0:00
This is the smart buildings Academy podcast with Phil Zito Episode 249. Hey folks, Phil Zito here and welcome to Episode 249 of the smart buildings Academy podcast. In this episode, we are going to be looking back at a previous episode on cybersecurity with everything going on in the world right now. And with the most recent attack on the oil pipeline, we are going to be focusing in on what is cybersecurity and what can building automation contractors and owners do in regards to cybersecurity to make their buildings more cyber secure. So we're going to be unpacking that. As always, everything can be found at podcast at smart buildings academy.com Ford slash 249. Once again, that is podcast at smart buildings academy.com Ford slash 249. I do want to take a second real quickly and point out that today is the last day that you can go and register for our VA s sales boot camp. So if you are new to sales, or you're looking to grow your skills in sales, this is our first ever live building automation sales training for VA s sales professionals. So if you're interested in learning how to design solutions, and become that technical expert for your customers, I encourage you to go check that out. You can find more information about it at podcast at smart buildings academy.com forward slash 249. With that being said, let's get in to the podcast episode. Personal cybersecurity, what is cybersecurity? So you you here's the word cybersecurity. And I want you don't do this if you're driving, but I want you to just close your eyes for a second and visualize what that word means to you. If you're like most folks, you close your eyes. you visualize cybersecurity and you think of hackers, you think of people attacking your building automation system sitting in a dark room somewhere in Russia or wherever. And hacking in right. Well, cyber security is not just hackers. It's much more than that it is securing cyber assets. So what are cyber assets? Right? So cyber assets are technology assets, network assets, things that utilize cyber technologies. So when we think about building automation systems, it used to be they were just a bunch of field controllers with a supervisory device. And they had their own little kind of standalone internal network. No one really logged into the computer sat in the closet somewhere we didn't have a whole lot of concerns. But now as we start to expand our technology, as we start to look at cloud technologies, as we start to look at IP controls, as we start to enter this world of IIT, we have to start considering cybersecurity. And for a lot of OEMs. And for a lot of just building automation contractors. This whole cybersecurity thing is very overwhelming because we are entering a technology market. here's here's the perspective, I want you to think about right? For the longest time we were at like 1960s technology. I mean, we weren't vacuum tubes and things like that. But we might as well been we had these gigantic panels that took up entire rooms and that stuff's still running in some places, folks on our on some of the Facebook groups are posting pictures of that. And so we've got that. And then we've got these OEMs out here who are bringing about you know, web based controls, IP controls, cloud based computing for our controls all these technologies. So we have leapfrogged into what I would say is, you know, 1990s it technology all, all the marketing out there makes it seem like all this stuff, so frickin advanced and amazing, when in reality, I mean, embedded controls on IP are things that have been around for quite a bit clouds been around for quite a bit. virtualization has been around for quite a bit. It's just we as an industry tend to lag technology by 10 to 20 years and we're just now catching up. The problem is, we're catching up, but
Phil Zito 4:32
the whole cybersecurity and the whole ecosystem of just threats and cybersecurity providers and all of that is so much further frickin mature than we are. We're like Stone Age, and they're, you know, in modern age, and we're catching up rapidly, you know, you see OEMs who are starting to actually take cybersecurity as a important point of their offering Mainly because some vulnerabilities detected in their software, as well as owners are now expecting that because that's what consulting engineers are telling them to expect. And so you see this whole thing about cybersecurity is becoming important. But that's just one little piece, right? So we're, we're so focused on this hacking and got to keep our systems cyber secure data, data data, but we don't really think about, it's much more than just defeating hackers. It's about securing the technology that the building automation system is requiring, in order to be functional. So we're going to focus on that largely in this episode. I mean, after all, what threats really exist against the VA s? You know, so I started, I went to the Google and I typed in cyber security for building automation systems. And you start reading and you get all the usual folks commenting, and you get the manufacturers who provide cybersecurity services and products, you know, painting a Doom gloom, the world is horrible that a die and half of them don't even know what we do. I'm not talking about building automation OEMs. I'm talking about folks in the cybersecurity world. I mean, I'm looking at one thing from a cybersecurity provider, and they're talking about PLCs. And they're saying in your building automation system, you need to protect your IoT device cameras, and I'm like, Do you guys have any frickin clue what we do? Actually, in building automation? All right, it's I don't know, what advisors and consultants these companies are talking to. But whoever they're talking to, is not advising them on what the building automation industry actually is. And what we actually do. So there in itself is part of the conundrum right, is trying to figure out what threats are real and what threats aren't. When you have these folks who have a vested interest in you thinking there's a threat, telling you to do things that aren't really viable, because they don't really even understand what you do. All right. So let's talk about what threats really exist. And in order to do that, we have to understand a couple terms, right? First, we have to understand an attack vector, you'll you'll hear this term in cybersecurity circles. And the attack vector is the method the approach the entry point at which a hacker or or an attacker doesn't have to be a hacker doesn't even have to be an attacker could be an accidental attack and accidental issue. As we'll see later in the episode, those are the points at which it all begins. Okay. So an attack vector is where it all begins. And ultimately, an attack, whether it be intentional or unintentional, is going to affect one of three things. There's this concept called the CIA triad. And that stands for confidentiality, integrity, and availability. Now, the reality is the majority of what we do in building automation, we don't give to fly and flips about confidentiality. I mean, we are not sending information that we really care if someone else knows if you know, my zone temp setpoint I really don't care. If you know that my air handlers running, I does not matter. What we're primarily concerned about is integrity and availability. Now integrity is a huge one, right? integrity and availability, I will say are equal, but integrity can affect availability, whereas availability doesn't really affect integrity. Okay, so integrity is making sure that the data that the commands are actually the right commands that were issued. So this
Phil Zito 9:00
is an area that a lot of folks are concerned about, especially with BACnet. Because BACnet is an unsecure over UDP protocol protocol that basically is in plain text often times, and the fear is since there, it's a connectionless protocol. In a lot of cases, even the sync act can be interrupted, sync ack I'm talking about, or no confirm ack, whatever it is for confirmed property requests request, confirm request act. And what I'm talking about there is like if you submit a right property request, and then it will confirm back or if you do a read property request, it is a confirmed request an old confirm back. Well, what I'm talking about there, then is this integrity. If you are able to interrupt those right property or read property requests, then what's going on is you could have a command To turn the chiller on, and you actually turn the chiller off, or you could go and close valves while the chillers running take down a central utility plant to a data center. These are all issues that could occur and could cause problems. Okay, so that's all well and good. And that's that's bad, right? It's not well, good, actually, it's bad. But then you also have availability. Okay. So we have availability and availability is quite simply the availability of the system, right of the building automation system to be accessed, right. It's available, it's up, its uptime. It's running, everything's all well and good. So if integrity is data being manipulated, and availability is things being up, I mentioned integrity can impact availability. Now, at this point, some of you might be tuning out, you may be like, Oh my god, what the heck's Phil talking about? You can't frickin do that right? Now you've got to pay attention. Because if you don't understand what I'm teaching you right now, I can teach you how to secure a VA s. But it's not gonna mean crap. Because you don't understand why you're securing it. You don't understand the concepts behind it. You don't understand what's a real threat, what's not a real threat, you're going to be duped by anyone out there who has credentials in the saying, I'm a cybersecurity expert. And they're telling you to do stuff. That doesn't frickin make sense. Because you don't understand these core concepts. So I'm telling you don't tune out. Pay attention. This stuff's important. Alright. So integrity, right? That is making sure the points are right, making sure our commands and our reads are all good availabilities, making sure stuff is up, integrity can damage availability, because if you can go and make equipment go down, you can make it not available. Alright, so make sense, right? Great. So all that we've got our attack vectors, we've got the CIA triad, and then we've got this thing called OEM security. You know, it was always the hardest thing for me to go and sell software upgrades and patches and stuff like that to our customers. But nowadays that cybersecurity is more of a concern for owners, and OEMs are taking it seriously. They are rolling out patches, and so they are largely responsible for the cybersecurity of products, internal configurations. Now, I want to repeat that real quick. The OEMs are largely responsible for a product's internal configurations. How does it handle HTTP? How does it handle HTTPS? How does it do encrypted logins, what's the authentication process, how's communication traffic in encrypted, how are certificates managed, etc, etc. that stuff you can't change, okay, you cannot change that. You can go and make sure an IP address is, you know, secure behind an ACL and firewalls and all that jazz, you can use a different form of in dedication, like LDAP, things like that. And that's all well and good, but that's on top of this OEM security. Now, OEMs, they go and often go through a process. Nowadays, they didn't in the past of securing their systems, they'll often do external testing, both white box and black box text testing of their systems. And if there is a security issue found, they will often fix it before you're even aware of it. And if there are issues that are found that they weren't aware of those often will go into the ICS cert website under a CV E, which is common vulnerability. What is that acronym? Again, let me remember
Phil Zito 13:45
that current common vulnerabilities and exposures, that's what I thought it was by just wanting to double check that. And that will be on ICS cert, and they'll talk about CVE. And there's a whole rating structure to it that I'm not going to get into, it's pretty clear on their site anyways. But you can look up manufacturers and see what vulnerabilities they have in what version of their software, how to fix things and what the recommended fix is. Now granted, software is software. And I mean, there's depending on who you ask, there's so many bugs per line of code. And there can be issues in hardware and software, and you can be completely unaware of them. And so that's something that I recommend like once a year just checking ICS cert understanding what your if your products in there. And then if it is, what do you need to do to reconcile that. But what we really got to focus on is how can you secure a B as well, there's really three major attack vectors for building automation, folks, right. The first is the network, right? So network has multiple attack vectors, public IP addresses, not putting stuff on VPN, not protecting access to ports, having all the ports open, etc, etc. Then there's the application, right, not using hardened passwords, everyone using the same password, all those kinds of things. And then uptime, okay, so uptime is making sure that the building automation systems up through, you know, redundant power and making sure the server is not sitting under someone's desk or right under the coffeemaker, I actually saw a frickin coffeemaker sitting on top of the VA s server, with the server laying on its side. It was one of those desktop servers, and it was laying on its side and the frickin coffeemaker was sitting on top of it at this university, which shall not be named. But that was straight up comical. I mean, this building automation systems controlling the HVC in this pretty large. I'm trying to not identify them, pretty large university, and there's coffee maker on the VA s server. Okay. Anyways, you guys, I don't think I have to worry about that with. So as far as these attack vectors, I like to break them into internal attacks and external attacks. So the internal attacks, they're pretty much the low hanging fruit, and they're the things you really should just take care of upfront.
Unknown Speaker 16:27
So threat number
Phil Zito 16:28
one, in my opinion, based on my experience, and this probably contrast to a lot of cybersecurity experts out there who will tell you you know, the number one threat is making sure your system isn't publicly exposed to the internet, because then everyone can try to attack it all the time. Well, in my opinion, the number one threat is server uptime and failover. You know, if your building automation servers not up and running, it doesn't matter if its IP address is secure or not secure, because it's not running. So making sure it's not sitting in a broom closet making sure it has ups on it. Ideally, you're utilizing virtualization, like we talked about in last week's episode, to make sure you're going and keeping the building automation server up and running. These are all things that are really important and are so easy to do. So easy, stick it in a rack in an IDF with redundant power, and make sure that it's actually a virtual machine and could failover to another virtual or over to another physical server as a virtual machine. In the case that that IDF goes down, and it's so simple to do, it's really not hard. Number two threat. And you all you all know this is done. So don't tell me Oh, I've never seen this done before. And this is password sharing and single user login, you know, you go to the facilities, folks, and they're like, can't we just make it facility? username facility? password facility? No, that's a horrible freaking idea. Because that exposes you to so many things, right? For one, which person was logged in when the issue was created? I mean, we talked about accidental attacks earlier, what if someone goes and changes a bunch of set points or uses a mass edit and deletes a bunch of objects? I mean, that is essentially affecting the cyber readiness of that product, right? It's no longer functional and ready. So by going and sharing accounts and sharing passwords, you have no accountability of who's logged in. When they're logged in. You have no way of regulating login, let's say some disgruntled employee quits and goes somewhere. Well, I mean, there was a hospital where I think it was, it was Allerton, maybe it was a tritium system. I don't remember I'm pretty sure was an allergen system. It honestly, it really doesn't freaking matter. It could be any system because the issue was not what the system. The issue was with that they had bad user right control. So the security guard had access to the vas system, and he had a password and he had access to everything. And he was changing a bunch of set points and all sorts of stuff. And then he got fired. And I think he continued to do some stuff after he got fired. So password sharing and account management is huge. Make sure you go and resolve that. And I'll talk through methods of securing all of these in just a second. And then finally data and system manipulation. So folks coming in manipulating data, manipulating the system, changing the system, changing the data, etc, changing set points, deleting points, deleting controllers, that so all of this can be resolved pretty easily right server uptime and failover put in a virtual server makes sure that it fails over to another virtual server make sure it's in a in an environment where it's physically separated from one another. So if a fire happens in one building, it doesn't know fact all buildings, right. So take care of
Phil Zito 20:02
that password sharing, make sure you're using unique usernames and passwords for each individual. And finally, data and system manipulation, make sure you're using a single, or sorry, not using a single make sure you're using multiple different user access layers that are appropriate to the level of access you want to have the user having. So you don't want the maintenance folks having the same level of access as the controls engineer who's actually designing the database and designing the architecture, the building automation systems, so regulate those. Now, external attacks, this is where everyone in our industry seems to focus. And this is what pisses me off, right? I mean, this really upsets me, I get that external attacks are a focal point, I do not dismiss that, I do not dismiss that we should be focused on external cybersecurity of our systems. However, in my experience, working on hundreds of systems and talking with 1000s of students who have also worked on hundreds of systems, the majority of the time, we have issues with our building automation system, it is an accidental attack and accidental issue. And it's related to those three kind of things I just listed out, I can count on a single hand, the amount of times I have heard of external attacks, damaging building automation systems, now folks will quickly rush to point at SCADA systems and power plants, and things like that. Those are not building automation systems, I get that they are kind of like the same thing and parity and that they're embedded controls. And I also acknowledge that there are systems that have been attacked. However, I'm saying, If I had 40 hours in a week to allocate my focus, I would focus first on those internal issues prior to focusing on external issues. So hope that was clear in my focus point. And in where I see these as far as ranking of importance. So external attacks, what are external attacks? Well, the most common ones are going to be attacks that take advantage of the network attacks that take advantage of the application. Okay. And then we're going to have attacks that take advantage of the person. Okay, so the end, these aren't ranked in any order of occurrence or order of severity. It's just literally the order that when I was doing the show notes and planning this out that they popped in my head. So first is network. So now here we're talking about a publicly exposed IP address, we're talking about not utilizing VPN, not properly segmenting, controls networks from the public network at a university all sorts of things, right. And the threat is that someone can see the building automation system. And we already know that BACnet traffic is open and clear text, someone can see the building automation system, they can manipulate the traffic, they can go affect the traffic. And they can really affect the building automation system just through an availability and an integrity attack of manipulating data man in the middle attacks, where they intercept data traffic, and then alternate, all sorts of things like that, right. So methods of securing are pretty straightforward. Utilizing VPN in order to go and access your building automation network is a quick and easy fix. For a lot of solutions. properly implemented, a VPN will go and essentially make your VA s seem like it is a internal only network. And when you need to access it remotely use VPN software that creates an encrypted connection. And it allows you to extend the external network to your machine. Now nothing's foolproof. And if your machines been compromised, and all that then mean VPNs are useless. But in most cases, VPN is a great approach to network security. I mean,
Phil Zito 24:14
rather than having that public IP address and worrying about that you just have a VPN that passes through the internal network to your VA s segment, which is another thing VA s segments making sure you segment your VA s network. Now I'll be honest, having the VA s be accessible through the Wi Fi is pretty freakin awesome when you're in a campus environment. And you've got to go and work on the VA s and you're right next to an access point you're trying to configure VA v controller fan coil controller and you're in the ceiling and you got your laptop and you're working on things because you don't want to plug directly into the controller. Because you want to see kind of some other variables that may be in other systems are impacting them, especially VA v troubleshooting. So you're sitting there and you connect to the Wi Fi. And that's awesome because you don't have to be wired anything. Now, on the flip side, that could be a problem, because even if you hide your wireless access, you know, you keep it as an unpublished wireless network, people can still find it. But once again, physical networks, wireless or wired, they still have things you can do in order to secure them. So you can go and ensure that, hey, for this external or this internal external Wi Fi network, you can make sure that only certain devices can connect to it. You can do all sorts of network security things that we just don't have time in this episode to get into. So I said right, the first one is network. The second threat is applications. There's really not a whole ton you can do to applications other than deleting the default username and password, making sure you use certificate ng making sure that you have VPN is implemented so that folks aren't publicly accessing your network and interrupting communications. But most of the applications cybersecurity is handled by the OEMs. I mean, that's just the reality. It is mainly their responsibility to manage their software, they should give you the recommended settings based on their security threat analysis on how they want you to securely implement their applications. But it is their mission and their job to maintain the security of their applications, as long as you are meeting the recommended implementation guidelines. So they should be providing you guidelines on how to implement their applications securely. But once you've done that, that should reasonably address the cyber threat. I mean, they can't guarantee the addressing of every threat. That's not reasonable. But you know, it should address a reasonable amount of threats. Okay, so we talked about our networks and application threats. Now let's talk about person threats. Now, one of our most famous threats and attacks that is quite often referenced is the whole target attack with the Fazio mechanical, and quote unquote, building automation attack, which a bunch of folks were like, Oh, it's a building automation attack. Si Si building automation, cybersecurity, when in actuality It was a phishing attack with someone who had improper credentials to access a part of the network that they should not have been accessing. Actually, it wasn't that they shouldn't have been accessing, they should have just been segmented better. So the issue with personal attacks or people attacks, and this is one of the better ways to attack our industry. I don't know if I should be saying that. But the reality is, we're not the most tech savvy industry in the world. We try to make ourselves out to be but I mean, I stand in rooms, teaching folks about IP controls. And you know, half the room doesn't know what an IP addresses and at, you know, I'm not judging anyone, don't get me wrong, I am not on a high horse sitting there pointing my finger or going look at you. I'm just saying the reality is we are a mechanically oriented industry. And we need to accept that. And the sooner we accept that the sooner we can go and start training the gaps. And one of the gaps is,
Phil Zito 28:42
I mean, even understanding what phishing is understanding what phishing emails are understanding, I mean, we all think that it comes in the form of the Nigerian prince who has a billion dollars, and he just needs you to send him $10,000. And then he'll free up his money. And he'll pay you to know I mean, phishing attacks are much more subtle than that, a lot of times research will be done. And the attacks these days can look very sophisticated. They can look very similar to real emails. And because of this, you have to train your folks, both your customers as well as your own personnel on attacks that can take place because technically, if you aren't properly implementing cybersecurity program, and your devices get attacked, and your people get attacked and compromised and do damage to your customers, I mean, depending on the state you live in and the country you're in, you could be liable and, you know, cybersecurity laws are ever changing. And so I'm not even going to pretend to be a lawyer. I don't represent or provide legal advice, but I will tell you that laws are definitely getting more in the favor of folks who are the people who are the result of an attack or are affected by an attack. And they are becoming much more damaging to the people who didn't prepare their own employees to be able to defend against an attack. So that's something that is just training. I mean, there's really not a whole lot you can implement other than training and processes. I mean, if people are requesting passwords or requesting you to install certain applications or services, and you know, they're masquerading as your OEMs, technical support group, you need to validate that I mean, shop on a phone and ask someone. So a lot of information was covered in this episode. I hope this doesn't. My fear with doing this episode was that it would just make you feel worse about cybersecurity and you would feel like crap, there's just more crap that I need to know. And I'm already overwhelmed. I'm just gonna put my head in the sand. I'm hoping that you come out of this episode. And you're like, you know what, this isn't as bad as I thought it was. Those internal attacks that Phil talked about, I can really see myself going and implementing some security measures around those. That seems reasonable. And then I also understand the responsibility level my OEM has, as far as security, I don't want to be more demanding on them of educating me on what I need to know about the security level of my products that I purchased from them. And so that's what I hope you take out of this episode. And as always, I'm freely available for you to reach out in the discussion board, or if you're watching this on YouTube to go the comments below the video. Alright, folks, I hope you enjoyed this podcast episode recap. As always, please go to podcast on smart buildings academy.com Ford slash 249. Ask your questions there. If you really enjoyed this podcast, then I encourage you to go and leave a five star review on iTunes that does help with listenership. So I encourage you to do that. As always, as I've already mentioned, please do not hesitate to ask any questions you have related to the topic. We love to answer those questions. Thanks so much, and I'll talk to you again next week. Take care
