February 25, 2021
Hosted by Nick Holland, Director of Editorial, ISMG
Holland, an experienced security analyst, has spent the last decade focusing on the intersection of digital banking, payments and security technologies. He has spoken at a variety of conferences and events, including Mobile World Congress, Money2020, Next Bank and SXSW, and has been quoted by The Wall Street Journal, CNN Money, MSNBC, NPR, Forbes, Fortune, BusinessWeek, Time Magazine, The Economist and the Financial Times. He holds an MSc degree in information systems management from the University of Stirling, Scotland.
Guests: Tim Keeler, CEO & Co-Founder, Remediant
Keeler is co-founder and CEO of Remediant. Previously, he was a leader on the security incident response team at Genentech/Roche and served as a security consultant, with clients that included UCSF, Genentech/Roche, Gilead Sciences and CardioDX. He is a GX-certified Security Incident Handler and earned his GX Security Leadership Certification from GIAC. He holds U.S. Department of Defense Level 3 8750 IAT and 8750 IAM Management certifications; CHFI (Computer Hacking Forensic Investigator) from EC Council and a certification as a CCFE (Certified Computer Forensics Examiner) from IACRB.
The SolarWinds supply chain attack is another example of the damage that lateral movement by system intruders can cause. Tim Keeler of Remediant says detecting lateral movement is challenging because of the size of today’s systems and the difficulty of filtering bad behavior from benign behavior in remote work environments.
"How do I know whether this is just an admin doing their regular activity, versus someone using those credentials in a malicious manner to get access to other systems? Because if you're dealing with an environment that's one or 200,000 systems, it's really hard to scale this out,” Keeler says. “And how do you actually discern and understand what is malicious and what is just your day-to-day behavior?”
In a video interview with Information Security Media Group, Keeler discusses:
Hello, this is Nick Holland with Information Security Media Group. Today, I'm talking with Tim Keeler, who is a co-founder and CEO with Remediant. Tim, thanks for joining me.
Yeah, great to be here, Nick. Thanks for having me.
Very welcome. One of the things we're going to talk about today, or the thing we're going to talk about today is, I guess, the elephant in the room for cybersecurity these days, which is the SolarWinds breach that occurred back in December. You wrote an interesting blog post on that. Let me just share my screen, if I may, so that the viewers can have a look at that.
So here, if you go onto the Remediant website, you can see here, there's a resources tab, Remediant Blog. This is what we're going to talk about today, the role of admin credentials in the SolarWinds attack. Tim, you wrote this back in December. It's a great piece of analysis, I think. But maybe we can just go through the various components of this particular blog piece.
First one, and it's something that's come up a lot, I think, in the last year or so, is the damage that can occur with lateral movement. So this blog post starts out talking about lateral movement and how that has been particularly dangerous in the context of the SolarWinds attack. So maybe talk me through that first.
Yeah. I mean, lateral movement has been one of these attack factors that's... It's really plagued the industry for several decades now. It's a strategy that originally came from interested actors and now it has made its way all the way through.. Even then we take a look at ransomware where the various strains actually have built in lateral movement and credential compromise. But, when we take a look at all the breach reports, it seems to be the common theme that comes up overall. And it's like, Hey, it's much easier to get a hold of some credential in order to move around other areas within the network, from system the system, whether it's going from workstation to another workstation or workstation into a server environment that helps you get to whatever your objective is.
And so, here will be seeing this being a key theme around the SolarWinds attack. And there's been a lot of really interesting technical breakdowns around the SolarWinds. And the thing that really was interesting to me was how this actually played its role in lateral movements. And more particularly, it's like, once a company was affected and they had their command control set in place, what was happening from that point on, what were these nation state actors doing in order to get to their objective? Because their objective wasn't just to get to the SolarWinds server, it was to use that as a pivot to get to other places. And that's why [crosstalk 00:02:32]
I mean, the fallout from that is still occurring in real time. We're only scratching the surface of the real implications of this.
Yeah. I really think we're going to see this be popping up for one or two years to come. A lot of companies are doing their own analysis. We obviously saw for many legal reasons, governments had to publicly disclose their breaches, companies are still going through their analysis to see, is this something that I have to publicly disclose or something that I can manage internally and even understanding if they were affected or not, which is really, really hard with this because we saw the evasion techniques so high, in these cases, where it's really hard to understand. It's like, what accounts were compromised? Was the lateral movement going on? Because this really looks like regular behavior at the end of the day, and a lot of large companies aren't well-equipped to understand, hey, is this just regular normal activity or, do we even have the logging mechanisms to turn on that, to actually understand if lateral movement was going on?
Well, that plays into the next question really, which is, I mean, clearly that is the biggest challenge with this lateral movement, the detection. So maybe talk me through just the key challenges there are overall and how could they be alleviated.
Yeah. Coming back from my [inaudible 00:03:57] states, this is a question that was so extremely hard answer because once there was a compromise in place it would always get asked like, how do I know whether this is just an admin doing their regular activity, versus someone using those credentials in a malicious manner to get access to other systems? Because if you're dealing with an environment that's one or 200,000 systems, one, it's really hard to scale this out, it's like, we're going to collect all log on events from all systems. And then two, it's actually, how do you actually discern and understand what is malicious and what is just your day-to-day behavior? And this is a classic scenario here in the SolarWinds attack.
The way we've been thinking about this is really coming in and changing some foundational ways that we're looking at this problem instead of like, we can talk about AI, anomalous behavior and all this stuff that's really great and interesting. The reality is nobody is actually doing this really, really well. But we need to understand, how do we actually stop this in the first place if we go in with the mindset assuming that the credentials are already compromised and then ask ourselves, how do we put these mechanisms in place? How do we build strong security to actually prevent this from happening, and then allow us to start detecting these things?
And that's where we really come in with this idea of least privileged with a gardener is calling zero standing privilege, all this lines up with zero trust with the premise of, we need to understand and validate the identity. Because that is the focal points of all security rate, but network security is just a bystander in this case, we have to go in and understand. It's like, how do we actually stop that lateral movement from happening in the first place, and then understand, what might be anomalous and malicious?
Yes. The real issue for a lot of organizations is that after all everything looks anomalous. You've got people signing in at weird times, you've got people coming in from different regions, and as you said, nobody knows you're a dog on the internet. So it's the big issue. Knowing the credentials of the people or even the machines accessing your network is absolutely mission critical now.
Yeah, absolutely. I mean, just take a look at the COVID world that we're living in where we have large amounts of our workforce completely remote, goes absolutely to your point of, Hey, I'm doing my job all hours of the day, all hours of the nights. That just makes things even more. And then when you take a look at this from an external perspective, and it's like, "Well, I can't use IP-based detection because everyone is coming either through a VPN or some mechanism that helps them get access to the environment." And that just raises a huge amount of challenges there.
So, your recommendation in the blog is that you remove 24/7 admin access so lateral movement can't occur, even if the intrusion occurs. So talk me through that a little bit, if you might.
Yeah. I mean, this is the area that attackers leverage the most. And that's because when we take a look at the role of an IT administrator, whether it's someone in your service desk that might have admin rights on every single workstation, because they need to be able to manage and troubleshoot those systems versus your server admins that also have 24/7 admin rights across the whole server environment or large sets of the environment. So when you couple this over time, we end up having this pervasive administrative sprawl. And this is the key part that attackers, they're leveraging. So their understanding is like, okay, there's so many people out there that have blanket admin rights, all it takes is one account to get compromised that allows me to start the initial steps of lateral movement, then it can start harvesting more credentials and makes it really easy to get to my objective. And a lot of these we see full domain controllers compromised and that means a high level of persistence for a very long time in these environments.
And so, that is the key principle that we walk in with. It's like, Hey, if attackers are exploiting this, and when we start talking to IT admins and seeing what they're doing on their day-to-day jobs, they're just logging into a small number of systems at any given time. So, on one side we have this huge amount of risk out there because they have this 24/7 admin rights, but then you see what they're doing and it's like, there's a big discrepancy there. And the best way to handle this is a philosophy that we bring forth of Justin time actors where it's like, Hey, if we start with the fundamentals of least privilege by removing all of this persistent admin rights, we can then dynamically add then remove admin privileges when somebody needs to do their job, whether they need to log into a workstation, to troubleshoot an end-user or they need to log into a server, and then you can combine this with strong authentication, so multi-factor authentication.
And this vastly reduces a lot of that risk because when you see one account that gets compromised, well, that shouldn't have blanket admin rights to all these different systems out there. And this is the fundamental approach which solves not just this area, but a lot of other different types of things including ransomware out there, that's leveraging credentials.
That makes it enormous, Tim. I mean, the default just should be zero access, right?
Yeah. And we get asked this all the time. I was like, well, why wasn't this just built from the ground up? And it's like, "Yes. We even inherited bad security practices and legacy architectures for a really long time, but that approach really helps you get scaling." And when you think about this, it really shouldn't be like, I'm just doing this across these hundred or 200 servers, which I deem as critical, this should be deployed across the entire enterprise because lateral movement, as we know, always starts at an end-user workstation or in this case, it was a SolarWinds server which nobody really deemed as like, this isn't mission critical, but this is a key vector to get access to other systems.
Yeah. Okay. Fantastic. Really enjoyed the discussion. Thanks so much for... Great blog post, again, on the Remediant website. That's Tim Keeler, who's the co-founder founder and CEO with Remediant. And from Information Security Media Group, I'm Nick Holland. Thank you very much.