April 7, 2021
Hosted by JD Sherry, Chief Strategy Officer, Remediant
This week we have Ryan Cash, a Senior IT Security Analyst in the Energy/Oil and Gas Industry with us to discuss a few different topics. We chat about the current state and pains of privilege access in the current world of COVID. Also, we’ll share some great conversation around the power of integrating EDR and PAM and reducing the attack surface...doing more with less. Enjoy.
Hey everybody. Welcome to this week's installment of Cyber Whiskey, Water and Wine Wednesdays. I am so excited to have our next guest on. It's a privilege to welcome Ryan Cash. He's a Senior Security Analyst in the energy sector. Ryan, full disclosure is a current Remediant customer and has agreed to come on and talk a little bit about his background, his journey. It would be only fitting for him to also talk a little bit about what he's going to be drinking tonight since we both are from Colorado. I should know better. I'm going to be vulnerable a little bit here and let that what he's drinking, I didn't even know what it was, I had to look it up and I feel really sad but he educated me on that. But I'd also be remiss if I didn't thank Ryan for his service. He's Army Veteran of, geez, nearly 18 years. So thank you for your service, sir. And let's get into this. So first things first, cheers I'm having a Maker's Mark tonight in my green screen.
Yep. So I got the Groennfell Meadery, Valkyrie's Choice, if the green screen allows that. So this is just mead. So honey, water, yeast.
Cheers man. Thanks for coming on.
Mead's like... Basically, it's a honey wine
I didn't even know that. So I'm fully educated now. So Ryan, let's talk a little bit about... Touch on your background, some of the things you like to geek out with, and then once you give us a little bit of background on how you work and how you tick, I'd love for you to talk about the challenge you saw there in your organization, which is in the energy sector, oil and gas. You've got a multi-year background, probably double digit years in oil and gas and energy. Right? Is it [crosstalk 00:01:52]. Yeah. I was thinking. So yeah, give us a little bit about your background, what you like to geek out on, and then talk a little bit about some of the challenges you saw in your organization and maybe writ large what's going on in oil and gas with access management, and in particular privileged access.
As far as geeking out, I pretty much geek out on nearly everything. I mean, I got into the computers and especially the security, just because I was just sampling all the different pieces. And eventually my college offered me an Ethical Hacker Course, which just sold me. Now, I'm not even close to being a pen tester, but it's sold me on the idea of security. So I'm a die-hard Blue Teamer, but I do some of the pen-testing ideology. So I'm more of a Red teamer or a Purple teamer. But I started doing that with my previous company, which is also another oil and natural gas. I started them with the security minded help desk guy. And then I got hired into the security team. And then I moved to my current company little over, it's almost five years ago now. And yeah, so we pretty much work on all of the security stuff.
And we have a pretty limited team. We've only got including my director of four people. And one of those is not even really a security analyst. She does GRC and Awareness Governance Risk Compliance. When I got hired, it was just me and the interim director. We had to deal with all these limited resources, especially in the terms of manpower. We actually had a moderately decent budget, which is rare. But for two people, it was hard to figure out where that needed to go. So ever since we've been trying to remove those admin access rights, because when I got brought on, everybody had admin rights to every workstation in the enterprise, period. Domain users was part of the local admins group [crosstalk 00:03:46].
Seen that movie. Yeah, for sure.
Yeah. So we've gone on to remove a lot of that. So we were trying to remove admin rights without still allow them to do certain administrative functions, which was a challenge in and of itself with a competitor of yours. And we still have them in some limited aspects, but we're actually looking to get that component out. So we started with Remediant as a solution that was both easier to administer, easier to implement and in terms of... One of my help desk guys said elegant to use.
I love that word.
That's why I said it.
You don't hear that very often when you're talking about it or security solutions. The more with less thing as I read into that, those are my words, certainly before COVID we all had to deal with that. We've been, I think in the security industry fortunate due to primarily either the breaches that are in the news, maybe we potentially in our organizations have been victimized by Ransomware or some other breach that has driven executive leadership to wake up and realize that we can't operate the same way anymore. We've always had to do more with less. I think COVID hit and the attention around getting people remote, getting people productive, but then also, companies really struggling with figuring out how to manage costs, how to increase revenue in really difficult times. I think put a tremendous amount of strain on folks like you and IT Security Organizations that it's like, "Wait a second, the goalposts just shifted on me."
Did you see with the shift to remote work... A lot of us already work remote, so it isn't a big deal from security in IT perspective, but how did that play a role in the movement of the attack perimeter, for example, where workstations are getting outside of your network now, and maybe they're kind of on the corporate network that kind of on their own network, and you've got this hybrid approach. How did that kind of evolve for your security posture and how you approached in particular, how you're going to lock these remote workers down?
I got insanely lucky in that regard. My network manager actually is also a former coworker at my previous... He's actually the one who hired me into the security team at the previous company. But he's a former security guy as well, taught me a lot of what I know. And so he was building our network infrastructure to be highly, highly redundant. I mean, we have quadruple redundancies in some cases, but we also have a VPN that is always on. You turn on the machine, it runs as a service and very few people have the ability to turn it off. So we were able to enact that and that gets us access as long as they get to the internet, it immediately forces a single tunnel VPN back to our home network. And so all the traffic goes out of one of three exit points.
And so that connects us both to the authentication systems, to my privilege access systems. And of course it's to SecureONE as a result of that, and also allows us to protect against malicious URLs by forcing all that traffic through a next generation firewall. So we were lucky in that we had that foresight to do that in the first place. And that's directly attributed to Ben, who's the network manager and his predecessor as well, who laid the groundwork for him, that we just have that redundancy. And he believes in the security. So when I need something to be fixed, he's just all like, "I'm on it." And he's built a great team to do that.
So when we move to almost a hundred percent remote workforce, other than people not driving down the road, we really didn't feel a lot. We did see, of course there are upticks and phishing campaigns because of all the same stuff that happened, but that's what's going on. And so we started as SecureONE journey in this remote environment. So yeah.
Yeah. Interestingly enough, while everybody was kind of getting remote and I think a lot of breaches towards the end of last year, present company excluded thankfully were primarily due because everybody was hustling to get remote. Limited security and IT teams rightfully so took their eye off the ball internally and watching the bad guys to get people remote. And I think things got missed. And then once people are starting to come back, you're starting to go, "Oh, something's different. Something's different in my logs. I'm seeing different indicators of compromise." And I think we started to see an influx of breaches towards the end of last year. Ransomware was at an all time high.
I didn't see Ransomware too much, but we definitely saw an uptake in phishing susceptibility. We do run our regular testing in that regard. And we saw people start to fall for that more and more often. And what we found is that people had a different mentality. They had a, 'I'm not at work' mentality, 'even though this is my work laptop.' And I started actually basically teaching many classes on that, going guys, "We can't drop that. I mean, these are lessons you shouldn't just learn for my company. These are lessons you should do at home as well." And so we started building some mentality. So you're right. A lot of people, a lot of companies miss that ball because they were never on it to be real.
Yeah. You bring up phishing and God, everybody's sick of talking about it. Everybody's sick [crosstalk 00:09:35]. It's drilled in our brain, but it does set the stage for what I think we want to talk a little bit more about is, it is literally the tip of the spear. And in some cases, Spear phishing is a major attack against corporations to get key executives and things like that to divulge. But once that happens, and it will happen to people, this goes into the access management component of when I get a credential, whether I phish them or I happen to get on a machine and dump [inaudible 00:10:11] and get the hashes accordingly, what can I do with that credential? Or what can I do with that hash once I have it? And if you assume that they'll get it either through a phishing exercise or some kind of a hash on a computer, the real challenge for us as IT and security professionals is how can we limit, and I hate to use the terminology because we're both in Colorado, how can we limit the spread of the forest fire? And that is lateral movement.
So how did you guys look at... You said you have existing privileged access management tools. It sounds like they may have been a little bit challenging and less elegant to use. I'm going to spin elegance against that. What did you see as some of the major obstacles with the current, I would say legacy tool sets that maybe aren't focused on just in time administration? And then ultimately what light bulb popped up that made you look at, well, Remediant is doing something different than all these other cats.
So you got it on there. It's the administration of that other tool that really turned us off to it. And to be fair, we actually bought them for another purpose. We were just using this aspect because it basically came bundled and we didn't want to buy two tools, but it became a nightmare within about six months to administer it, trying to add new servers to it because servers where we were focusing on at the time, and we never did fully push this out to the end points, the actual workstations that we wanted to, because it was such a massive headache managing just 100 end points. So imagine what would happen if we got the 1500 end points? It became just nightmarish to deal with and still is.
I get a new server now I got to deal with provisioning it in the system. And the people who were accessing those servers were sharing a single access account for their team. So my applications developers would be using the, let's say application SIS Accounts. So, and when they all access the server, they all look like one account when they were on that particular machine. So in order to figure out who did what, you'd have to go there, okay. Figure it out. It happened, this who checked out the account, it was these people. You had to go to three places to do attribution. So when we saw SecureONE, and I want to say, my director of found you guys through a Podcast, most likely [crosstalk 00:12:42].
Risky Biz. Yep.
Yeah. I know he listens to that one too.
Yeah. That's probably where it was. We're a great partner of Patrick's Podcast. And we love that.
He never told me, it just tends to be Podcasts where he finds people. So anyway, he started looking into you guys because of a mention and the glowing reviews that was mentioned on there. And the more we looked, the better liked. Okay, well, the cost is pretty decent. We look at value not costs. And then when we really started to see how it worked, that's what really, really, really started to sell us because I could say, "Okay, well, Sarah in the applications could access this server. And when she'd performed the action onset server, it looked like Sarah, not like this shared account. All right, great. Well, how do we provision that? Put the AB group on the machine and forget it really." And then in reality, the 80 group doesn't exist. After a few minutes, it doesn't exist on the machine, but it still allows access to the machine via the SecureONE.
This is insanely easy to figure out. You guys have some issues that I'm still trying to work through on the initial implementation. But the professional services guy I'm working with, his name is Thanos, no relation to the supervillain, but he's insanely good. He's very, very responsive. And every question I had, he was just right on the ball within five or 15 minutes, I had an answer. He's like, "Nope, nope, this is what you're doing wrong. This is how you fix it. Cool." And once he got my questions answered, I was able to push those to about 120 servers. It took five minutes once I had the listing. I just had to go, "Hey, team leaders, tell me which servers get which groups." That's all I asked for. Once they got me that I built the spreadsheet. Once I hit go, I think it was done in five minutes literally. And everybody was ready to go.
Yeah. That sounds right. I mean, the automation of that is mind blowing and...
And you're improving it still, from what I understand.
Oh, yeah. Absolutely. I mean, we've really ramped up our engineering team and all that to look at different integrations. I know you're a huge endpoint detection response fan. You've had a lot of experience working with the top ones out there. From our perspective, tying together that engineering effort around locking down privileged access. And what you just described for everybody that maybe isn't following is what we do with you is a partnership through our customer success team. And I'm sure Thanos will appreciate the props. And everybody there in that team is just phenomenal and they work from the largest of fortune 500 companies to smaller organizations that have just as much need in demand. It's very important for us to tie together the story and the fabric of endpoint detection and response alone won't save you from a complete Ransomware attack. It won't save you from lateral movement.
But together when you tie together just in time administration, which is what Ryan just alluded to, you're removing the excess privilege from the endpoint so that attack surface goes from the proverbial big ice cube or the iceberg, for lack of a better term, to a much smaller, manageable attack surface. So when you combine the better together story of EDR and an identity centric privileged access manager approach to Jetta, you are really focusing on a hardened zero trust strategy for both servers and endpoint protection. So I think all that to say, what are your thoughts around how folks can tie together endpoint hardening through EDR and stopping that lateral movement? That's what your journey is right now. You're trying to stop that attack surface from being so big and shrink it by over 90% and automate that in minutes because that's what you just described essentially, right?
Yeah. I mean, do we want to have a good EDR connection in addition with our privilege access management? Absolutely. But I think you're going to see the greatest benefit of that in your SIEM, whether that's Exabeam or Rapid7 or key better, whatever the case may be. And what I like with the SecureONE, not having that weird user translation is our STEM is now going to see who was doing what not try to guess who was doing what or misattribute that, I'm pretty sure I spoke that wrong.
No., that's all right.
But basically the idea is do SIEM is going to figure that out more appropriately and your EDR is what's going to tell your SIEM, this data... SecureONE is going to tell your SIEM this data and you're going to have an idea of, "Okay, did that person go through SecureONE to begin with." Which would be nice. And that means, okay, I've got strong authentication systems. We're using SAML in our with MFA in front of the SAML provider. So I've got strong authentication in there. Then they became an admin for 90 minutes on the machine and then they did their action. So I can be reasonably sure that that person is either, A, really that person or B, an insanely skilled attacker in which case I need to get super, super paranoid.
Yeah. And I would say we all do.
Professionally paranoid is how I tell people I have to be. They think I'm freaking out about security and I'm like, "No, I've just seen what can happen."
No, you're exactly right. I love your comments on the, for lack of a better term, I think you signals guy back in the day, does that sound right back in Army? So the triangulation of these signals is important. I think the ones you hit on the head with, with the SIEM, with the privileged access telemetry that we provide on a continuous basis, that quite frankly just isn't in the industry. And then the other piece of that is what am I getting from Colonel Level Endpoint Detection and Response. Because there's already an agent on there that that's exactly what they're monitoring for. They're looking at tools, tactics, and procedures and indicators of compromise when you tie that telemetry and the triangulation of all that, you have a pretty good comfort level that you kind of know what's going on in an automated way in your environment. And then maybe throw behavior analytics on top of that would be the icing on the cake or the cherry on top.
So, the good news is with that kind of telemetry and that kind of triangulation for that word, when something happens, I am reasonably certain that's the person who did the action because there's very few people in the world who can bypass all of that and still perform this action. And it came up unfortunately a little recently where it was person X and it became an HR event. So, but those are the types of things you need to look out for. And if you only had these disparate systems that were looking at, or your SIEM was really not a SIEM, it was just a log collector So you can go look at logs in the same place. There's no attribution, there's no cross-referencing.
SecureONE is now performing a partnership. I'll say it, we've got Falcon X from CrowdStrike coming in. And you guys, as I understand it are either about you or currently merging, not merging, partnering with them and that's going to be great for us because that agents can provide additional information that I need and give me that analysis that I'm going to desperately want to look for.
Yeah. You're a great straight man. So we are the first PAM provider to roll out what it will be a series of leading kind of integrations and use cases with some of the large EDR vendors, right? And telling that better together story. And we were filing some patents associated with that as well, because we still want to stay agent-less. You don't need another agent on that environment. We've always considered ourselves a good corporate citizen, meaning we want to use REST APIs. We want to write integrations that are thoughtful. We want to write integrations that facilitate automation. And we want to use your term. We want to use integration to create value, to reduce your time to detect and reduce your time to respond. Because at the end of the day, that's the key piece. And if we can get the added bonus of, and I think we do with just the time administration of prevention, meaning you can't use a stolen credential against you.
That's the big bonus. So, Hey, I want to wrap up with a couple of things in the last minute or so here. What, from your perspective has really solidified the relationship you have with Remediant, your investment in what I would say, a shift to just in time administration versus traditional just-in-case meaning even vaulting companies have, "Hey, I'm going to put an account or a group out there just in case I need it. I might make it go through the vault, but it's still persistent out there as a group or as an individual account." How has just in time administration changed your perspective on locking down your enterprise in the oil and gas industry?
My perspective it's not really changed all that much, but I do have to change a few perspectives in my company with relation to that. I am still fighting that battle. And there are certain cases where they've left those. I hate to use the word backdoor, but let's face it. That's what it is. It's a backdoor account. Just better protected into various services. What's helping to change some of that is reliability, huge, huge reliability, because a lot of these guys are a little bit older school in the IT industry. And they've just learned the newer school stuff as they went along. So they've kept up, but they still have that fear of what if X goes down. And that was one of the questions I got. Well, what if SecureONE goes down? I said, I got a quadruple redundancy going on with SecureONE.
And they wanted a six type of redundancy, I just couldn't foot that bill. But I literally have four servers that it would take at least three to go down before I have a problem, before I have a significant problem. And so I could show that. I could show how these are all working. I got three servers up and running so I can show them that reliability. I can show them how they're always all good to go.
And if something were to happen, I got three other servers I can deal with. Now, if I go to my furthest back, the disaster recovery case, yes, I've got some manuals work to do to get there, but I've got that option. And just like a backup I've got multiple deals on here and then a separate physical location almost 200 miles away. So there's not much of a chance of a geographic issue causing us all four servers to go down. But it's that reliability that is going to change that perception, I think. If we get past that idea of what if, what if, what if, and just show people that, what if is, what if the sun hits into us tomorrow is that unlikely.
Right. What you just said there really resonated with me. And I get this a lot when I'm talking with customers is walking through the high availability, the disaster recovery scenarios. Because when you're dealing with privileged access, you want to make sure it works and it's got to be reliable. It can't be a single point of failure. So I think everything you just alluded to indicates how we think about architecture and redundancy.
The last thing I'll say in close with is the importance of the cost of ownership. So you talk about value on the front end. I want to talk now and close with value on the back end in the sense that most of the companies that we deal with don't have a huge professional services or consulting budget. They don't have a bunch of people sitting around that are full-time upgrade in maintenance people, right?
You're doing a lot of day-to-day stuff that you don't have a lot of time to say, "Hey, I can dedicate three days to upgrade my Remediant platform." It doesn't work that way. So what I love about what we're doing to support good folks like you is we're creating an architecture... Part of it is we have the benefit of coming up at a certain time where microservices, Docker, all these technologies that allow for the nimbleness and the agility of tech have benefited us. But it's very exciting when Thanos, works for you and works to upgrade your environment, and he doesn't bring down your services to upgrade it, right? And there's no zero downtime upgrade.
If you wanted to upgrade it during the middle of the day, you probably could. But it's one of those things where the lift and the effort for security tools that are in the critical path, that we really embrace the fact that we can do it in such a lightweight low touch fashion, because that's the last mile, right? The last mile is making sure that you can keep up with the ingenuity that a company like Remediant can provide, but you can do it in a way that doesn't impact your business. So I think that's the key takeaway
It doesn't require us to be up two in the morning.
Exactly. And my hair indicates how many of those maintenance windows and outages I've had to deal with in the past on that. So, Ryan, I could talk hours with you man. This was so good. Thanks for your support. And thanks for taking some time to chat a little bit about what's going on in the oil and gas industry, your experience with Remediant, and just fundamentally your knowledge in the space. It has been very, very beneficial. So thanks for the time. Cheers, my friend. I now know what mead is, and looking forward to having a socially distant drink here soon in Colorado. Thanks for your support and have a great night, man. Thanks for the time.