Search
 

The Feedback Loop

This episode is a feedback loop in action. A few months after this show covered Project Grimace, ArmorText’s automated CTI triage project with Gonzaga University, a listener heard it on his drive home from a Coast Guard Auxiliary patrol, pulled over to take notes, and realized it was almost exactly the intelligence pipeline he had been hand-building himself. That listener is Kurt Hoffmann, who built one of the energy sector’s premier threat intelligence programs from the ground up and now runs cyber threat intelligence for a top-three global insurer. Host Matt Calligan, a colleague of Kurt’s for seven years going back to the ArmorText threat-sharing community under the DOE’s CRISP program, reconnects with him to trace the path from the substation to the boardroom: why CTI has to keep evolving as the flood of intelligence turns into a tsunami, where a human still has to stay in the loop as AI takes over more of the pipeline, and why the threat-sharing culture in OT and energy may actually be ahead of enterprise IT.

Listen on :

  1. The origin story is a feedback loop. Kurt heard this show’s Project Grimace episode while driving to a Coast Guard patrol, realized it was almost exactly the CTI pipeline he was already building, and reached back out to a colleague of seven years. They first met in the ArmorText threat-sharing community under the DOE’s CRISP program.
  2. CTI’s job is to cut through the noise, not chase the shiny thing. Kurt frames CTI as telling decision-makers what the threat actually is and what to do about it, in both IT and OT; the only difference is what you are protecting. It became urgent in OT after the December 2015 Ukraine grid attack, because in energy a wrong signal can cost a lineman’s life.
  3. A threat model is what makes the flood manageable. Step one of the CTI Capability Maturity Model is a structured threat model, and it tunes everything downstream: a 30-year-old Game Boy zero-day gets filtered out because it is not in your model. Across 90,000-plus employees and 200-plus stakeholders with different requirements, the goal is to answer questions before they are asked.
  4. The hard part of AI in CTI is judgment, not volume. AI is strong at screening the “tsunami” of intel and surfacing the quiet signals lost in the noise, like the early rumblings of a NotPetya. What it still cannot do well is run structured analytical techniques or spot bias in a source, because, as Kurt puts it, AI is math: it only knows the ones and zeros it was given.
  5. Keep the human in the loop, and make the AI show its work. Organizations trying to hand all of their CTI and SOC triage to AI are setting themselves up for failure; the black box is the problem. Any AI in the pipeline should document the technique it used, the bias present, and the knowledge gaps, and a human still has to catch the mitigation that would break a bespoke OT system.
  6. AI does not shrink the CTI team; it exposes how understaffed it is. Because CTI is preventative, like a vaccine, its value resists metrics: you cannot count the incidents you quietly prevented. But when AI distills the noise into actions that genuinely fit your environment, the honest conclusion is usually that you are under-resourced, which makes AI a case for more headcount, not less.
  7. Share TTPs, not exact gear, and OT is not the laggard here. A study of incidents like Stuxnet and Industroyer argued sharing would not have helped because each used bespoke TTPs, but Kurt disagrees: the same adversary reuses tactics across water, energy, manufacturing, and maritime, and documented TTPs feed straight into red-team emulation. Counterintuitively, the sharing culture is stronger in OT and energy (CRISP, ISACs, NDA-backed mutual assistance) than in the competitive financial sector.
  8. AI is flattening OT’s obscurity, so old assumptions need a rethink. OT’s defense has partly been complexity, with attackers needing to know five or six proprietary protocols, but AI reads the public protocol documentation most humans cannot, lowering the barrier for opportunistic attackers (the Mexican water utility, the Dragos exercise where AI won). Kurt expects a shift toward genuinely air-gapped OT that looks more like the nuclear power industry.

Navroop Mitter:

[00.00.03.21–00.00.29.18]

Hello! This is Navroop Mitter, founder of ArmorText. I’m delighted to welcome you to this episode of the Lock & Key Lounge, where we bring you the smartest minds from legal, government, tech and critical infrastructure to talk about groundbreaking ideas that you can apply now to strengthen your cybersecurity program and collectively keep us all safer. You can find all of our podcasts on our site armortext.com and listen to them on your favorite streaming channels.

Navroop:

[00.00.29.20–00.00.30.21]

Be sure to give us feedback.

Matt Calligan:

[00.00.30.22–00.00.51.20]

All right. Welcome again to another episode of the Lock & Key Lounge. I am your host for today, Matt Calligan. And you know, I’m actually quite proud of the ground this podcast has covered over the past 18 months. It seems like not a lot of time has gone by, but we’ve had some really fascinating guests, very insightful topics, I hope.

Matt:

[00.00.51.22–00.01.31.04]

At least I think so. But today we’re going to add a new accomplishment, and that is reuniting old colleagues that this podcast has been able to do. So, some backstory here. A few months ago, we talked on this show about our first client event, Field Notes — our sort of inaugural one, that we’re going to do annually — and we detailed an internal project at the time that we had partnered with Gonzaga University on, called Project Grimace, which is an automated CTI triage and curation integration that feeds into the tenants of our ArmorText clients.

Matt:

[00.01.31.06–00.02.02.13]

Today’s guest heard that episode while driving home one evening and actually had a stroke of inspiration, pulled over and started making notes and kept going until 4:00 in the morning. And his ideas on the topic actually ended up fitting right into something that he had been working on internally himself. And honestly, it’s emblematic of this guy. I’ve known him seven years, and he’s always pushed for the prioritization of cyber threat intelligence across his career.

Matt:

[00.02.02.13–00.02.26.06]

In the time I’ve known him, in fact, he was one of the first to join our earliest version of a threat-sharing community. It’s actually the longest running one now, under the DOE’s CRISP program, where the members use ArmorText for the actual sharing and collaboration piece. And he was doing that while he was working at, you know, one of the largest energy companies in the country.

Matt:

[00.02.26.07–00.02.40.07]

Now he’s running CTI for one of the top three largest insurance companies in the world, largely by hand-building the tooling nobody else was going to build for him. Kurt Hoffmann, I hope that wasn’t too embarrassing. Welcome to the show.

Kurt Hoffmann:

[00.02.40.08–00.02.42.02]

Thanks, Matt. Looking forward to our conversation.

Matt:

[00.02.42.03–00.03.17.19]

Absolutely. So, for listeners: for about the past decade, Kurt has been building cyber threat intelligence programs at the points where they matter most. He built and led the cyber and physical threat intelligence programs at the investor-owned energy company that we referenced. He actually developed one of the sector’s premier intelligence capabilities from the ground up while doing this, including the strategic threat analysis and the executive briefings, and he coordinated very closely with government and law enforcement partners across that landscape.

Matt:

[00.03.17.22–00.03.42.18]

He has spanned OT and IT, physical security, and geopolitics. He is now the Senior Manager of CTI at a global insurance company, leading a 24/7 threat coverage team. Of course, he has all the accolades and letters behind his name that you would expect — the CRISP, several certifications including the CTI — and an MBA, because why not?

Matt:

[00.03.42.18–00.03.57.12]

And he stays active with the Coast Guard Auxiliary and Scouting America, because apparently, he has free time after all that stuff. And he speaks French and Russian? Something like that?

Kurt:

[00.03.57.14–00.04.01.22]

No French and Russian. But I do have other languages.

Matt:

[00.04.02.02–00.04.20.22]

All right. So Kurt, today’s conversation — usually I pick a topic, and then we kind of break it down with questions. But our conversation has kind of unfolded in layers. So today is really going to be sort of stacking those topics on top of each other. So, I was hoping to get your version on a number of things.

Matt:

[00.04.20.22–00.04.47.08]

We’re going to talk about your background, what got you into CTI specifically in OT, because that is not a normal place for CTI to be really emphasized. And then obviously the trajectory into the enterprise side. I’d love to hear your version of we’ve heard of Navroop’s version of the roadside, you know, moment. But I’d love to hear your version of that, and how it flowed into what you were working on there.

Matt:

[00.04.47.10–00.05.09.22]

I’d also like two things that are more broad: why cyber threat intelligence needs to continually evolve, and why this envelope needs to be pushed culturally and from a capability standpoint. And then I’d like to get a little in the weeds on some of the differences you see between the enterprise IT version of CTI and OT, with what you did prior.

Matt:

[00.05.09.22–00.05.14.16]

So, if that’s good with you, let’s jump right in.

Kurt:

[00.05.14.22–00.05.16.02]

Yeah. Sounds great.

Matt:

[00.05.16.02–00.05.39.08]

Awesome. So, stepping back here — you had pushed for CTI to be treated as a priority for at least as long as I’ve known you. And, you know, your resume speaks to a longer trajectory, including the OT side of the house where that isn’t quite the norm. Walk us through that trajectory, if you wouldn’t mind.

Kurt:

[00.05.39.10–00.06.11.11]

Yeah. So really, when it boiled down for me, for CTI, when it was hey, we’re here for CTI to try and inform decision makers of here’s what the threat really is. Here’s what you really need to be paying attention to — not chasing the usual new shiny thing that vendors are putting out there. The new product that seems like, oh yeah, this is going to solve all your problems, when in reality it’s not.

Kurt:

[00.06.11.13–00.06.35.23]

And taking it back to basics, going to those basics and understanding the why behind whatever the headline is — why do I need to be paying attention to this? And then the next step is what do I need to do about it? And that’s really where CTI fits in when it comes to the cyber realm — not just in IT, also in OT.

Kurt:

[00.06.36.00–00.06.48.00]

They’re very similar in that regard. It’s just a matter of what is it that you’re protecting, what are your priorities? And that’s how you define the difference between IT and OT.

Matt:

[00.06.48.06–00.06.57.16]

Was OT where you started with the CTI trajectory from a career standpoint, or did it start elsewhere?

Kurt:

[00.06.57.18–00.07.21.06]

I actually started with physical security and then pivoted into cybersecurity from there. And at the energy company I was with at the time, there was no differentiation between IT and OT. You had to pay attention to both. And then you had the December 2015 cyberattack in Ukraine, and that changed the game.

Kurt:

[00.07.21.07–00.07.49.08]

That was: all right, we really need to understand the OT threat, and be able to articulate it, and really know the ins and outs of what the adversaries are trying to do in OT. Because when it comes to energy, lives are on the line. If a line is hot and everything says it’s not hot and the lineman goes to repair it, they’re dead, right?

Kurt:

[00.07.49.10–00.08.09.01]

Or energy goes out to a hospital. There’s only so much fuel in those backup generators, right? And if the adversary messes with those backup generators as well, like what happened in Ukraine at the substations of the control centers, then people will die. Period.

Matt:

[00.08.09.03–00.08.30.14]

I’ll pull on this thread a little bit later, but there seem to be two sides of this camp around OT, CTI and that is: well, it just happens so infrequently, and it’s such a proprietary tech stack that there’s nothing that’s relevant across the different environments. Why share, because it’s not the same, right?

Matt:

[00.08.30.15–00.08.54.08]

We don’t have STIX/TAXII, as an example. So, we’ll get to that, because I love that topic in general. I’ve been coming across that a lot. But so, you started on the physical side, moved into the cyber-OT piece of it. Were you doing both the IT and OT side at the energy company?

Kurt:

[00.08.54.13–00.09.06.13]

No, it was both IT and OT. The team that I had, it was myself and two others, and it was IT and OT. We were the CTI team for both.

Matt:

[00.09.06.14–00.09.13.07]

Fantastic. And how did that lead to the jump into the insurance side?

Kurt:

[00.09.13.11–00.09.41.19]

So, I was part of — or I am part of — a trust group where we openly share different information related to cyber threats. And somebody reached out to me on there and said, hey, I’ve got a position I’m interested in having you fill. And we talked at Black Hat / DEF CON last year; the right offer was made, and the rest is kind of history from there.

Matt:

[00.09.41.20–00.09.50.11]

There you go. But that ended up, for a window of time, with you as just a team of one, correct?

Kurt:

[00.09.50.12–00.10.12.21]

For a very, very short time. I had two other analysts. However, I was the only one that had any kind of intelligence background, so I was the only one that knows the intelligence lifecycle, knows the structured analytical techniques, how to look for bias, that type of thing. However, the rest of the team is highly technical.

Kurt:

[00.10.12.22–00.10.49.11]

So, someone that is really good at malware analysis is on the team. Someone that’s really good at being able to spot brand impersonation and typosquats, malicious domains, that type of thing, is on the team. So, I had those two to begin with. So, I had, and still have, a very good technical team. And from there we’ve just been able to expand, and I’m working on teaching them the more traditional intelligence side, while they’re teaching me even more in depth on the technical side.

Kurt:

[00.10.49.13–00.10.53.21]

And every day they’re just amazing. It’s a great team.

Matt:

[00.10.53.23–00.11.03.11]

When you say intelligence, do you differentiate between the military definition and the private sector cyber intelligence piece? All in one?

Kurt:

[00.11.03.15–00.11.12.01]

Yeah, I try and follow the ODNI guidance as closely as we can when it comes to intel.

Matt:

[00.11.12.03–00.11.26.05]

Got it. So, there you are — what were you actually tasked with taking over? What was there, or did you actually build it out? Did it exist prior to you getting in that seat?

Kurt:

[00.11.26.11–00.11.55.04]

They had a semblance of a CTI program, but nothing very formal. They had a TIP, but it was more of an indicator-of-compromise repository versus an actual threat intelligence platform. And the first thing I did — and we’re still working through it; it’s going to be a multi-year project — is working through the CTI Capability Maturity Model and moving our entire team and the organization up in the model.

Kurt:

[00.11.55.06–00.12.21.05]

So that’s a huge lift that we are working through. And my newest member of the team, she’s spearheading that for me and doing a great job at coordinating with all the different stakeholders that are involved. And when you have an organization as big as ours, as far as the company goes, over 90,000 employees globally, there’s a lot of people that we have to talk to, and everybody’s intelligence requirements are different.

Kurt:

[00.12.21.07–00.12.33.23]

We went from having originally when we started, we’re like, okay, we’ll, we’ll hash out some basics. We figured we might have 100. We’re almost 200 now, and we’re not even halfway through our stakeholder engagements.

Matt:

[00.12.34.03–00.12.38.01]

Two hundred individual people that you’re creating.

Kurt:

[00.12.38.01–00.12.41.01]

Different requirements for? From our stakeholders, yes.

Matt:

[00.12.41.02–00.12.42.08]

Yeah. Gotcha.

Kurt:

[00.12.42.10–00.12.43.03]

Yeah.

Matt:

[00.12.43.09–00.12.58.16]

And is that what you were talking about building — the hundreds of OSINT feeds going into this LLM kind of project you were already working on? Is that part of that process?

Kurt:

[00.12.58.18–00.13.27.23]

Absolutely. So, by having a very structured threat model, which is the step one of the Capability Maturity Model, that’s what you use to tune everything else. So, for example, let’s say one of the OSINT feeds that comes in is talking about Nintendo games and some vulnerability in a Game Boy from 30 years ago. That’s not going to be in our threat model.

Kurt:

[00.13.27.23–00.13.49.10]

We’re not going to care about a zero-day in a Game Boy that’s still in spinach vision, right? That’s not going to be in our threat model. So, we don’t want the LLM, or even our analysts, to be worried about that. So that’s where it’s key for us to have a solid threat model so that we can go through and filter out the things that don’t apply.

Matt:

[00.13.49.15–00.14.01.07]

And are you routing this to anybody that asks for it, or are there particular channels that this stuff flows into now, or at least as a goal?

Kurt:

[00.14.01.09–00.14.28.04]

We have some established channels. And from there, there’s a lot of ad hoc, like, oh, this needs to go here, in addition to our normal channels. There’s a lot of that happening as well. With as many stakeholders as we have, trying to identify who needs what and when is definitely a challenge. We’re doing our best to be the Radar O’Reilly of cybersecurity in the organization and answer questions before they get asked.

Kurt:

[00.14.28.04–00.14.30.12]

But there’s a lot going on.

Matt:

[00.14.30.13–00.14.49.13]

And the threat landscape is a constantly evolving thing. So, when you were driving back — and I want to hear your version of this — when you were on your way home and you were listening to this podcast about Project Grimace, what was it that jumped out?

Matt:

[00.14.49.14–00.14.53.15]

Kind of walk me through that one because I want to hear your version of this. Yeah.

Kurt:

[00.14.53.15–00.15.18.23]

So, here I was driving to go to a Coast Guard Auxiliary safety patrol for the weekend. And when I drive, I listen to podcasts — I listen to podcasts a lot — and Lock & Key came up. I started listening to it. I got to my destination, finished listening to it that evening, and I was like, okay, that sounds almost exactly like what I’m trying to build.

Kurt:

[00.15.19.00–00.15.38.17]

So, I reached out to Navroop, and then we kind of reconnected there. I went on the patrol, had a bunch of ideas spinning in my head after listening to the podcast, and wasn’t able to write them down then. So, when I got back to my hotel room, I wrote them all out — at least the ones I could remember.

Kurt:

[00.15.38.17–00.15.52.19]

There’s still some that every once in a while, are coming out like, oh yeah, I forgot to write that one down. And then on the way back, I stopped at the rest stop and got to have a phone conversation. It was really good.

Matt:

[00.15.52.23–00.15.57.08]

Did you actually call him at four in the morning? Was that?

Kurt:

[00.15.57.10–00.15.59.20]

This was a couple of days later.

Matt:

[00.15.59.20–00.16.02.22]

He would have answered. I know he would have.

Kurt:

[00.16.03.00–00.16.12.12]

But we got to talk, and I’m like, man, we’re kind of working on almost the same concept. Almost the same thing. We’re going on parallel tracks.

Matt:

[00.16.12.14–00.16.18.20]

In your words, what was that similar concept? I’m curious.

Kurt:

[00.16.18.21–00.16.44.19]

So, I want something where I can take OSINT — multiple RSS feeds, for the most part — run them through, compare them to my threat model, filter out the ones that don’t apply, and then use structured analytical techniques to analyze those, to ask additional hypotheses, to go: is this really true, or is there any bias in the writing here?

Kurt:

[00.16.44.19–00.17.14.06]

So, I can tease out what I really need to know about whatever that article is. In addition to that, I want to have all the usual CTI fun things: the MITRE ATT&CK model. I want the indicators of compromise. I want any vulnerabilities that are identified, and to map those. I want to know all those things along with it, I want those teased out, and I want it in a way that’s easy to digest as the CTI leads.

Kurt:

[00.17.14.07–00.17.33.15]

And from there, the CTI analyst can take a look at what comes out and go. Okay, yeah, this really is relevant, this needs to be a priority. Or, this can go out as more of a bulletin, an awareness type of thing. Or this is something that we need to action now. And that’s kind of a goal.

Kurt:

[00.17.33.20–00.17.58.10]

Now, trying to build AI skills that actually do the structured analytical techniques, and do them well, has proven to be quite a challenge. And trying to get an LLM to identify bias in writing has also proven to be a big challenge.

Matt:

[00.17.58.12–00.18.00.02]

Can’t just hard-code it.

Kurt:

[00.18.00.04–00.18.18.14]

We have to remember that AI is math. That’s what it is when it boils down to it — it’s ones and zeros in the end. Whereas humans can apply different logic to it and have different perspectives. It only has the perspective of the ones and zeros that the programmers put into an LLM.

Matt:

[00.18.18.17–00.18.28.09]

Right. It doesn’t extrapolate too well or theorize in ways that aren’t already prescribed.

Kurt:

[00.18.28.10–00.18.29.23]

Yep. Exactly.

Matt:

[00.18.30.00–00.18.42.03]

And is that just a, is the outcome there, is that a benefit just to what’s unique to your team? Or do you see that missing in technology as an option now?  

Kurt:

[00.18.42.09–00.19.06.07]

Oh, it’s missing in the tech. Walking around Black Hat, AI was the theme of Black Hat this year — whether or not it was official, I don’t know. But every single booth, every vendor was talking about, check out our AI. And a lot of them were talking about an agentic SOC. I can tell you right now, maybe for tier one triage, right?

Kurt:

[00.19.06.08–00.19.32.11]

But if you get rid of a SOC altogether, you are in trouble pretty fast. Look at what happened to Hugging Face. And what we’re seeing more and more in the reporting is that you can’t trust the AI to stay within its bounds. And it is going out of bounds — that’s a recipe for disaster if you’re that heavily reliant on AI.

Matt:

[00.19.32.16–00.19.35.11]

If you’re blind to it. Right. Yeah. Yeah.

Kurt:

[00.19.35.12–00.20.06.19]

And I think a lot of organizations that are being very quick to adopt AI aren’t necessarily putting the time and effort into identifying the risk of what happens if their AI says something wrong or does something wrong. I was reading about one example — I think this was yesterday, actually where a company had relied on their AI to answer customer emails.

Kurt:

[00.20.07.00–00.20.17.06]

Turned out, the AI was sending the wrong answers. Somebody caught it, and now they’re having to rewrite everything.

Matt:

[00.20.17.10–00.20.18.01]

Yeah.

Kurt:

[00.20.18.07–00.20.20.12]

Not a place you want to be.

Matt:

[00.20.20.14–00.20.43.03]

I mean, we don’t need to go more into AI than we already do in general conversations. But you see these examples abound where it tries to think outside the box a little bit and usually goes off the rails. And this one was completely unrelated to cyber.

Matt:

[00.20.43.03–00.21.05.23]

It was AI tasked with doing some sort of marketing outreach or something for another company. And it found an old file of this guy’s that was just his random notes and thoughts on something. Treated it like it was an actual markdown file and started generating stuff. But it was also agentic, so it was triggering workflows into these other things, and they had no auditable trail.

Matt:

[00.21.05.23–00.21.23.00]

They had no idea why. And it took them a couple of days of interviewing this AI — like, where did you come up with this thing? — to find out that it had gotten into that document. So yeah, the black box piece of it is like, just trust us, let it run.

Matt:

[00.21.23.01–00.21.25.18]

Yeah, not there yet. Sorry.

Kurt:

[00.21.25.19–00.21.35.06]

I mean, you remember the movie WarGames, and the WOPR. Yeah — it’s good until it’s not.

Matt:

[00.21.35.07–00.21.44.19]

Right. And then you can’t stop it. You don’t know where to stop it, I guess, or how to — that’s probably the better way to put it.

Kurt:

[00.21.44.20–00.22.03.02]

AI is like fire, really. I mean, fire can be beneficial and it can be really, really bad. But you have to know how to use it properly and contain it properly. And I don’t think we’ve figured that out yet.

Matt:

[00.22.03.03–00.22.25.00]

Well, containment is what makes fire useful. There aren’t scenarios where uncontained fire is a good thing, unless you’re a forced remanager or something — literally, that’s the only scenario I can even think of. And so, the constraints are what makes it useful. The direction and containment are what make it useful.

Matt:

[00.22.25.02–00.22.46.04]

Is AI the thing that you see solving for a lot of this? Obviously, it has a long way to go, but is AI the thing that CTI most needs right now in order to evolve? Are there other things missing? Where does the practice of CTI need to evolve?

Matt:

[00.22.46.06–00.22.52.02]

Maybe culturally, maybe people need to start thinking about it differently, but also just features and capabilities from your side.

Kurt:

[00.22.52.03–00.23.17.17]

I think a lot of CTI teams are trying to figure out how to use AI. Feedly, for example, has put out some really great skills publicly for anybody to use when it comes to CTI, and those skills lay down some of the basics that you need in order to do some CTI.

Kurt:

[00.23.17.18–00.23.48.09]

So, some of those basic steps could already be done for you with some really nice-looking skills there. There are others putting out skills as well that are worthwhile — just look at GitHub, there are tons of them out there. You have to vet your sources a little bit to make sure you’re not getting something you don’t want, but there’s a lot of good skills out there that are able to help with the triage of the absolute flood of information that’s coming in.

Kurt:

[00.23.48.09–00.24.02.01]

And I used to think it was a flood before AI came about. Now, compared to what it was maybe two or three years ago, it’s a tsunami now.

Matt:

[00.24.02.04–00.24.24.22]

Just exponentially grown. From a cultural standpoint, do you see it being prioritized internally, as well as inter-organizationally — going back to more of the community aspect of it? Do you see a wider recognition of the importance of this in any kind of security program?

Kurt:

[00.24.25.00–00.24.47.19]

It really depends. It’s a hit or miss based on the organization. From what I’m finding, some organizations are what I would call forward thinking. And hey, we need CTI analysts for this, this, and this. We need CTI to help us understand these threats. And then you have others that haven’t quite figured that out.

Kurt:

[00.24.47.22–00.25.07.02]

Now, for those that haven’t figured it out yet: if you can get your executive to read Robert M. Lee’s book, Cyber Threat Intelligence and Me — I got some executives to read it and it changed their mind. Now they understand what CTI can do for them.

Matt:

[00.25.07.02–00.25.09.08]

Isn’t it laid out like a kid’s book?

Kurt:

[00.25.09.09–00.25.22.12]

It’s written like a kid’s book, illustrated and everything. But the concepts explained in there — I couldn’t do a better job explaining it. It’s an outstanding book.

Matt:

[00.25.22.16–00.25.26.12]

Well, sometimes that’s what you need.

Kurt:

[00.25.26.14–00.25.29.13]

And it takes what, ten minutes at the most to read.

Matt:

[00.25.29.18–00.26.01.23]

Right. And there are pictures. It’s fun. And it gets to a theme that I see recurring in cyber in general. For the longest time, the technical skill set trumped everything, because it was very human-skill driven. And so, the priority was, and often sort of the weighting of who should be in this seat was given to those that demonstrated the farthest reaching, deepest level of technical skills.

Matt:

[00.26.01.23–00.26.25.14]

And what I see a lot of managers now inside SOCs, and even at CISO level, say is that the pendulum pulled a little too hard toward the technical skill, and it has created an awareness gap between people who are technical and can grasp what’s happening, and the value and importance of it.

Matt:

[00.26.25.15–00.26.50.19]

And then there’s this gap between that and non- cyber people. A lot of times in decision making roles who are often asked to prioritize or decide funding and all these things for these programs where they can’t really kind of figure out why it matters, you know? And so, this sort of human element is missing in that gap between the super technical and the non-technical people who need to know why it matters.

Kurt:

[00.26.50.21–00.27.31.12]

And I see that same gap. One of the other challenges that CTI faces is metrics. There’s not really great metrics that are available right out of the box to be able to show to an executive to say, here’s the value of having CTI. It’s really hard from a CTI perspective to be able to articulate: well, if we didn’t do this, or we didn’t block this, or we didn’t put these mitigations in place based on intelligence we received or intelligence we generated, this would have been our cost if the incident occurred.

Kurt:

[00.27.31.15–00.27.57.06]

That’s really challenging to figure out. So you can’t use the number of indicators analyzed. You can’t use number of CTI products produced. You can’t use number of things blocked, or tickets, or anything like that — it doesn’t work like that. We need to come up with really good metrics.

Kurt:

[00.27.57.06–00.28.12.20]

And that’s been a challenge — I think every single CTI-specific conference I’ve been to has at least one talk about metrics, every single time. Yeah. And to be honest, nobody has a good solution yet.

Matt:

[00.28.13.02–00.28.22.10]

Because what you’re doing is preventative. So you can’t be like, see, we stopped that thing, because it doesn’t exist.

Kurt:

[00.28.22.14–00.28.23.21]

Yeah. It’s like a vaccine.

Matt:

[00.28.23.22–00.28.24.09]

Yeah.

Kurt:

[00.28.24.09–00.28.32.10]

That’s the challenge. You can’t say, oh, we prevented this many people from getting XY disease because of this vaccine.

Matt:

[00.28.32.13–00.28.33.16]

Yeah.

Kurt:

[00.28.33.18–00.28.34.21]

We don’t know.

Matt:

[00.28.34.23–00.28.41.15]

You just know that once upon a time people died from it and now they don’t. And so it looks like it’s working.

Kurt:

[00.28.41.16–00.28.42.21]

Right. Exactly.

Matt:

[00.28.43.00–00.29.12.06]

So what’s the correlation? What’s the analogy — I don’t know if analogy is the right word — but, okay, the polio vaccine: people were dying from this thing, then we invented this thing and now they don’t, therefore it is helping. Is there a tight enough connection between industry prevention and, maybe, uptime, downtime, number of incidents, whatever?

Matt:

[00.29.12.07–00.29.16.20]

Are there metrics like that that can be tied to CTI programs?

Kurt:

[00.29.16.22–00.29.38.12]

I don’t think so yet. There’s a lot of gray area. We could talk about all the major incidents and then, looking back at it, what are the things that CTI could have picked up on ahead of time? But hindsight is always 20/20, right? You’re looking for a needle in a stack of needles, oftentimes.

Kurt:

[00.29.38.12–00.29.54.16]

So sometimes you pull the right needle out and sometimes you pull the wrong needle out. But you’re making the best educated guess as to which needles to pull out of that stack of needles, and which ones to say, nope, we’re not going to play with this needle.

Matt:

[00.29.54.18–00.30.03.02]

Do you see AI being helpful in that regard, as far as volume screening, pattern recognition, that kind of stuff?

Kurt:

[00.30.03.06–00.30.12.15]

Absolutely, absolutely. That is where we can really harness AI to, to help us and to help expedite some of that analysis.

Matt:

[00.30.12.16–00.30.16.00]

But not in a black box.

Kurt:

[00.30.16.02–00.30.40.19]

Exactly. You’ve got to be able to show your work. And that’s where it goes back to having skills built that show what analytical technique was used, how it was used, and what bias was there in the analysis and in the source materials. Those types of things need to be documented. What are the gaps in the knowledge, as well?

Kurt:

[00.30.40.19–00.30.47.04]

Those need to be documented as well. And that’s very challenging for AI to be able to do.

Matt:

[00.30.47.07–00.31.08.21]

What would be the dream capability, the dream outcome, of an effective CTI tool? The stuff that you’re building, the others you mentioned, and obviously we’ve got Grimace. What would be the ideal outcome? What would it do that maybe it’s not doing right now?

Kurt:

[00.31.08.23–00.31.41.18]

Help me prioritize the tsunami of information that’s coming in. Prioritize it to my org based on our threat model. Prioritize it based on how big of a threat this is in general. So, a NotPetya type of thing: if we had AI back then and we started seeing rumblings of this thing coming out of Ukraine, what actions could we take?

Kurt:

[00.31.41.19–00.31.57.00]

What could we have done ahead of time to stop NotPetya from spreading, based on those signals that may have been very, very quiet, that were lost in the noise? Right. And that’s where AI can help us identify those things that are being lost in the noise that we really need to be paying attention to.

Matt:

[00.31.57.01–00.32.16.16]

Do you see this reducing the humans necessary for this? Because there’s going to have to be a human in the loop for the foreseeable future on a number of these things. Do you see this impacting CTI teams in a good or bad way, as far as that automation?

Kurt:

[00.32.16.16–00.32.49.18]

I think it’ll help CTI teams. It’ll help the efficacy of CTI teams. You have to have that human in the loop, otherwise you’re going to make mistakes. The organizations that think they can have AI do all their CTI, their SOC triage, all that — they’re setting themselves up for failure. But those organizations that are leveraging CTI and AI together to help triage and find those things within the noise that fit their threat model that they need to be paying attention to, that’s where those teams are going to excel.

Kurt:

[00.32.49.20–00.33.12.07]

That’s where those CTI teams are really going to excel — and probably find that, oh, we are way under-resourced in manpower to be able to effectively action the information coming in that they do need to be paying attention to. And so there’s AI justifying greater headcount for CTI.

Matt:

[00.33.12.09–00.33.37.07]

That’s fascinating. Because everybody’s always talking about the doomsday, like it’s just going to be robots running everything in the dark. But to your point, people don’t. Outside of departments such as yours, it’s hard to understand the value of it, because there’s no clear ROI attached to something measurable that everybody understands.

Matt:

[00.33.37.08–00.34.01.23]

So you’re looking at people complaining about all this noise, and I can’t figure out how to handle this deluge of intel. And it’s like, well, then it doesn’t sound like there’s anything that needs to be done. But if you deploy tools that are capable of actually identifying the signals and distilling them into action items, then suddenly it’s like, no, we need more people because we have X number of action items that are directly relevant to our environment, our model, whatever.

Kurt:

[00.34.02.03–00.34.34.10]

Exactly. And not necessarily actions, because we’re not the ones tuning the SIEM or the firewalls or anything like that. We’re the ones making the recommendations — hey, we need to be paying attention to this, here are the mitigation measures that are recommended. So that’s where CTI having additional analysts comes in, to go: yeah, I don’t think these mitigation actions that came out of the LLM, or were in the original reporting — they’re saying do X, Y and Z.

Kurt:

[00.34.34.13–00.35.02.21]

But ABC, we can’t do them the way they’re recommending or we’re going to break things. And that’s where having OT awareness definitely comes in. Every OT system is unique. They’re all configured for whatever that bespoke purpose is. You can’t say that one pipeline is going to be exactly the same as another pipeline, because they may be carrying different products.

Kurt:

[00.35.02.21–00.35.12.18]

They’re going over different terrain. They have a different environment that they’re operating in. They’re all going to be different. And that’s where you definitely have to have the human in the loop.

Matt:

[00.35.12.20–00.35.34.20]

So this actually dovetails directly with the end of a conversation I had with an OT pen tester at Con Ed earlier this week. I asked him a similar question about the value of CTI and sharing. More broadly, there was a report done — I forget who did.

Matt:

[00.35.34.22–00.36.11.08]

I don’t know if it was CISA — no, it was a third-party analysis organization — but they were looking at: would CTI have been helpful for these OT incidents? They looked at Stuxnet and Industroyer and a number of other things coming out of MITRE’s libraries. And their conclusion, which is concerning at a very high level, was that cyber threat sharing would not have been helpful because of the bespoke nature of the TTPs used in each of these incidents.

Matt:

[00.36.11.08–00.36.53.13]

And so I asked this pen tester, does that mean there’s just no argument for cooperative, community-based CTI like you see on the IT side, like a CRISP or something like that? And he said, actually, no. Yes, we may have a different configuration, but the TTPs, the big themes and the architecture itself, are useful. Even if we don’t actually have this Rockwell thing or this Siemens thing, knowing that they moved from here to here and used this to get over here is useful, whatever the brand name of those things is.

Matt:

[00.36.53.18–00.37.16.05]

So it seems like there’s a necessary missing piece. You’re closer to the fray on this, so maybe it’s just my perspective that’s missing something, but it seems like everybody in OT just kind of throws their hands up and says it’s not useful, because this utility can’t use our stuff, it’s so different proprietary-wise.

Matt:

[00.37.16.07–00.37.20.10]

Where do you fall on either side of that line?

Kurt:

[00.37.20.14–00.37.39.16]

OT environments may be configured differently, but the threat actors’ TTPs are going to be similar, based on whoever that adversary is. So, you look at what’s going on right now in the water sector in the US.

Matt:

[00.37.39.18–00.37.41.03]

Yep.

Kurt:

[00.37.41.05–00.38.08.23]

It’s not just water sector OT incident responders that are involved. You have them from every walk of OT life involved in the incident response and helping out, and identifying those TTPs, because that same actor may be targeting energy, they may be targeting manufacturing, they may be targeting maritime. Anybody else that relies heavily on OT, they may also be targeting with those same TTPs.

Kurt:

[00.38.09.03–00.38.40.08]

And your Con Ed friend — from a red team perspective, that’s where CTI can definitely help, because CTI is going to document those TTPs and turn around and go, hey, here are some TTPs for you to follow for an adversary emulation. And so now that red teamer can turn around and act out those TTPs against their environment in order to help protect it.

Kurt:

[00.38.40.09–00.38.58.17]

And being able to share those TTPs in an OT environment is critical. That’s one of the challenges that actually more pertains more to IT than OT, from what I’m finding.

Matt:

[00.38.58.18–00.39.01.09]

Explain that. I’m curious.

Kurt:

[00.39.01.10–00.39.21.13]

In the OT environment, there’s a lot of information sharing going on, whether that be through the ISACs, through trust groups, or through, like in the energy sector, CRISP, or just in general. I know at least in, in the energy sector, there’s cyber mutual assistance where things can be shared under an NDA.

Kurt:

[00.39.21.13–00.39.48.20]

So you can get really deep and very technical and share what, what’s going on with the participating organizations. And you don’t have that in, at least, the financial sector. You have FS-ISAC. Okay, that’s great, but nobody’s really going to be sharing the nitty gritty, super sensitive, here’s what we’re seeing — because your competition is there too.

Matt:

[00.39.49.02–00.39.51.16]

I was going to say, it sounds like — is that a cultural thing?

Kurt:

[00.39.51.17–00.40.17.10]

Yeah, it’s definitely a cultural thing. Whereas in at least in the energy sector, you may be competitors when it comes to the bottom line and it comes to the financials, but your territory is protected. Right. And so just like in major storms where you see all those utility trucks driving across the country to go help with a hurricane, the same thing is true when it comes to cyber, at least in that sector.

Matt:

[00.40.17.12–00.40.18.12]

Yeah, yeah.

Kurt:

[00.40.18.13–00.40.20.19]

So there’s a lot of very open sharing.

Matt:

[00.40.20.21–00.40.49.18]

Sounds like more people in the financial sector need to read Rob’s book. And obviously, in electricity particularly, there is a clear willingness — even a cultural embracing — of the value that collective defense can have around identifying these patterns, recognizing them and sharing them.

Matt:

[00.40.49.19–00.41.11.05]

Do you think that’s unique to electricity? A lot of folks pick on FS-ISAC, because competition really does kind of trickle down even into the defensive, the security side. Is that unique to FS-ISAC, or is the sharing and the willingness unique to electricity, from your experience?

Kurt:

[00.41.11.08–00.41.39.07]

I think it’s more the competition side. You see pretty open sharing when it comes to the oil and natural gas sector — all the other energy subsectors, we’ll put it that way. Yeah. And you see it when it comes to health care. You see it when it comes to maritime. I can’t speak to manufacturing, but at least in those sectors, you see pretty open sharing.

Kurt:

[00.41.39.08–00.41.55.21]

Most of that in the other sectors is done through their ISACs. And they have some great ISACs. And that’s really what the ISAC’s purpose is: to serve as that hub for all the spokes, the different organizations within that sector.

Matt:

[00.41.55.23–00.42.24.09]

And my impression has always been that the focus has been on the IT side when it comes to cyber, just because that’s where the volume of activities and incidents is. Is that because there’s just less activity on the OT side? Or is it because people in are less convinced that OT,what they know about their OT, and the scenarios they experience, is relevant to other people?

Kurt:

[00.42.24.13–00.42.51.16]

Hopefully it’s because there’s less. If they’re using the Purdue model and they are properly securing their OT, there’s going to be a lot less. There may be more noise hitting the different DMZs between the different levels, which is fine as long as it’s blocked. And oftentimes organizations aren’t looking at their blocked traffic, just because they don’t have time.

Kurt:

[00.42.51.17–00.42.56.22]

Their resources aren’t available to look at blocked traffic in addition to what’s coming through.

Matt:

[00.42.56.23–00.42.58.19]

To actually come through. Right.

Kurt:

[00.42.58.20–00.43.15.20]

So if organizations are using the Purdue model, or something similar, their version of it, to protect their environments — their OT, which is their crown jewels — there will be fewer incidents. Now, the ones that make the headlines are the ones that aren’t.

Matt:

[00.43.16.00–00.43.19.00]

You mean aren’t following that framework.

Kurt:

[00.43.19.01–00.43.27.23]

Yeah, that aren’t properly protecting their OT. There’s a reason why you don’t want your PLC exposed to the internet.

Matt:

[00.43.28.01–00.43.32.00]

Under a passwordless engineering credential or something.

Kurt:

[00.43.32.02–00.43.37.12]

Or default creds. Exactly. Good cyber hygiene is the key here.

Matt:

[00.43.37.13–00.43.37.21]

Yeah.

Kurt:

[00.43.37.22–00.43.38.15]

Yeah, yeah.

Matt:

[00.43.38.17–00.44.28.22]

This is a very specific question. One of the things everybody comments on is the infrequency of events in OT versus IT. And yes, there’s certainly a world of difference in the interconnectedness of it. But I also see a lot of reliance on assumptions that might be getting shaken up by AI a bit. If you think about IEC 62443, or whatever the combination there is — the architecture basis, people adopting that architecture or following that methodology, deploying most of their resources against a wide area of opportunistic, low-skilled people.

Matt:

[00.44.28.22–00.44.57.12]

And like we saw with the Mexican water plant, or even the Dragos capture the flag where an AI is actually winning — it’s capable of quickly figuring out that proprietary firmware, or the ladder logic, getting into whatever details it needs. It’s able to quickly overcome the complexity and the opacity that has generally been in OT because of that DMZ and that structure.

Matt:

[00.44.57.15–00.45.12.20]

It’s giving the opportunistic folks a much deeper skill set out of the gate. Do you see that changing the volume of OT incidents they have to deal with?

Kurt:

[00.45.12.22–00.45.33.06]

I would say that’s true. You’ve got to remember, AI is looking at the documentation for those different OT communication protocols. That documentation is publicly available. It’s pretty highly technical language that most people don’t understand — but an AI will.

Matt:

[00.45.33.09–00.45.34.06]

Yeah, yeah.

Kurt:

[00.45.34.07–00.45.58.01]

And that’s where an AI is able to leverage that documentation in order to be effective in attacking OT. Whereas prior to AI, you had to have pretty technically knowledgeable people who knew OT in order to develop and execute attacks on OT.

Matt:

[00.45.58.03–00.46.04.07]

Or even knew that particular proprietary vendor protocol or capability.

Kurt:

[00.46.04.08–00.46.08.23]

Knowing OPC or Modbus or one of those communication languages.

Matt:

[00.46.09.00–00.46.33.15]

It’s a debate that’s part of the CTI debate, but it seems like there might be something happening that’s going to challenge the assumption that the Purdue structure alone, and relying on the proprietary nature of a lot of the tech stacks, is going to be too big a hurdle to overcome.

Matt:

[00.46.33.16–00.46.38.19]

Which is based on human skill sets a lot of the time.

Kurt:

[00.46.38.20–00.47.04.04]

And we have to remember, AI is lazy. So are humans. And that’s where a lot of the misconfigurations, leaving default creds or things like that, comes into play. If you tell your LLM, do not leave any default credentials, it’s not going to. Tell a human that — all right — oh shoot, I missed three or four.

Matt:

[00.47.04.05–00.47.06.15]

Clocking out. Right.

Kurt:

[00.47.06.16–00.47.10.22]

Exactly. It happens. It just happens.

Matt:

[00.47.10.23–00.47.16.00]

It’s human nature. It’s not a lazy thing, it’s just tough to keep track of it all.

Kurt:

[00.47.16.04–00.47.26.13]

Or you miss one. It’s not because you maliciously tried to — somebody just missed one.

Matt:

[00.47.26.15–00.47.37.11]

Doesn’t that put an onus on vendors, though — like, hey, stop doing that. Don’t ship it like that.

Kurt:

[00.47.37.13–00.47.58.16]

It would be nice. However, you’ve got to remember, the vendors like a Siemens or a Rockwell — they’re shipping that PLC and it’s basically a blank piece of paper. And when it gets to the end customer, they’re the ones that are responsible for configuring it to do what they needed to do in the environment. They needed to do it in.

Kurt:

[00.47.58.18–00.48.19.04]

And Siemens, Rockwell, all the others, they’re not going to know that unless they have somebody on site with the institutional knowledge. Oftentimes in commissioning, yes, they have people on site putting it all together, but they don’t necessarily have the institutional knowledge for that customer.

Matt:

[00.48.19.07–00.48.21.02]

Right.

Kurt:

[00.48.21.04–00.48.33.10]

And that’s during the commissioning process where you have to have somebody from the customer organization basically embedded and holding their hand and going, no, I don’t need it like this. I need it like that.

Matt:

[00.48.33.11–00.48.35.07]

Right. Fascinating.

Kurt:

[00.48.35.12–00.48.46.04]

And oftentimes security, as we’re finding, especially in OT, is bolted on. Well, oftentimes I say it’s just duct-taped to the side. It’s not even bolted.

Matt:

[00.48.46.04–00.48.47.18]

Chewing gum holding it in place.

Kurt:

[00.48.47.19–00.48.58.00]

Yeah. Versus baked in. Did that PLC ever have security? They’re not made for that. They’re made for availability. They’re not made for confidentiality or integrity — they’re made for availability.

Matt:

[00.48.58.01–00.48.59.09]

Yeah.

Kurt:

[00.48.59.11–00.49.17.07]

And putting the confidentiality and the integrity part of it in, baking that in from the very beginning, that’s proving to be a challenge,, especially with the amount of equipment you have out there in the OT space. Some of that stuff was meant to be out there for 50 years.

Matt:

[00.49.17.08–00.49.18.03]

And it’s still running.

Kurt:

[00.49.18.03–00.49.19.14]

Out there for 25, 30 years.

Matt:

[00.49.19.15–00.49.42.22]

It is a different animal altogether. Everyone’s talking about AI changing this and that, but I think OT is one of those places where I sense it’s going to catalyze some shifts really fast: how vendors build things, how security is implemented, the assumptions you make around different models.

Matt:

[00.49.42.22–00.49.45.15]

It seems like all of that’s changing at the same time.

Kurt:

[00.49.45.17–00.49.56.06]

I think we’re going to see a lot more true air-gapped OT systems. Not air-gapped with air quotes, where, oh no, it’s on its own VLAN. For real.

Matt:

[00.49.56.09–00.49.58.06]

The OT thing, and this. Yeah.

Kurt:

[00.49.58.07–00.50.04.13]

And this vendor has direct VPN access, so they can do firmware updates. No.

Matt:

[00.50.04.15–00.50.05.07]

Yeah.

Kurt:

[00.50.05.12–00.50.09.14]

It’s going to look a lot more like the nuclear industry — the nuclear power industry.

Matt:

[00.50.09.15–00.50.32.16]

That’s true, exactly. Nuclear seems to have not missed a step on that one. That’s fascinating. So, back to — sorry, I can go down this rabbit hole for a long time. With CTI at the end of the day, do you argue that both OT and it, the model can work like what works in enterprise

Matt:

[00.50.32.17–00.50.40.12]

IT CTI can work in OT? Or do they need to be handled differently?

Kurt:

[00.50.40.14–00.51.06.05]

I think it could work. However, you have to have CTI analysts that understand OT — not necessarily who can write ladder logic and put PLCs out there in production, but who understand how it works and why it works the way it works, so that when they see something come across, they can recognize it. Pick one of the OT communication languages.

Kurt:

[00.51.06.05–00.51.14.23]

Let’s just say it’s OPC UA, for example. They can recognize that and go, okay, this is OPC, this shouldn’t be here.

Matt:

[00.51.15.03–00.51.15.17]

Yeah.

Kurt:

[00.51.15.18–00.51.35.09]

Why is this configuration trying to do this? This isn’t what should be within the set points. That’s where it differs — having that knowledge within the person that’s sitting in the analyst role when it comes to CTI.

Matt:

[00.51.35.13–00.51.45.00]

Which AI could even help with, even in the model that it is. It doesn’t have to be a super special deployment. It could facilitate some of that knowledge quickly.

Kurt:

[00.51.45.02–00.52.19.05]

You look at what Dragos has built, what Rob has built there, as far as he’s got experts. He has experts — not just in OT in general, as far as generalists go, but experts in the specific use cases of OT. So when Dragos gets called in, they don’t only know, oh, this is just OT; they know the application of the OT within that environment, so that they can properly respond to an incident.

Kurt:

[00.52.19.06–00.52.38.00]

And also properly protect that environment. That’s where OT differs. Whereas in an IT environment you can call in a major vendor like a CrowdStrike or something like that, and they know IT really well.

Matt:

[00.52.38.01–00.52.39.00]

Right. Well, it’s also more standardized.

Kurt:

[00.52.39.00–00.52.55.11]

They know the systems. It’s pretty standard. And there are only a few major vendors out there. I mean, Windows is Windows. It doesn’t matter what kind of box it’s sitting on — Windows is Windows and Linux is Linux. And so, they’ve gotten very good at protecting those things.

Matt:

[00.52.55.14–00.53.17.16]

I thought that was going to be my last question. But with OT — there’s always been more of a push toward standardization inside IT versus OT. Is that a strength or a weakness in OT, as far as the proprietary nature of the vendor stacks and stuff like that?

Kurt:

[00.53.17.18–00.53.43.09]

Let’s compare it to IT for just a moment. How many zero-days, how many vulnerabilities, how much malware is written for, and how many cyberattacks have there been targeting — I’m just going to pick on them in this case — targeting Windows? It’s because Windows has become pretty ubiquitous.

Matt:

[00.53.43.12–00.53.44.04]

Yeah.

Kurt:

[00.53.44.05–00.54.20.05]

And that’s a difference between IT and OT. In OT, you may have — let’s just call it a target that has five or six different types of OT systems operating within their environment, with different communication protocols. That’s where it gets more challenging for an attacker, because to get to that action on objectives, they need to know five or six different OT communication protocols.

Kurt:

[00.54.20.06–00.54.50.20]

That makes it very challenging for the attacker. Whereas the defender, it’s their environment. They know this environment. It’s their home. Maybe a good analogy is that IT is like walking into a neighborhood where all the houses have the exact same layout, the same floor plan, whereas OT is like walking into a neighborhood where every house has a different floor plan.

Kurt:

[00.54.50.21–00.55.01.18]

And only the builder and — say it’s an HOA, the people that do the maintenance know that floor plan.

Matt:

[00.55.01.22–00.55.02.10]

Interesting.

Kurt:

[00.55.02.10–00.55.09.23]

So it really puts OT defenders at an advantage to have a widely different landscape.

Matt:

[00.55.10.01–00.55.21.21]

I can see that. If you had a final closing thought here, to put an exclamation point on the end of this, what might it be for you?

Kurt:

[00.55.21.22–00.55.48.04]

AI is changing the game, not just from a CTI perspective but from a whole-society perspective. And I had a good friend who, unfortunately, is gone now. But he used to say there are two ways to deal with change: you can either embrace it, or you can get run over by it. And that’s where we’re at when it comes to AI right now.

Matt:

[00.55.48.09–00.55.54.15]

What are you learning about that’s new to you? What idea has you really excited right now?

Kurt:

[00.55.54.17–00.56.21.11]

Well, I’ve been trying to tune those skills, getting an AI agent or an AI model to be able to do a structured analytical technique and map it out and show me, show their work. Haven’t found one that can do it yet. I’ve burned a lot of tokens trying to do that, to the point where I was just like, you know what?

Kurt:

[00.56.21.13–00.56.28.10]

I’m just going to have some models on a local machine and try it that way. It’s a lot slower, but.

Matt:

[00.56.28.13–00.56.29.09]

Right, right.

Kurt:

[00.56.29.10–00.56.31.07]

I’m still working on it.

Matt:

[00.56.31.09–00.56.32.11]

Fascinating. Yeah.

Kurt:

[00.56.32.11–00.56.34.17]

Yeah, yeah.

Matt:

[00.56.34.21–00.56.36.02]

I look forward to hearing more on that one.

Kurt:

[00.56.36.04–00.56.43.18]

If we can have AI help us with traditional intelligence analysis, that’s going to help not just CTI but the intel community in general.

Matt:

[00.56.43.19–00.57.03.10]

Is there anything — I’m most curious about things that people have changed their mind on lately either something you’re not quite sure of the answer to, or where suddenly you’re like, actually, no, I now look at this differently? Anything that stands out for you?

Kurt:

[00.57.03.12–00.57.24.12]

AI in general, especially when it comes to having local models. Originally I was very, very skeptical. I’m still pretty skeptical about the capabilities, but the more I play with it, the more I’m finding that there are some pretty significant capabilities that you can run on some pretty cheap hardware. Which is good, but also scary.

Matt:

[00.57.24.18–00.57.32.08]

Right. And both of those are going to keep continue to get cheaper and better exponentially.

Kurt:

[00.57.32.10–00.57.33.22]

Yeah. Yeah.

Matt:

[00.57.34.00–00.58.00.04]

All right. Well, we’re at the fun part of the interview here — obviously with the Lock & Key Lounge, the lounge being the operative theme. We always close with this question, and I always love to see where it goes. So I like to frame it a little differently than Navroop. Imagine that you’re sitting in one of the nicest bars you can recall being in.

Matt:

[00.58.00.05–00.58.25.04]

I always find that the nice bars are the ones you don’t have to yell in — I always like to qualify that. And so you’re in one of those. And at the other end of the bar is someone in security, however you want to define that, that you’ve really been wanting to talk to about something. So what are you ordering as a cocktail, if you drink, and who’s at the other end of that bar?

Kurt:

[00.58.25.06–00.58.47.19]

That’s a great question. I have a pantheon of cybersecurity heroes, we’ll put it that way. There are a few in there, and with some of them I’ve gotten a chance to go in depth and have some great conversations and really pick their brain. And honestly, it’s mainly sitting there and asking probing questions and letting them tell the stories.

Kurt:

[00.58.47.19–00.58.53.21]

And that’s where you get the good stuff.

Kurt:

[00.58.53.23–00.59.21.12]

Honestly, I think what I would do, if I’m in a bar like that and there’s a certain person from that pantheon at the other end — in this case, because of the person and knowing where they’re from, I would buy them a beer from Minnesota. And the person at the other end of that bar is General Paul Nakasone, because he’s from Minnesota.

Matt:

[00.59.21.13–00.59.25.02]

Okay. Do you have a beer in mind from Minnesota?

Kurt:

[00.59.25.04–00.59.30.17]

Nope. Whatever they have that’s from Minnesota at the bar. That’s the challenge, though.

Matt:

[00.59.30.19–00.59.35.17]

Yeah, I know Michigan beers. I don’t know Minnesota beers.

Kurt:

[00.59.35.19–00.59.44.18]

I know Wisconsin beers. But you don’t order a Wisconsin beer for a Minnesotan. That’s like being a Packers fan in Minneapolis. You don’t do that.

Matt:

[00.59.44.23–00.59.49.18]

What would you be drinking, along with this Minnesota beer?

Kurt:

[00.59.49.20–01.00.02.15]

Oh, I’d probably have an old fashioned in my hand. And if it’s a nice bar, it would be an old fashioned with black walnut bitters, using Woodford Reserve Double Oaked.

Matt:

[01.00.02.16–01.00.03.21]

Sweet. Okay.

Kurt:

[01.00.03.22–01.00.06.06]

Yeah, very specific there.

Matt:

[01.00.06.07–01.00.24.18]

No, I just got into the walnut bitters thing — it wasn’t even on purpose. It was in a sample box, and I was like, oh, this is delicious. I have a walnut tree in my backyard, and it stinks and leaves gooey black everywhere. They’re terrible trees. Like, why would you name this delicious thing after that?

Matt:

[01.00.24.19–01.00.25.15]

Yeah.

Kurt:

[01.00.25.17–01.00.29.07]

I have nine of them in my backyard. I know exactly.

Matt:

[01.00.29.09–01.00.32.17]

That’s a nightmare scenario. You can’t even grow anything in your backyard then?

Kurt:

[01.00.32.18–01.00.34.22]

Nope. My backyard is weeds.

Matt:

[01.00.35.00–01.00.36.06]

You have to mow it, though.

Kurt:

[01.00.36.08–01.00.46.03]

It’s weeds. I mow creeping Charlie — that’s all I mow. And the other thing, if you really want to take it up to the next level:

Matt:

[01.00.46.04–01.00.48.04]

I always do. Yes.

Kurt:

[01.00.48.06–01.00.52.15]

Maple simple syrup. Maple simple syrup.

Matt:

[01.00.52.16–01.00.53.06]

Okay.

Kurt:

[01.00.53.07–01.00.53.22]

Use that.

Matt:

[01.00.53.22–01.00.58.12]

Even better than brown sugar. Okay, all right.

Kurt:

[01.00.58.13–01.00.59.14]

Yeah.

Matt:

[01.00.59.16–01.01.08.03]

I can definitely see that one. What’s the difference between just putting maple syrup in it and maple simple syrup? Are you watering down the maple syrup?

Kurt:

[01.01.08.04–01.01.12.15]

Yeah, you’re adding more sugar to it so it’s not as maple-y,

Matt:

[01.01.12.18–01.01.13.04]

Okay.

Kurt:

[01.01.13.04–01.01.18.15]

So to speak. But it still has a little bit of that flavor. And then amarena cherries.

Matt:

[01.01.18.17–01.01.24.01]

Yes, got to do that. Bourbon-soaked, or straight?

Kurt:

[01.01.24.02–01.01.29.14]

No, because I like to put a little bit — I put like half a spoonful of syrup in there in my old fashioned, too.

Matt:

[01.01.29.19–01.01.50.02]

Oh man, I think that’s what I did wrong one time. I was in a pinch and I just grabbed maple syrup to make my old fashioned. It was a little strong. I was like, yeah, it’s syrupy, too much. So making it into a simple syrup is the trick. Okay. Well, Kurt, thank you for this, for the time here.

Matt:

[01.01.50.04–01.01.58.20]

It’s been enlightening and very good. I enjoyed our conversation, I hope you did too.

Kurt:

[01.01.58.22–01.02.02.05]

I did, I absolutely did. Thanks again, man. It was great to reconnect.

Matt:

[01.02.02.06–01.02.06.12]

Yeah, exactly. All over a podcast.

Kurt:

[01.02.06.12–01.02.07.20]

And to the listeners — you’re doing a great job.

Matt:

[01.02.07.22–01.02.37.03]

Oh, appreciate that, man. It’s nice to have a couple of fans. I don’t need too many of them, but it’s nice to have a few. And for everybody listening: the point of this podcast isn’t just to be a one-way broadcast or a monologue. The intent is to give space for new ideas, and old ones, to be hashed out, debated, built on. And our career trajectories aren’t straight lines either.

Matt:

[01.02.37.04–01.02.57.11]

They’re orbital. We circle through each other’s orbits as we hurtle through this thing called life, and sometimes those intersecting orbits produce something more valuable than when we parted. And this conversation with Kurt is living proof of that. So for those who have stuck with us through this, thanks for circling back to this episode of the Lock & Key Lounge.

Matt:

[01.02.57.11–01.03.03.17]

And until next time: be well, stay curious, and do some good work while you’re at it.

Matt:

[01.03.03.19–01.03.36.16]

We really hope you enjoyed this episode of The Lock & Key Lounge. If you’re a cybersecurity expert, or you have a unique insight or point of view on the topic and we know you do — we’d love to hear from you. Please email us at lounge@armortext.com, or our website: armortext.com/podcasts. I’m Matt Calligan, Director of Revenue Operations here at ArmorText, inviting you back here next time, where you’ll get live, unenciphered, unfiltered, stirred, never-shaken insights into the latest cybersecurity concepts.

Search