AI vs. AI: The Future of Cybersecurity Will Move Faster Than Humans React

00:02 On this edition of Fakegov Today with Francis Rose sponsored by SYNNEX, securing the next generation of artificial intelligence, Agentic AI. This is where agents are using the, models to ask questions in in a more rhetorical manner. So imagine you have two LLMs talking to each other, and they're iterating on a problem just like we would iterate, in a conversational manner. Former technical director at NSA, Mark Korr, on FedGov Today in just a moment. The new chief financial officer at the Department of Housing

00:33 and Urban Development already knows his way around the agency. Irv Dennis is back in that job, and he'll be on FedGov Today TV this week. You can watch the show this Sunday morning at ten thirty on ABC seven in Washington and on the Fed Gov Today YouTube channel. If you missed the debut of the show Sunday morning, you can watch it on demand anytime at fed gov today dot com and on YouTube. The General Services Administration's rolling out a new generative AI tool for its employees. Zach Whitman, the chief AI officer and chief data officer at GSA,

01:04 says the agency will use staff input to drive the evolution of the tool. Mark Korr is chief technology officer and cofounder at Synack. He's former technical director at the National Security Agency. Synack sponsors today's FedGov Today podcast. Mark, welcome. It's great to talk to you. This is just one example of a number of different tools that agencies across the government are using that revolve around generative artificial intelligence. What does that shift what does that intense adoption mean for

01:34 the security of the networks, for the security of the agencies' data, and for the way that data moves through those networks? Welcome, Mark. It's Nice to be with you, Francis. Thanks for having me. Yeah. It's a it's a transformation that every business is going through. The government's not unique in this. You know, there is a proliferation of tools now. You know, whether it's the, ChatGPT, the Anthropic Cloud, Google Gemini, everybody is is adopting these tools. And, you know, enterprises and governments alike are worried, you know,

02:06 what what what happens when my data is entered into those tools? Is the data being used to train the model? Does it become part of the model? And one, you know, one thing that I learned talking to OpenAI is that if you're on the free plan with OpenAI, all that data, does get used for the for the model training. You know, they're analyzing the questions. They're analyzing the files you input, the images, and and the responses to make future, queries,

02:36 more optimized and, more accurate. So, you know, it's something that everybody needs to worry about. You know, are the proper data controls and hygiene in place? Is the workforce trained on what data is suitable to be input to these, systems. And so the good thing about having an agency that's leaning forward and putting out their own tool is that you can encourage people to use the agency's tool, not, just a random one from the Internet. And, you know, this this actually came up,

03:06 a few months ago if you recall the DeepSeek, news story about China's DeepSeek application that they launched in the US app store, for everybody's mobile phones. Well, that Chinese model, was receiving inputs and a lot of traffic from across the US, and and there were a lot of concerns that, you know, sensitive corporate information was gonna be input to that and and end up in Chinese hands. So it's it's a common thing that we're gonna have to adapt and train our employees and,

03:36 staff on how to manage corporate data and which systems are authorized to process that data. You have a unique perspective because you have private sector experience and public sector experience. What are the differences and similarities into the thought process and and the strategy development of using generative AI and securing generative AI for a private sector enterprise versus a federal government organization? I think there's a lot of overlap in in this concern around data privacy.

04:06 You know, the the tools are quite good at analyzing data. Right? They can save you a ton of time, you know, taking spreadsheets and saying, you know, do this analysis on it or taking, raw data from your email, for example, and saying, well, what's a suitable response to this this inquiry? That's a problem across government and, commercial, you know, companies alike. And one of the things that is a challenge is just the processing of of sensitive data and how you make sure it's handled securely. You know, one of the things that we're starting to realize is that the

04:38 tools are so prolific that, you know, employees are are picking them up on their own and they're using them in their personal lives. And it's blending the personal use of AI with the the business use of AI, especially as people are working remotely and and they're so integrated with, their their same device being used at work and at home and bring your own device, world. It's it's really an interesting time because everybody's using AI to make themselves more efficient. And the efficiency gained is sometimes seen as, you know,

05:11 more important than the privacy that you're handing up. And and this is this is gonna be, you know, a reincarnation of the old privacy debate. You know, do you want more profit, and more efficiency in your work, or do you want to sacrifice this this privacy? And so, you know, the adoption of these tools, we're not gonna stop. You know, people are gonna continue to use them, and it's just gonna be really important that we put out the right guidance of what data is suitable for which systems. And in in government, you know, if they're gonna launch their own like GSA is doing,

05:42 I think that's a great approach. You know, most of the major cloud providers or providers, if you're using Azure for government or or Amazon government, they have they have authorized models that you can process sensitive data on, and you're not gonna end up causing a leak. But it's really important to separate that personal use from from business use. That's exactly the reason that, another agency, person told me, not GSA, that they're developing their own tool internally because

06:12 they recognize the efficiency gains and effectiveness gains, but they don't want their people running that data places where they don't know. It's kind of the twenty first century equivalent of shadow IT where people used to go off and buy their own servers and stick them under the desk and plug them into the network, and nobody knew about it until it was too late. And that's exactly what these folks are trying to avoid, it sounds like. And it sounds like you're endorsing what they're trying to avoid. Yeah. Absolutely. And and, you know, I run a small company. Our company is about two hundred people.

06:43 And I I went to the major providers of AI's tools, and I said, hey. Give me the, users with anybody with my domain name that's registered with your service, and basically sent them a request for data. And they came back with way more users, that were using the service that I had realized. And, you know, our employees had just signed up, for that. And that's, you know, that's something every business needs to watch out for, and make sure that they're they're, you know, educating their workforce and controlling that access.

07:13 And, you know, you look at the optimizations that you're getting from the workforce, you're actually getting way more effective, work out the door with the use of AI. And so it shouldn't be something that slows down adoption. We need to just put the right controls around it and the right policy frameworks. I mentioned generative AI, and there's a term that I'm starting to see on the in the market that I would appreciate your definition of. What is what does agentic AI mean in in your view, Mark? Oh, agentic AI. This is the next wave. This is where agents are using the,

07:45 models to ask questions in in a more rhetorical manner. So imagine you have two LLMs talking to each other, and they're iterating on a problem just like we would iterate, in a conversational manner, right here. And so they'll ask a question, the other one will answer, and then they'll keep asking questions back and forth, by themselves to to resolve the the inquiry. And so it's a really a a unique way to leverage, just the the power of these agents because they will,

08:15 through question and answer, iterate and get you the best response. And in many ways, it mimics how humans solve problems. And so we're seeing the adoption of agentic systems, but more importantly, multi agent systems that have different specialties and different, different roles in the conversation. So you can think of it as a product management, function. Let's say we're building a web application. You may have an agent that is a designer helping you with the design of it. You may have a front end engineer building how the website looks. And then you may have a back end agent that is, you know,

08:45 constructing the the how the actual site is served from the servers itself. And all these agents work together to to fulfill your requirement of building that application. And so it really is a force multiplier, and and this is gonna be the next wave of how people leverage, Gen AI. You know, one of the things that we're seeing now is multiple organizations putting out agentic frameworks that allows developers really to customize how the agents operate. And you can build these, you know,

09:15 just like we would train a human to execute a particular task. You're gonna be able to build an agent that can execute that task, and do it at a cost that's that's very cost effective. You know, not not to pair it for OpenAI, but they're they're even contemplating a product this summer to be released that's a twenty thousand dollar product that is an agent, that could be an employee on your staff. And so we're gonna see the blending of human workers and agent workers and humans driving swarms of agents to accomplish tasks that would have taken swarms of humans before.

09:46 There's a lot of theory and a lot of philosophy there that I would love to discuss with you another time. But I wanted to ask you, as as organizations are thinking about AI, especially folks I talk to in the federal government, they're thinking about AI from two perspectives, especially with the development of generative AI and now agentic AI as you describe it. One is how does one go about securing the tools that one is using, and we've touched on that already. And the other is how does one use those tools to secure their

10:16 entire enterprises in ways that they haven't been able to do so before? What's your sense of that? And in particular, I'm thinking about vulnerability management in an AI environment that looks dramatically different than even the AI environment looked a year ago or two years ago, especially. Yeah. For the for the products that are in AI today, I mean, they're served up on APIs, application programming interfaces, and web applications. And so, you know, securing them is is gonna be very similar to

10:47 securing other web applications. The main difference, you know, with the AI applications, is is the prompts and how do you secure what what prompts are are putting in into it and and what those responses are. For example, we don't want our prompts to be biased in certain ways, and we don't want our prompts to to say mean things to us. So there there are things that we want the LN to provide to us, and there are things that we don't. You know, we don't really want it to provide us with, how to make a dirty bomb, for example. So we want some security controls around GenAI in that regard.

11:20 But pivoting to the other question of of, you know, security and how does it impact it, I really think the pace of the things is gonna pick up. Let's think about a world where, you know, the adversaries are also using AI tools, and they're building AI agents that are constantly, you know, looking around the Internet for known vulnerabilities and and building exploit code that is that is polymorphic. You know? And so you can just ask the AI agents now to build you a new exploit and make sure each exploit is different. And so, you know, you're gonna see offensive, you know,

11:52 weapons basically built by criminal gangs and by nation states that are going to be, you know, exploiting vulnerabilities in infrastructure, not in days, not in weeks, but in minutes. And so this pace of things is gonna pick up because if our adversaries are moving at the pace of minutes and it's all AI driven, well, then our defenders have to move at that pace as well. So for vault management, you know, this is not a a time where we can go back to, you know, five years ago when people would say, well, a critical vulnerability has to be patched in fourteen days or seven days.

12:25 No. It's gonna be minutes, and it and we've gotta figure out how to get the basics right on patching quickly. And as soon as things are available, we've gotta push that out. Because the the agent frameworks that are out there now are are nascent, but they're evolving quickly. And adversaries are picking these up and they're gonna be building, you know, pretty complex toolkits that, are gonna accelerate the pace of exploitation. And and our defenders, they're really not ready for it just yet. When you say the good guys are gonna need to move at the pace

12:55 of the bad guys, maybe even faster if they wanna try to actually ever get ahead. It seems to me that if they're moving at the same pace as the bad guys, then they're just kinda staying static. Is that the fair observation? Francis, it's Francis, the pace is gonna be so much so that you can't really have a human in the loop. It really needs to remove humans from the loop entirely. And this is kind of what we're seeing in the in the battle space where AI is being applied to to warfare as well. You know, in Ukraine, for example, you know, this is not a this is not a place where humans need to

13:26 operate on the front line. It's all drone versus drone. You know, in our first wave in in the future, AI driven warfare is gonna be AI driving robots to the to the site of contact. And humans aren't gonna need to be in that space because it's gonna be swarmed with with drones and robots. And same thing in the cyber space, you know, it's gonna be constant AI on AI. And this is this is gonna pick up the pace of just about everything we're doing. But it's more important than ever that we get a handle on our assets. We understand what assets we have,

13:56 how to patch them quickly, and know how we how to shut down, you know, those those attack paths, that can be exploited. You know, the the the technology that we're using under the hood is not evolving, you know, at that same rate. Right? So we're gonna be stuck with the technology footprint that we have for many, many years. The main planks of security in the government right now are FedRAMP and making sure that a system is FedRAMP certified, and and an application is FedRAMP certified and the zero

14:28 trust concept. What has to happen with them to evolve in a way that it hopefully, eventually can get ahead of that pace that we have just been talking about, Mark? Yeah. So, you know, right now, the FedRAMP program is is very robust in terms of the controls and and the documentation that's required, but it's very much a paperwork driven process. And so what the FedRAMP office is trying to do now, this administration is move to more of a machine to machine verification of compliance.

15:00 And that's that's probably what we should have done five years ago, but we're we're getting there now. The idea is that, you know, instead of that stack of that paperwork, that three to five hundred page, system security plan being verified by an auditor by hand, we would have the compliance, controls checked by software and then push those compliance controls up to the FedRAMP office so that they can constantly verify that FedRAMP programs are meeting the security requirements.

15:30 Right now, that is just a it's a human verification and and human analysis to some that data. And it needs to be more machine and analyzed and the machine flagged and and ideally machine remediated. So that's really the next step there. I mean, the, you know, the the zero trust initiative is a great idea just to modernize, you know, how we authenticate. But all of this rolls together into, you know, we've gotta keep pace with the with the technology. We've gotta start sunsetting the legacy systems. And and we can't hold on to these legacy systems if they

16:00 they don't give us the proper security controls. There's there's no need to go back and, you know, have more data breaches across agencies because we're we we didn't modernize quick enough. It's it's time in twenty twenty five to start to start to sunset things. You mentioned that the changes that are coming to FedRAMP probably should have happened already. What else should be happening already that you would like to see in the next, say, six months or year to get to a more, secure, more hardened cyber position in your view?

16:32 Yeah. I mean, in government specifically, you know, there's just a there's a lot of assets out there. You know, the government footprint on the Internet is quite large. You know, even from the invention of the Internet. Right? We the military and and DOD ran, ARPANET and that turned into the Internet. Well, that means they own a significant amount of IP space. You know, the addresses on the Internet. And so we've gotta we've gotta get a handle on our shadow IT. Gotta get a handle on what's running out there and the legacy technology is is one of the main issues that I see.

17:04 You know, one of the other ones is just the inability to scale testing across the government attack surface. You know, there's only, you know, nine or ten NSA certified red teams that they use across DOD to do this testing. And if we look at the broader, you know, civilian agencies, they just don't have enough people. And and in a time where we're reducing footprint of government staff and contractors, you know, more machine, testing is gonna need to be done. So we're gonna need to involve Agintiq AI.

17:34 We're gonna need to involve, scanning infrastructure that we've been using and just get get more, vulnerabilities detected sooner and faster. You know, if the pace is gonna pick up, you know, we're gonna have to, you know, meet that asymmetric threat with an asymmetric solution. And, you know, I'm a big fan of AI, fighting AI. And, you know, Synack is certainly working toward that for our customers. It's it's really a critical time to start to adopt the latest technologies before the adversaries do. Mark, it's great to have you on the program.

18:04 I really appreciate the chance to learn from you. Thanks for joining me today. Oh, thank you, Francis. Thanks for having me. You can read more about securing AI tools on today's show page at fed gov today dot com. A two volume cloud security playbook is the guiding document now for cloud security in the defense department. Civilian agencies are studying that playbook and other best practices for guidance on securing the clouds they use. You'll learn the latest best techniques for cloud security on cloud security in-depth. Industry and government leaders tell you what works for

18:35 securing your multi cloud environment. You can watch it on demand anytime at fed gov today dot com and on YouTube. The defense department has a data problem that's costing it both money and capacity. You'll learn why and how to solve the problem on the next Fed Gov Today podcast. Alyssa Siz of the government accountability office will be here tomorrow. You can listen to every Fed Gov Today podcast anytime you want on Apple Podcasts, Spotify, or on demand at fed gov today dot com. I'm Francis Rose. Thanks for listening.

19:05 Talk to you tomorrow.