Federal Cyber Incidents vs Federal Cyber Budget
20 May 2018

An ROI Analysis of DHS Bug Bounty Initiatives

Anne-Marie Chun

There’s a common problem in security: cyber spending is increasing, but the rate of cyber incidents is not slowing down. Or simply put, more money, still more problems.

Just look at the uphill battle our nation faces. While federal cyber investments increased 162% from 2006 to 2018, the number of federal cyber incidents have increased over 1500% in the same time period.

Federal Cyber Incidents vs Federal Cyber Budget

One look at this chart, and it’s clear that we need a strategy with better ROI.

The Department of Homeland Security is quickly driving towards this with the new cyber strategy they released this week. The DHS cyber strategy aims to improve the “security and resilience across government networks and critical infrastructure,” while prioritizing cost-effective approaches that both reduce risk and achieve maximum ROI. The agency put together a forward-looking plan that emphasizes efficiency and effectiveness of security programs, all while keeping taxpayer dollars in mind.

There’s a lot of good in this strategy across its five pillars. One of the pillars that particularly nails it: the need to reduce federal systems’ vulnerabilities.

A healthy security program should see a reduction in the number of vulnerabilities and an increase in attacker resistance over time. Of course, with new releases and changes in the environment, new vulnerabilities will be introduced. The trajectory to security health may not always be linear, but it should be positive. Security teams should find that as their programs get stronger, it takes attackers more effort and more time to find vulnerabilities.

In the background, Congress is nudging agencies like DHS to adopt crowdsourced security programs to help agencies unearth their vulnerabilities. There are several models that agencies could adopt:

Landscape of Crowdsourced Testing

Before jumping in, agencies should define their goals up front and evaluate their options. DHS defined their goal as: “reduce vulnerabilities of federal agencies to ensure they achieve an adequate level of cybersecurity.” How will they measure progress towards this “adequate level”?

To reduce vulnerabilities, agencies should make sure that the crowdsourced security programs they adopt have the necessary pieces in place to not just find vulnerabilities, but also measure progress towards the goal of vulnerability reduction.

To date, security organizations have struggled to measure incremental security improvement or ROI, and the industry has failed to produce a reasonable metric or methodology to date. As an industry, we have a responsibility to help organizations measure their improvement and their ROI.

That’s why Synack launched the Attacker Resistance Score – It’s not good enough just to find all of the vulnerabilities you may have, you need to have a way to easily triage, prioritize them, and get a realistic measure of your relative and dynamic security risk from a hacker’s perspective. Enterprises and government agencies are already using this metric to measure their ROI, reduction in vulnerabilities, and increasing attacker resistance over time.

Watch Mark Kuhr explain how realistic risk scoring works: