Gartner panel - CSC Global, GDIT, Montgomery County, Synack
17 July 2019

Smart Security Testing at Gartner: Aerospace, Defense & Local Governments get 4X ROI with AI Augmented Crowdsourcing


Montgomery County processes $5B in online transactions every year. General Dynamics Information Technology (GDIT) is a multi-billion dollar business. CSC Global serves almost all of the Fortune 500. What do their security teams have in common? A strong commitment to protecting their customers and their businesses with best-in-class cybersecurity programs. With an ever-changing business climate, the scale and complexity of digital transformation, increased regulations, and scare resources, it’s necessary to be innovative and creative in leveraging solutions that work smarter and more effectively than they have before.

Three industry pioneers — Mike Baker, CISO of General Dynamics Information Technology (GDIT), Scott Plichta, CISO of CSC Global, and Keith Young, CISO of Montgomery County Maryland — are having a big impact today by adopting crowdsourcing and artificial intelligence in their security testing practices. These three highly regarded CISOs joined Synack CEO and Co-Founder, Jay Kaplan, at the recent Gartner Security & Risk Summit to talk about how they are using new methods to scale their security testing to protect their businesses.

Gartner panel - CSC Global, GDIT, Montgomery County, Synack

Keith Young, CISO of Montgomery County Maryland Gov., “receives critical data from police officers – if that were to get out, someone could get killed. We cover and secure everything from a jail system to traffic lights, so it’s a complex, highly regulated environment, and we are doing it without a lot of money.” Riviera Beach and Lake City, both cities in Florida, recently paid out huge ransoms to a cyber gang to avoid 45,000 people from being forced to live completely off the grid with their PII exposed. These are the types of scenarios that Young thinks about often and actively works to defend his county against.

Scaling Security Testing with a Crowd

Traditional pen tests have failed to keep up with the ever-changing threat landscape; attack surfaces are increasing dramatically and new vulnerabilities are being introduced as quickly as new software is being built. Assigning two or three consultants to a testing program, no matter how skilled, just can’t keep up. Crowdsourcing ethical hackers from all over the world provides the skill and scale necessary to test multiple assets at the same time.

“Right off the bat, we knew it was a good idea to leverage crowdsourcing, augmented with AI technology, to help us solve the scale problem,” Mike Baker of GDIT, the leading-edge technology group within defense giant General Dynamics, remarked. GDIT is always looking for ways to adopt innovation and push for nontraditional methods of pen testing (such as crowdsourcing) to become more effective in offering their customers the best possible services.

Many companies have adopted crowdsourcing methods to replicate the diversity, creativity, and skill sets of malicious actors in the form of ethical hackers. However, most companies are now releasing code every two weeks with Agile methodology, and it can be challenging for humans to ensure continuous coverage on assets that are dynamically changing. To scale a crowdsourced security test, you need the best of humans and technology. As Scott from CSC Global put it, security testing “still needs humans, but you can use automation as a force multiplier so you can put your talented humans exactly where you need them.” Servicing almost all of the Fortune 55, CSC Global knows what it takes to protect their customers and stay committed to scale.

Scaling a Crowd with Artificial Intelligence

However, automation must be approached in a smart way. For many, attempts to scale through automated scanning or broad bug bounty programs have increased the noise factor and decreased efficiency, taking a toll on internal security teams. You need an effective testing platform that can run continuously without creating extra work on the internal resources. You need a solution that leverages the optimal balance of humans and technology and empowers the customer with critical insights and quality results.

“Vulnerability management is like shoveling sand against the tide. It’s hard to show metrics that showcase progress. Whether it’s using Synack or other prioritization mechanisms, how do we get to categorizing which assets are critical, high, or low?” Mike Baker posed to the audience. He’s been on the forefront of leading GDIT to think differently about how they build new technology in a smart, secure way.

It was clear during the discussion that more and more security leaders are looking for nontraditional, more effective methods of pen testing that allow them to filter through the noise that comes with scale. The US National Security Agency (NSA) is now using machine learning in automation to combat against this, and increase the efficiency of security teams that need to protect a large number of assets. “We’re going to need, at the very least, ML techniques to pull signal out of the noise so that the defenders, the operators can be informed [and] spend their time on the most critical events or anomalies rather than trying to make sense of this huge data space manually,” Neal Ziring, NSA’s Technical Director for Capabilities, told Cyberscoop. Customers are now looking to automation to help with prioritization in remediation and filtering of the noise.

In addition to helping filter through noise, machine learning-based methods of automation can also scale continuously. “I’m looking for automated, continuous feedback on where my environment is. Security today is still very much point-in-time testing and highly dependent on talent,” Mike explained. Adding a layer of automation across all assets augments crowdsourced humans and can serve as a change indicator in the attack surface, allowing coverage while still being able to leverage the creativity and discerning intelligence of human discovery in finding and understanding exploits.

Synack recognizes the dependency on talent given the continuously expanding scope of attack surfaces, but also recognizes the need for some augmentation with our Synack Red Team (SRT). More than 70% of the vulnerabilities that our Synack Red Team finds in customer assets aren’t detected by a scanner, but these talented humans can’t scale to cover trillions of applications all of the time. We need humans and technology together for the perfect combination. Synack has come out with a revolutionary proprietary AI-enabled scanner that continuously scans assets and alerts the SRT to any potential exploits, while the SRT also conducts open vulnerability discovery on top of these alerts. Synack’s optimized approach leverages crowdsourcing and automation together to provide noiseless scanning in tandem with comprehensive pen testing, making for a much smarter, continuous pen test.

Let’s take a look at some of the results the panelists found using Synack’s smart security test, harnessing both the crowd and automation. These tests were on a small number of assets. The results highlight where the security teams were able to save time, and focus on the most high-risk vulnerabilities.

CSC Global

“We have 78 scrum teams developing on a continuous basis. For us, we started with a typical 2 week pen test. Two things we learned from going crowdsourced: 1) The power of a surge of people 2) Difference in pay for performance vs pay for time spent. The deeper you get, the pen tester gets paid more and you see deeper uses!” said Scott Plichta, CISO of CSC Global.

General Dynamics Information Technology

GDIT, who acquired CSRA this past year, is a multi billion dollar business. “Scale was a problem and we’re working on solving it with crowdsourcing and automation,” explained Mike Baker, CISO of GDIT.

GDIT Synack Platform Stats:

  • 15,584 Findings via Automation
  • Automation piece filtered >99% of the noise and passed 34 potential exploits to the Synack Red Team
  • 32 hours of Synack Red Team triage and validation offloaded from internal team
  • 1 Exploitable and Human-Validated Vulnerability passed to GDIT security team (Additional 50% noise reduction)

A Smart Pen Test’s ROI

As thought leaders in the industry, Mike, Scott, and Keith encouraged the audience to focus on what’s important: security ROI. Scott talked about how much data “information solutions like Synack’s are putting in the hands of a SOC analyst. Before with traditional methods, we could only pick 10% of the environment. Now, for the same amount of money, we can hit 90% and understand where to put our resources.” The amount of data that AI/ML augment crowdsourcing provides to security teams and across organizations allows businesses to make more informative decisions on how to best use limited resources efficiently.

Because of this new technique, AI/ML augmented crowdsourcing, ROI on a pen test now has a different definition. Security teams are no longer overwhelmed by copious amounts of low impact vulnerabilities, or nervous for what they haven’t covered, but instead are cutting through the noise, and focusing precious remediation resources on human validated exploits. These days, it’s quality over quantity. As Mike stated, “I don’t see crowdsourcing as a choice – I see it as an evolution and will be mandatory as more people cannot maintain teams.”

Along with delivering ROI to executive boards, CISOs often have to lead the organization in thinking outside of the box and adopting innovative methods such as crowdsourcing. Gartner predicts that by 2021, over 50% of organizations will be using crowdsourcing and automation to secure their assets. Each CISO shared the most important elements to them when it comes to investing in crowdsourcing and automation as they look to convince their boards.

“In terms of controls – do you know who’s pen testing your sites by traditional methods? Can you play it back? Synack provides control – you can play it right back and figure out what happened at 3 am. We don’t get the same control with a traditional pen test,” said Scott. Keith talked about clearance checks, credentialing, and how having a rigorous process of validating ethical hackers puts management and legal departments at ease: “When I first brought up continuous pen testing, I envisioned a cartoon bubble photo of a hacker with a hoodie on and figured that was going through others’ minds. One of the biggest challenges is going back to management and legal and saying, ‘Here are the benefits of doing it this way.’” With Synack, a customer is able to control a crowd of ethical hackers start and pause penetration testing with a push of a button. It’s clear that adapting innovative methods have a long term ROI for smart pen testing. Synack has changed the view of what an ethical hacker is.

The combination of automation and crowdsourcing in penetration testing are clearly paving the way for smarter pen tests; this new model offers a solution to stay ahead of the malicious actors who continue to be creative, incentivized, and persistent. Neither machines nor humans are as effective on their own as they are together, but it’s important to couple the two together in a way that provides efficiency and control to security teams. As organizations like Montgomery County Maryland, GDIT and CSC Global continue to grow their digital presence, their security is becoming as smart and effective as ever.