09 March 2017

What’s Your Pen Tester’s “Word” Worth?

Derek Athy

Now, let’s not sell Tommy Boy short – in a triumphant victory, he does get the euphemism right later on in the movie:

“I can get a good look at a T-bone by sticking my head up a bull’s ass, but I’d rather take a butcher’s word for it.”

But, when it comes to security, at what point do you settle for someone’s “word”, with little-to-no proof or validation?

Let’s get creative:

  • replace the T-bone with your pen test team’s report,
  • the “bull’s ass” with their pen test “checklist”, and
  • the butcher with the pen test team …

…. Do you rely solely on the team’s “word” for what they did exactly?

Extreme analogy? By characters, yes. By concept, no.

Whether your regular “hired gun(s)” is independent, from a respected security company, a world-renowned consulting firm, or even a number of bug bounty participants, at the end of the day, you’ve been placing a lot of faith – and money – in just that – their word (albeit, probably delivered more professionally than Tommy Callahan here). Traditional penetration testing engagement and bug bounty programs simply lack visibility into just how much of the assessment scope was actively targeted by the personnel performing the testing, as well as the nature of the techniques deployed – leaving you to trust and rely on their word and reports.

Pen testers are paid the same regardless of the results; what’s preventing them from stopping once they’ve reached the “good enough” checkpoint? When bug bounty reports stop pouring in, is it because you’re all of a sudden “secure”, or because the efforts have instead focused on the program paying 10x what you do for vulnerabilities on that same platform? Essentially, you’re left in the dark unable to answer these questions, or simply stuck with a final report that’ll look a little something like:

Coverage: < www.examplescope.com >

… but how much of the attack surface was actually tested, how many of the associated URLs and subdomains were targeted?

Testing Effort/Methodology: A checklist of vulnerability categories searched for and attack types attempted

… but how much time was actually spent on penetration testing and vulnerability discovery efforts? How much effort was actually focused on specific attack techniques?

Results: Vulnerability reports prioritized based on common or custom vulnerability scoring system and “handed off” in final report

… but how resistant is the rest of the in-scope attack surface in comparison?

… but were no vulnerabilities reported in additional areas of the attack surface because they’re more “hardened”, or because they received no attention?

Quit bullshitting the answers to these questions, or having nothing but “word” + summary reports – not quantitative data – to back you up. Synack’s recently launched Coverage Analytics feature is here to bail you out by bringing front-and-center the analytics and metrics that security assessments have too long gone without.

Say no to vague reports, say yes to auditability!

Synack’s Coverage Analytics provides organizations with true visibility and transparency into the extent of attack surface coverage, showing them exactly when/what/how applications and assets have been targeted, in addition to the contextual vulnerability data that results. Essentially, Synack’s Coverage Analytics feature lets clients see:

  • How many Synack Red Team (SRT) researchers have actively participated, and how many collective hours of testing have been put in
  • Proof of exploitation effort and attempts, as Synack’s proprietary technologies begin to classify, with real data, the number and classification of attack techniques attempted by SRT members
  • Exactly when and what applications and assets across their in-scope attack surface were targeted, or “hit”, and subsequently compare resiliency & vulnerability

And as Synack continues to develop proprietary attack classification algorithms and further build out our real-time coverage mapping capabilities – Synack clients and Synack Red Team members are both to benefit.

Whether it’s displaying more and more valuable information, analytics, and data-backed results of Synack security testing assessments to our clients that can be communicated across almost any arm of the organization – security or not – or better targeting SRT efforts toward areas of the attack surface that are more likely to be vulnerable to certain attack techniques (and subsequent bounties!), our aim is to drastically improve the Synack experience for both our clients, and our Synack Red Team members.

Learn more about Synack’s Coverage Analytics feature, or contact Synack to reap the benefits yourself!