Scoping Adventures is a series of blogs about some of the more interesting penetration tests that the Synack Customer Success teams have worked on over the last few months. Each blog outlines how we engage with the client to achieve the best results from a pentest.
Pentesters love colors—red, blue, purple, black, white and grey to name but a few. While red, blue and purple are given to different security testing teams, black, white and grey are given to different types of penetration tests. Pentests are classed based on the level of information, knowledge and access given to the tester prior to commencing the test.
It can quickly become overwhelming to know what type of testing will achieve your security goals.
Looking at the different testing methodologies, such as black box vs white box and authenticated vs unauthenticated, explains why scoping is not always that simple. To get the most out of your pentesting, Synack’s Customer Success team can help you scope your tests with precision and care.
The Synack Red Team (SRT), our global community of 1,500 security researchers, performs Synack’s offensive security testing along with our intelligent platform and internal support teams, like Customer Success. The SRT are typically working with a cohort, so when you set up a pentest with Synack, it’s never just one or two pentesters. But for the sake of simplicity, we’ll refer to one pentester in these examples.
Black Box Testing
In a black box pentest, the tester acts similar to a typical hacker, where no internal information or knowledge of the target system is given. The tester is not provided with architecture diagrams, credentials or source code, unless these are publically available. A black box pentest searches for vulnerabilities in a system from outside the network.
The limited details that the tester is provided with makes black box tests the quickest to run, as it depends on the tester’s expertise and ability to discover and exploit any vulnerabilities. However, if the tester is unable to exploit these and breach the system perimeter, then the remainder of the system (and any other potential vulnerabilities) will not be found. If a potential attacker acquired this information by other means, or was provided with credentials, then there is a possibility the attacker could exploit vulnerabilities not discovered by the black box tester.
White Box Testing
A white box pentest is the complete opposite to a black box test, and the tester is usually provided with full access to architecture diagrams, credentials, databases and source code. While this can give the tester a big advantage, there is considerable work to sift through the documentation and data to identify potential vulnerabilities, thus making it more time consuming than a black box test.
White box testers will usually conduct more static analysis of the system, which involves analyzing the source code and other systems, as these can be a valuable source of information to detect vulnerabilities or misconfigurations that cannot be detected by dynamic analysis of running systems (as is done in a black box test).
Grey Box Testing
In-between black box and white box is grey box testing. While a black box tester has no knowledge of a system and a white box tester has a lot of knowledge, a grey box tester has some knowledge of a system, such as IP addresses, TCP ports and credentials.
The purpose of grey box pentesting is to conduct a more focused and efficient test of a system, where the tester can focus efforts on the part of the system which has the greatest risk and value, rather than spending time collecting this information themselves.
By providing the tester with credentials, tests beyond the public/unauthenticated part of the system can be tested, which simulates a scenario where a hacker would have spent a long time on the system poking around and perhaps gained entry. It also is a more common scenario as the information from a data breach, such as logins and passwords, are available for purchase on the dark web.
Engagement Accuracy
The accuracy (and potentially number of vulnerabilities), speed and efficiency of a pentest depends on the type conducted. As the accuracy increases, so does the cost of the engagement:
– Black box testing is usually the fastest type of pentest, but due to the limited information available to the testers, it increases the chance that vulnerabilities will be missed and thus decreases the efficiency of the test.
– White box testing is the slowest type of pentest and most comprehensive form of testing due to the amount of information available to testers, and therefore increases the chance that vulnerabilities will be discovered. However, it’s not guaranteed that the vulnerabilities surfaced are truly exploitable. More vulnerabilities doesn’t mean you’re focusing on the right ones.
– Grey box testing sits in the middle, with a trade-off in speed and accuracy. By providing testers with some information (and perhaps credentials) they can better utilize their efforts during testing.
Authenticated vs. Unauthenticated Testing
An authenticated test assumes that the tester has a valid set of credentials to login to the web or mobile application. During an authenticated pentest, the tester will use a mixture of manual and automated methods to explore which parts of the website or mobile app they can access. Not only will the tester have access to more of the site, but they can also try to gain access to pages and functions which should not be available, known as privilege escalation.
During scoping, the Customer Success Team always tries to acquire a set of credentials from the customer in order to improve the test results and allow the testers to discover more assets within the site. The more credentials a customer can supply, the easier it is to share them amongst the testers and allow them to test for vulnerabilities like IDOR attacks. However, it isn’t always easy to get these, for example:
– On production sites, the customers can only provide a single credential
– The site requires multi-factor authentication, thus locking a credential down to a single tester
– Credential verification/activations that expire within minutes, making it difficult to on board testers
– Customers unable to provide credentials to a protected site due to legal reasons
Unauthenticated
An unauthenticated pentest is done without any credentials, where the tester mimics the actions of a hacker who has no previous information about the site or mobile app. The scoping team receives a fair number of unauthenticated tests, which makes the scoping process easier, but doesn’t always make for a good test. A good comparison is an iceberg, where only a small part of it is visible on the surface, but the interesting parts are unseen below the surface.
Some customers will ask us to scope a site that requires authentication but will not provide credentials. Testing as an unauthenticated user involves looking for vulnerabilities on assets or pages that are public facing. One of the first tasks the tester will conduct is to attempt to login, bypass the login process or test for vulnerabilities in the authentication mechanism. However the number of possible testing techniques is low and the opportunity to discover vulnerabilities is even lower.
While some web apps allow users to access some or all their functionality without providing any credentials, most require some form of authentication to ensure users are authorized to access the content. From a scoping and testing perspective, a login page provides a boundary between the unauthenticated and authenticated parts of the site, and the goal of any unauthenticated test is to get past the authentication process.
Authenticated
During an authenticated web app test, most of the same testing methods are used but with the advantage of having a working set of credentials to access restricted content. This additional content has the potential to expose the application to more exploitable vulnerabilities. This is why Synack recommends authenticated testing, ensuring even a malicious user or an attacker with leaked credentials cannot access the application.
As the tester will know the credentials to login, they can spend more time testing the application for role escalation vulnerabilities. Privilege escalation is the act of exploiting a bug, design flaw or configuration oversight in a web app to gain elevated access to resources that are normally protected from a user.
Putting It All Together
When tests are handed over to the scoping team at Synack, they may come in any shade of grey. Some customers may not provide any credentials or information about their assets for a variety of reasons such as:
– No details readily available to share
– Only wanting the public parts of a website and the login page tested
– Production site where credentials cannot be provided or shared
– Legal reasons and regulations based on the country
While this is suitable for a simple public website, it may not produce the results a client is looking for. For example, a banking website or mobile app’s functionality mostly happens after authenticating.
White box tests are rare, as customers rarely want to share their confidential data or intellectual property. When we do come across them, the amount of information provided is too much to be shared with our researchers, regardless we work with clients to ensure the right information is provided to the SRT. It is estimated that about 90% of the scoping done by the team is for grey box tests, where the customer will provide credentials, JSON API files or mobile apps with client side protection disabled.
Getting the Most Out of Your Pentest
There’s no doubt that scoping is one of the most important parts of a pentest, as it acts as a useful guide for the pentesters to understand what needs to be tested and will ultimately determine if the test will produce good results or not.
If the scope is constrained because of time, budget or incorrectly defined objectives, then problems will arise. However, if a pentest is over scoped then it can lead to overspending and poor staff time management. A scope that is too narrow doesn’t allow the pentesters enough freedom to test thoroughly.
The Synack Red Team will have a finite amount of time on each assessment to find vulnerabilities, and their level of engagement with each test and what they discover will depend on the details given to them during the scoping process. To achieve the best results, we can help your organization balance the level and amount of information shared with our security researchers that will allow for thorough testing that surfaces exploitable vulnerabilities and reduces noise.
Dr. Huw Jones is an offensive security engineer and part of the Customer Success team.
