Since my day job is all about hacking, I get a lot of questions (and there appears to be a lot of confusion) about what a vulnerability scan, penetration test or red team assessment is.
This article is my attempt to clear that a little bit - it was already published LeMagIT, but in French (https://www.lemagit.fr/tribune/Retirer-toute-la-valeur-des-evaluations-de-securite), so here's a version in English :-) With that said, here we go ... 

There are many aspects to managing vulnerabilities in today’s complex IT environments. Performing security assessments is a popular way of identifying existing vulnerabilities, which then allows for proper mitigation. In this article we look at the differences between vulnerability scanning, penetration testing and red teaming, three security assessments that are popular, but that should be performed with care, in order to achieve best results.

Deciding which security assessment to perform depends a lot on an organisation’s security maturity level, and the best results will be achieved by performing them exactly in the order listed – let’s see why.

Vulnerability scanning (assessments) is something that every organisation should be doing on a regular basis. This is the first, and the most basic activity in managing vulnerabilities: the goal of a vulnerability scanner is to find low-hanging fruit and known vulnerabilities or misconfigurations. Vulnerability scanners will do a great job of enumerating installed patches, finding default accounts and misconfigurations. Modern scanners can also authenticate against target systems (log in), which will allow them to list installed patches and correlate such information with actively obtained data from a scan, thereby reducing false positive reports.

It is recommended that vulnerability scanning is performed regularly in every organisation, preferably with internal tools. The most popular network vulnerability scanners are Rapid7 Nexpose, Tenable Nessus and Qualys. Just keep in mind that these should be used for network scanning, while other, more specialised tools exist for application level scanning (i.e. for web applications).

Penetration testing should be the next step. When deciding on a penetration test, scoping is very important: the goal of a penetration test is to find all (or at least, as many as possible) of the vulnerabilities in the target scope. The target scope can be a set of IP addresses or networks (when we talk about a network penetration test), a web application, web services and so on.

Since a penetration tester is normally limited in time (typically penetration tests last between 5-15 working days), it’s obvious that the whole process should be as efficient as possible. This means that the scope should be as clearly defined as possible, and that any potential obstacles should be removed. For example, if you are testing a mobile application, it would be beneficial to provide a build without obfuscation for your penetration tester: we know what obfuscation can be circumvented and by providing such a build, the penetration tester will not waste time on deobfuscation, but on finding vulnerabilities.

Additionally, a penetration test will also test for vulnerabilities that a vulnerability scanner cannot find – for example, logic vulnerabilities. Business logic vulnerabilities are almost impossible for a vulnerability scanner to identify (at least for today’s tools), and they can be devastating. A good example is, if you are a bank and have an Internet banking web application; an attacker logs in and tries to create a transaction of -100 EUR, resulting in the payer receiving the funds instead of a payee. Or, another very common category of vulnerabilities is Direct Object Reference (DOR), when attackers try to access objects (for example, transaction records) belonging to arbitrary users by modifying object references. Since object references are typically just numbers (IDs), modification is easy, yet many scanners will miss such vulnerabilities since they do not understand the context i.e. the fact that by changing an ID and retrieving someone else’s transaction record means we have found a critical vulnerability.

The result of a penetration test is a report that should detail all identified vulnerabilities, with recommendations on how to fix them. All listed vulnerabilities should be verified by the penetration tester – there should be no false positives there!

It is now clear that penetration tests require a lot of manual work – that’s why they take quite a long time and are typically more expensive than a vulnerability assessment.

Finally, a red team exercise is the ultimate test of any organisation’s defences. In a red team exercise, the attackers are given a final goal and they can typically use any means they want to achieve their goal. This might include writing new exploits, using social engineering, moving laterally and so on.

The main difference between a red team exercise and a penetration test is that with a red team exercise there is one ultimate goal, while a penetration test aims to find all vulnerabilities in the defined scope. A red team exercise might miss some vulnerabilities and never report them, but it will show you how you stand against a real attacker. Quite often a red team exercise is a good test of an organisation’s blue team – it will show how good they are in detecting attacks and potentially preventing them while they are happening.

Although the above is the ideal process for managing vulnerabilities within an organization, ultimately, deciding which security assessment to select also depends on the maturity level of the organisation. If, for example, the target organisation is not regularly patching its servers, there is no point in doing a penetration test or a red team exercise as even the basic security hygiene is lacking. This must first be addressed. It is only after low-hanging fruit has been taken care of, that a more sophisticated security assessment should be performed.

Likewise, if an organisation is releasing a new application, for example, then a penetration test might be more suitable since the goal will be to find all the vulnerabilities in the new application. And, if you have already tested the majority of your services, and have a trained blue team, a red team exercise will help show how prepared the organisation is against a real world attack.

--
Bojan
@bojanz
INFIGO IS