Penetration Testing – But Why?

I once again tried my sketchnoting skills. In blue are findings from the paper, red are my own remarks.

I am a penetration tester – a legal, ethical hacker. But I am more comfortable with calling myself a security tester or a security analyst, or a SecDevOps professional.

The most common distinction between vulnerability assessment and penetration testing is that the former is automated and the latter manual. However, that’s an over-simplification. Reading this excellent research paper (“Does penetration testing need standardisation?”, Knowles, Baron, McGarr, 2015), the delivery of penetration testing services are of varying type and quality. Specifically communicating and fixing the findings often fall short. And truly  – isn’t fixing the issues the whole point?


The certificate I’m studying, Offensive Security Certified Professional, teaches how to perform exploitations end to end – from reconaissance to remote shell. This is a very useful skillset, because it requires you to apply the Hacker Mindset and truly understand the OWASP TOP-10.

However, the bulk of security work in the field is not fancy pwnage and haxx0r exploit development. In most people’s minds, however, this is what security professionals do all day long, and so they think that deep knowledge in that particular skillset is essential.

According to the paper, many companies don’t want actual exploit development in their penetration test. An intrusive penetration test can cause disturbances on the system. And a reasonable management is likely to accept a vulnerability, if the pen tester can present a plausible scenario in words, also without working exploit code.

What really upsets me in this study is the poor quality of the reports:

“Often basically a Nessus output in PDF format” (Knowles, Baron, McGarr, 2015, page 14)

Translated into human readable: any student in my higher vocational studies could do that in the first semester, but without the prize tag of specialist consultants.

The customer expects and pays for a human’s skill: prioritization, attack scenarios, root cause analyses. Such a delivery is downright unethical. The “ethical” in ethical hacking is not only about having permission, but also to deliver what you’re paid for.

Penetration Test Checklist:

  • Do you know what your assets and business values are?
  • Do you already have someone in your organization that has and takes responsibility for security? (CISO, a security minded sysadmin or developer)
  • Do you have a process for prioritizing bugs in general?
  • Are you working towards CI and DevOps?
  • Do you have a nice, prestigeless team?
  • Are you updating your software?
  • Are you running vulnerability scanners?
  • Do you have a vulnerability disclosure program?

If all those boxes are ticked: Congratulations! Performing penetration tests is a cost-effective and smart thing to do. If not, you should probably start somewhere higher up in the list.