Security testing
Penetration testing
A penetration test, or pentest, is an authorized attack on your own systems. A tester, and now often an AI agent, tries to break in the way a real attacker would, within an agreed scope and time, and reports what worked. It shows which weaknesses can actually be exploited, not just which ones might exist. It also tests the system as it stands on the day of the test, which is where its blind spot lies.
Updated
What a penetration test is
A pentest answers one question: could an attacker get in, and how far could they go? Unlike a vulnerability assessment, which lists known weaknesses, a pentest tries to exploit them, chains them together and proves the impact with evidence. Every test needs written authorization from the owner of the systems. Without it, the same activity is an attack.
Types of penetration testing
| Type | What it attacks |
|---|---|
| External network | Everything reachable from the internet: exposed hosts, services, VPNs and mail |
| Internal network | What an attacker could do from inside, after phishing a laptop or plugging into the office network |
| Web application | Login, sessions, access control, injection and the application's business logic |
| API | Authentication, authorization per object and per role, input handling and rate limits |
| Mobile application | The app on the device, its local storage and the backend it talks to |
| Cloud | Identities, permissions, storage and the configuration of AWS, Azure or Google Cloud |
| IoT and embedded | Firmware, hardware debug ports, radio protocols and the update mechanism. See IoT penetration testing |
| Social engineering | People: phishing, phone pretexts and physical access |
Internal vs external: an external test starts from the internet with no access, an internal test starts from a foothold inside. Most programs need both, because many breaches begin outside and do their damage inside.
Black, grey and white box: a black-box tester gets nothing but a target, a white-box tester gets source code, architecture and credentials, and a grey-box tester gets some of each. Grey and white box find more in the same number of days, because the tester spends less time on reconnaissance.
Penetration testing methodology
- Scope and rules of engagement. What may be attacked, when, from where, and what is off limits.
- Reconnaissance. Mapping the targets: hosts, endpoints, technologies and people.
- Scanning and enumeration. Finding candidate weaknesses with tools and by hand.
- Exploitation. Proving which weaknesses really work, and chaining them.
- Post-exploitation. Showing the impact: data reached, privileges gained, systems pivoted to.
- Reporting. Findings with severity, evidence and fixes. See how to read and structure a pentest report.
- Retest. Confirming that the fixes hold.
Testers usually follow a published method, such as the OWASP Web Security Testing Guide for web applications, the Penetration Testing Execution Standard or NIST SP 800-115, so results can be compared from one test to the next.
AI penetration testing
AI agents now run large parts of a pentest on their own: they map an application, form hypotheses, try exploits and check whether they worked. They are fast, cheap per run and can test after every release instead of once a year. They are weaker at business logic, at judging real-world impact and at knowing when to stop, so most teams pair them with a human tester. See AI penetration testing for how it works and where it falls short.
Human or AI, a pentest attacks the system that is running. That is its strength, and it is also why it misses the part described below.
What a pentest doesn't test: the build
A pentest looks at the deployed system on the day of the test. It does not look at how that system was built, and it does not see what changes after the test ends.
- Dependencies change every week. The release you ship the week after a pentest can pull in different package versions from the one that was tested. A clean report says nothing about them.
- Some attacks never touch the running app. Recent supply chain attacks ran inside build pipelines: a compromised package's install script or a poisoned CI action reads the build's secrets and sends them out. The production system looks clean to any tester, because the damage happened in CI.
- Hidden dependencies. Code downloaded at build time by install hooks or build scripts appears in no lockfile. A tester working from the running system or the repository cannot list it.
- New vulnerabilities. A component that passed in March can have a critical CVE in April. The pentest report does not update itself.
How often to run a penetration test
Once a year and after any significant change is the common baseline, and it is what PCI DSS asks for. Teams that ship daily add automated or AI testing between the annual tests. See penetration testing requirements by framework for SOC 2, ISO 27001, PCI DSS, GDPR and the Cyber Resilience Act.
A penetration testing policy
A short written policy keeps tests consistent and auditors satisfied. It should say:
- which systems are tested, and how often;
- who may test them: internal staff, an outside firm or an AI tool, and how independent they must be;
- who signs the authorization and the rules of engagement for each test;
- how findings are rated, who owns each fix, and the deadline per severity;
- when a retest is required, and where reports are kept and who may see them.
Covering the time between pentests
CRACI does not run penetration tests. It covers what a pentest leaves out. CRACI runs your GitHub Actions jobs on its own runners and records every package each build pulls in, including the hidden dependencies that no lockfile lists. You get an SBOM of every release, the SBOMs are checked again as new vulnerabilities are published, and the build history shows which releases contained a vulnerable package. An egress policy blocks a build from connecting to hosts it does not need, so a compromised dependency cannot send your secrets out, even in an attack nobody has seen before.
Hand the SBOM to your tester as well: a grey-box tester who knows every component spends less of the budget on reconnaissance.
Penetration testing: frequently asked questions
What is penetration testing?
An authorized, simulated attack on your own systems that tries to exploit weaknesses the way a real attacker would, within an agreed scope, and reports what worked with evidence and fixes.
What is the difference between a pentest and a vulnerability scan?
A scan lists known weaknesses automatically. A pentest tries to exploit them, chains them together and proves the impact. Scans are cheap and frequent; pentests are deeper and less frequent.
How long does a penetration test take?
From a few days for a small web application to several weeks for a large network or a multi-role platform. Reporting and the retest come on top.
Can AI replace penetration testers?
Not yet. AI agents find many common weaknesses quickly and can test after every release, but people are still better at business logic, judging impact and complex chains. Most teams use both.
How often should you do a penetration test?
At least once a year and after significant changes. Teams that release often add automated or AI testing in between.
Does a penetration test find supply chain attacks?
Rarely. A pentest attacks the running system, while supply chain attacks often run during the build and leave production looking normal. Recording what each build pulls in covers that gap.
Cover what happens between pentests
Record every package your builds pull in and track new vulnerabilities in every release, every day.
Book a demo