Security testing
AI penetration testing
AI penetration testing uses AI agents to do what a human pentester does: map a target, decide what to try, attempt exploits and check whether they worked. It makes testing fast and cheap enough to run after every release, not once a year. It has clear limits, and like every pentest it attacks only the system that is running.
Updated
How an AI pentest works
- Reconnaissance. The agent crawls the application, reads its API descriptions and maps the endpoints, parameters and roles.
- Hypotheses. A language model reasons about where weaknesses are likely, from what the application reveals.
- Attempts. The agent sends requests, adapts to the responses and chains steps, as a person would.
- Validation. Good tools only report what they proved, with the request and response as evidence, which keeps false positives down.
- Report and retest. Findings arrive in minutes or hours, and a fix can be retested straight away.
Automated, AI and human pentesting compared
| Automated scanner | AI pentest | Human pentest | |
|---|---|---|---|
| Method | Fixed checks and signatures | An agent that plans, attacks and adapts | An expert who plans, attacks and adapts |
| Exploits findings | No | Yes, within its guardrails | Yes |
| Business logic | No | Some | Yes |
| Speed | Minutes | Minutes to hours | Days to weeks |
| How often | Every build | Every release | Once or twice a year |
| Cost per run | Lowest | Low | High. See penetration testing cost |
What AI pentests are good at
- Common web and API weaknesses: injection, cross-site scripting, broken access control between objects and roles, exposed data.
- Coverage: every endpoint, every release, without getting tired or skipping the boring parts.
- Fast retests: a fix can be checked the same day, not at the next engagement.
Where AI pentests fall short
- Business logic. Knowing that a refund should never exceed the original payment takes an understanding of the business that agents often lack.
- Impact. An agent can prove a weakness, but judging what it means for your company still needs a person.
- Safety and scope. An autonomous agent sending real attacks needs strict guardrails, test accounts and a scope it cannot leave. Ask how a tool prevents destructive actions.
- Your data. Responses from your application may reach a model provider. Check where the tool runs and what it keeps.
- Compliance. Auditors and customers often still expect a human-led test. See penetration testing requirements.
Attackers use AI too
The same capabilities work for attackers: AI agents can probe applications and turn a fresh advisory into a working exploit faster than before. That shortens the time you have between a vulnerability being published and it being used, which is the strongest argument for testing more often, and for knowing within hours which of your releases contain a newly vulnerable package.
AI writes the code, too
AI coding agents now open many of the pull requests that reach production, and they add and upgrade dependencies as they go, sometimes suggesting packages that do not exist and that attackers then register. See slopsquatting and supply chain checks for AI pull requests. An AI pentest of the running app does not see any of this. It sees the app, not what went into the build.
What an AI pentest never looks at
CRACI does not run penetration tests, AI or otherwise. It covers the layer they skip. CRACI runs your GitHub Actions jobs on its own runners and records every package each build pulls in, including the hidden dependencies that no lockfile lists, such as code fetched by install scripts. Each release gets an SBOM that is checked again as new vulnerabilities are published, and an egress policy blocks a build from connecting to hosts it does not need, so a compromised dependency cannot send your secrets out. Pair it with AI pentests of the running app, and you test both what you run and what you built.
AI penetration testing: frequently asked questions
What is AI penetration testing?
Penetration testing done by AI agents that map an application, decide what to try, attempt exploits and validate the results, with little human input.
Is AI pentesting the same as automated penetration testing?
Not quite. Classic automated testing runs fixed checks. AI pentesting plans and adapts like a person, chains steps and validates findings by exploiting them.
Can AI replace human pentesters?
Not yet. AI covers common weaknesses quickly and often, while people are still better at business logic, impact and complex attack chains. Most teams use both.
Is AI penetration testing safe to run against production?
Only with strict guardrails: a defined scope, test accounts and protection against destructive actions. Many teams run it against staging first.
Does an AI pentest find supply chain attacks?
Rarely. It attacks the running application, while supply chain attacks often run during the build. Recording what each build pulls in covers that layer.
Test what you run, record what you build
Record every package your GitHub Actions builds pull in and see which releases each new vulnerability affects.
Book a demo