Coding agents such as Copilot, Claude Code, Cursor, Devin and Codex now open pull requests on their own. A person still reviews them, and that person reads the diff: the logic, the tests, the naming. That review is worth doing, but it is not where most supply chain risk shows up.
The changes that can hurt you most are the ones a diff shows worst. A one-line edit to a manifest can pull in a package nobody has heard of. A lockfile change runs to hundreds of lines nobody reads. An install script never appears in the diff at all, and neither does a file the build downloads at build time. This post covers the checks a team can put between an agent's pull request and a release, and what each one catches.
The diff is the wrong place to look for supply chain risk
Code review was built for code a person wrote and can explain. With an agent, the author can't tell you why it chose a package, and the reviewer is tempted to approve a change that compiles and passes its tests. Four kinds of change deserve more attention than they get:
- New or changed dependencies. The manifest line is short; what it brings in is not.
- Lockfile churn. Hundreds of resolved versions and hashes, usually collapsed by default in the review UI.
- Things the build does, not things the repository contains. Install scripts, build scripts that download, base images,
curlin a workflow step. - Workflow files. A change to
.github/workflows/changes what runs with your secrets.
None of these need the agent to be compromised. A model that writes plausible code also writes plausible package names, and a plausible name is all an attacker needs.
New dependencies: check that the package is real
Language models sometimes suggest packages that don't exist. A study presented at USENIX Security 2025 generated 576,000 code samples with 16 models and found that at least 5.2% of the packages suggested by commercial models, and 21.7% by open source models, did not exist, with 205,474 unique invented names. Registering those names and waiting is called slopsquatting. It is typosquatting without the typo, and an agent that installs packages on its own removes the person who might have noticed.
Before merging a pull request that adds a dependency, check:
- That it exists and is the one you meant. Compare the name with the project the agent was trying to use, character by character.
- How old it is. A package or version published days ago deserves suspicion. pnpm's
minimumReleaseAgesetting refuses versions published less than a set number of minutes ago, including transitive ones. - Who is behind it. A linked source repository, a release history and more than one maintainer are not proof of safety, but their absence is a warning.
- Whether you needed it. Agents add packages for things the standard library or an existing dependency already does. Fewer dependencies is the cheapest control there is.
GitHub's dependency review action runs on pull requests, lists the dependency changes and can fail the check on known vulnerabilities or disallowed licenses. It does not know about a package published yesterday with no advisory, so it complements the checks above rather than replacing them.
Lockfile changes: read them, and know what they leave out
A lockfile pins the exact versions and integrity hashes your package manager resolved. In an agent's pull request it is the most precise record of what changed, so make it visible: expand it in review, and fail the build when the lockfile and manifest disagree (npm ci, pnpm install --frozen-lockfile).
Two things to look for:
- A changed source. A package that used to resolve from your internal registry and now resolves from the public one is the pattern behind dependency confusion. Scope internal packages to your registry so the public one is never asked.
- Transitive jumps. A small direct upgrade can move dozens of transitive packages. Those are the versions nobody chose.
A lockfile still records only what the package manager resolved. It says nothing about what an install script downloads, what a Rust build script fetches, which base image a Dockerfile pulls or what a workflow step fetches with curl. These are hidden dependencies, and they ship in the product all the same.
Install scripts run before anyone has reviewed anything
Here is the uncomfortable part: CI runs on the pull request. The moment the agent's branch is built, its new dependencies are installed, and their preinstall and postinstall scripts run with whatever the job can reach. Review happens afterwards.
The Nx compromise in August 2025 showed where this goes. Malicious Nx versions on npm carried a postinstall script that, according to the maintainers' postmortem, tried to use local AI tools such as Claude and Gemini to search the machine for sensitive files.
Ways to narrow it:
- Don't run lifecycle scripts by default. pnpm treats dependency build scripts as unreviewed until you allow them; with npm,
--ignore-scriptsturns them off. - Hold back CI on agent pull requests until someone looks. GitHub's Copilot cloud agent does this by default: workflows are not triggered until a user with write access approves the run.
- Keep secrets out of pull request jobs. A test job rarely needs a publish token. See GitHub Actions secrets: how they leak and how to lock them down.
Compare what the build fetched with the last release
The strongest gate doesn't look at the pull request at all. It looks at what the build did: which packages and files the job actually fetched, and from where. A build-time SBOM records that while the build runs, so it includes the hidden dependencies a lockfile leaves out.
Diff it against the SBOM of your last release and the questions become concrete:
- Which components are new, and which changed version?
- Did a package start coming from a different source?
- Did the build download anything from a host it never contacted before?
Anything new should have a line in the pull request description that explains it. A component nobody can explain is the one to stop for. For the ways to produce an SBOM in CI, see how to generate an SBOM in GitHub Actions.
Workflow files need their own owner
An agent that edits a workflow can change what runs with your tokens and secrets. Treat .github/workflows/ as security-sensitive code with its own reviewers. GitHub's secure use reference suggests putting the directory under CODEOWNERS:
/.github/workflows/ @your-org/platform-security
In those changes, look for:
- Wider permissions. A new
contents: writeorid-token: writeshould have a reason. See GITHUB_TOKEN permissions. - New triggers.
pull_request_targetruns with a read and write token and the repository's secrets, even for pull requests from forks. - Unpinned actions. A new action referenced by tag rather than full commit SHA can change under you.
- Secrets in new places. A secret passed to a step that didn't need it before.
Limit what the build can reach
Every check above can miss. A brand-new malicious package has no advisory, a convincing name and a clean diff. What it needs is a network connection: to fetch its second stage, or to send your secrets somewhere.
An egress policy allows the build to reach the registries and hosts it needs and nothing else. It doesn't depend on knowing the attack, so it works the first time a new one runs in your pipeline.
Keep a person in the approval path
Agents should not be able to approve their own work, and neither should the person who asked for it. GitHub's Copilot cloud agent prevents the user who asked it to create a pull request from approving it, and existing branch protections and required checks still apply.
Make the same true for every agent with rulesets on the default branch:
- Require a pull request and at least one approval.
- Require review from code owners, so workflow and dependency changes reach the right people.
- Dismiss stale approvals when new commits change the diff, since agents keep pushing after review.
- Require status checks, including the dependency and SBOM checks above.
Some tools move the gate earlier. AI CodeOps, for example, opens a pull request only once the test suite has passed on that exact version, an AI review of the diff has come back clear and a person has approved it.
Gate the release, not just the merge
A merge gate sees one change. A release gate sees everything that changed since the last release, including dependencies that were fine when merged and have since been found vulnerable. Before shipping:
- Re-check the release's components against current advisories, and block on known critical vulnerabilities and known malicious releases.
- Check licenses of everything new since the last release.
- Keep the record of what the release build fetched, so you can answer "were we affected?" later. See Are we affected? How to assess a new CVE across your builds.
Where CRACI fits
CRACI doesn't review the code an agent writes. It doesn't detect new malicious packages either, so vet new dependencies with the checks above. What it covers is the build.
CRACI runs your GitHub Actions jobs on its own runners, about twice as fast as GitHub-hosted runners. You change runs-on to craci, and runs still appear in GitHub. As each job runs, CRACI records every package it actually pulled in, including packages restored from CI caches and hidden dependencies such as code downloaded by npm install hooks: a recorder, not a scanner. Each job's SBOM carries a completeness state that flags what CRACI could not see, and exports as CycloneDX or SPDX, so you can compare a pull request build with your last release. A built-in release diff is on the roadmap.
On the runner, an egress policy decides which hosts a job may reach. It is validated before the job starts, fails closed and can be set to default deny, so a malicious dependency in an agent's pull request can't reach a host the build doesn't need, even if nobody has seen the attack before. Policy violations trigger alerts. Policy gates can block a build on findings, including builds that contain a specific CVE, and license policy rules can block a build on a license. After release, CRACI keeps re-evaluating the SBOMs it monitors against new advisories, and build history shows which builds contained a vulnerable dependency.
Book a demo to see the record of one of your own builds.
