Security risks of autonomous coding agents
An assistant that suggests code is a productivity tool. An agent that runs commands, reads web content and pushes to repositories is a new identity in the environment. The article shows why a low per-page injection rate becomes a high daily probability, and which controls reduce the risk.
The first AI coding tools suggested text and waited for the user. The current generation takes actions. Coding agents open files, run shell commands, call APIs, read issue trackers and web pages, and open pull requests.
This is a qualitative change. An agent now holds credentials and takes actions, and its instructions can come from anything it reads. Any third party can publish text that an agent may later read. This article examines how serious the risk is when each attempt rarely succeeds, and which controls reduce it.
The relevant guidance comes from several standards and papers.
OWASP's Top 10 for Large Language Model Applications (2025 edition) lists prompt injection first and excessive agency among the top risks. NIST's AI Risk Management Framework (AI 100-1, 2023) and ISO/IEC 42001:2023 give governance structure. The underlying security principles are much older: Saltzer and Schroeder's design principles (1975), including least privilege, still describe most of the controls that work. The probability argument below is elementary, and the evidence argument uses the rule of three (Jovanovic and Levy, 1997).
Four risk categories
- Prompt injection through content. An agent cannot reliably tell instructions from its user apart from instructions embedded in what it processes. A README, an issue comment or a web page can contain text written to redirect it: run this command, add this dependency, send this file there. The situation resembles a new employee who acts on any note left on the desk, including one that asks for an urgent payment in the name of the chief executive.
- Excessive agency. An agent given a broad token to "fix the build" can usually also delete branches, change CI or publish packages. When something goes wrong, the damage is bounded only by what it was allowed to do.
- Secret exposure. Agents work where credentials live: environment variables, config files, SSH keys. Anything the agent can read, it can put in a request, a log or a commit.
- Supply chain through tools. Plugins and tool servers run with the agent's permissions. Each is a dependency.
Controls
| Principle | In practice |
|---|---|
| Least privilege | Scope tokens to one repository and one task; read-only by default; no production credentials |
| Isolation | Disposable sandboxes with an egress allowlist |
| Human approval for irreversible actions | Merges, deployments, releases and deletions go through a person, typically as a pull request |
| Content is data | Everything the agent reads is untrusted input; fetched text never changes its permissions |
| Short-lived secrets | Credentials per session; no long-lived keys in the environment |
| Accountability and audit | A distinct identity per agent, with logs of every command and API call |
| Vetted tools | Pin and review plugins and tool servers like any other dependency |
The pull request deserves emphasis: it is where an agent's work becomes a proposal a human accepts, with review, checks and history already built in. An agent that can push straight to the main branch bypasses a control that most teams already operate well.
Cumulative probability of injection
Suppose an injection works on one page in a thousand. Over independent exposures, the probability that at least one succeeds is
which for is 63% after 1,000 pages and 95% after 3,000. An agent that browses all day reaches those numbers quickly. Since cannot be reduced to zero, the factor that can be controlled is the impact of the one that gets through: expected loss is probability times impact, and least privilege shrinks the second factor.
Organisations already apply this reasoning to phishing. An employee who receives thousands of emails a year will eventually click on a convincing fraudulent one, however good the training. Security teams therefore limit what one click can do: multi-factor authentication, no administrator rights on everyday accounts, payment approvals by a second person. An agent reading thousands of pages needs the same limits.
The figure below illustrates two points. A success rate per page that sounds negligible becomes a near-certainty over a realistic workload, and a red-team exercise without successes provides less assurance than it appears to.
- Default settings. At 0.1% per page and 1,000 pages, the chance of at least one successful injection is 63%. Press "Simulate a day of browsing" several times and count the days with a hit.
- Improved defence. Improve the defence tenfold, to 0.01%. About 7,000 pages are then needed for even odds, a volume that a busy fleet of agents can reach in a day.
- Red-team results. Suppose a red team ran 300 attacks and none worked. The rule of three says the true rate could still be about 1% (3/300). Set to 1% and look at what 1,000 pages would mean.
- Even odds. The readout "pages until even odds" is . The approximation can be checked at several settings.
The calculation. Each page independently fails to inject with probability , so all fail with probability , and at least one succeeds with probability . Since for small , this is about : the risk depends on the product , so doubling the workload has the same effect as doubling the per-page success rate.
A coding agent is a new identity in the environment and should be secured accordingly: least privilege, isolation, short-lived secrets, human approval for irreversible actions, and logging. Before an agent is given access, one question is useful: if a third party could write one paragraph that this agent will read, what is the worst action it could cause? The answer should then be reduced to an acceptable level, since over enough pages such a paragraph will eventually be written.
References
- J. H. Saltzer and M. D. Schroeder, "The protection of information in computer systems", Proceedings of the IEEE, 63(9), 1278-1308, 1975.
- OWASP, Top 10 for Large Language Model Applications, 2025 edition.
- NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, 2023.
- B. D. Jovanovic and P. S. Levy, "A look at the rule of three", The American Statistician, 51(2), 137-139, 1997.
- ISO/IEC 42001:2023, Information technology - Artificial intelligence - Management system.