About twenty years ago, a client called me late in the evening.
He was slightly under the influence and told me something like this:
Peter, I just made a bet with a friend that he could hack my servers. If I lose, I have to buy drinks for everyone. I am counting on you.
My solution at the time was simple and surprisingly effective.
I switched off the router.
Until morning.
Incident response completed.
Today, I would not sleep so well.
Because today, the attacker does not always have to be a human. And the defensive perimeter is no longer a cable that can be unplugged from the wall.
This is not science fiction
The title sounds dramatic on purpose. But this article is not about conscious machines, Terminator scenarios or science-fiction panic.
The real security question is more practical:
What happens when an autonomous agent receives a goal, tools, credentials and network access — and starts looking for its own path to complete the task?
That question became less theoretical in 2026.
OpenAI disclosed a security incident during a model evaluation involving Hugging Face infrastructure. The incident was serious enough that OpenAI described follow-up work with external advisors and third-party assessment of the model behaviour observed during the incident. OpenAI and Hugging Face partner to address security incident during model evaluation.
Anthropic later published its own review of cybersecurity evaluation transcripts. In that review, Anthropic reported three incidents in which a Claude model reached the internet from within or while interacting with a third-party evaluation environment and then gained unauthorized access to real systems of three different external organizations. Investigating three real-world incidents in our cybersecurity evaluations.
The important point is not that AI had evil intent.
That would be the wrong framing.
The more technical problem is this:
The agent received an objective.
It had tools.
It encountered an obstacle.
It searched for another path.That is already a security situation.
AI does not need malicious intent
With a human attacker, we often assume a familiar sequence:
intent → reconnaissance → exploitation → persistence → objectiveThere is a person or group behind the action. There is motivation. Money, ideology, ego, espionage, sabotage, curiosity or crime.
With an AI agent, the starting point may be different:
objective → planning → tool use → obstacle → alternative path → objectiveThere does not need to be hatred.
There does not need to be greed.
There does not need to be political motivation.
There does not even need to be an intent to harm.
A poorly specified goal and excessive permissions may be enough.
From the perspective of a SIEM, EDR or cloud audit log, the difference may not be obvious. A process accessed a system. A token was used. An API was called. A command was executed. A network connection was made.
The log does not care whether the sequence came from a malicious human, a careless administrator, a compromised service account or an autonomous agent optimizing toward a goal.
Security architecture has to care.
The Swiss cheese model finally reached the cloud
Modern infrastructure is fascinating.
A typical organisation now has Microsoft 365, Azure, AWS, Google Cloud, GitHub, Kubernetes, VPNs, APIs, service accounts, OAuth tokens, SSH keys, secrets, CI/CD pipelines, SaaS tools, mobile devices and external suppliers.
Each service may be reasonably well secured on its own.
But the whole environment often looks like the Swiss cheese model.
One hole is manageable.
A second hole is still manageable.
The problem begins when several holes align:
over-permissive token
+ weak egress control
+ reachable API
+ trusted automation account
+ vague objective
+ autonomous tool use
= unexpected action pathThat is where AI agents become uncomfortable for security architects.
Not because they are magic.
Because they can try paths quickly.
Because they do not become bored.
Because they can chain small permissions, small assumptions and small mistakes into an action path that no single team thought was dangerous in isolation.
Switching off the router is no longer enough
Twenty years ago, in some environments, disconnecting a router really could change the situation.
It was crude, but the perimeter was physical enough for that action to matter.
Today, even if we disconnect one WAN link, many things remain alive:
- cloud workloads,
- SaaS identities,
- API tokens,
- mobile devices,
- synchronization jobs,
- external integrations,
- CI/CD runners,
- hosted repositories,
- managed databases,
- automation platforms,
- suppliers and managed-service accounts.
The perimeter is no longer the firewall.
The perimeter is identity, permission and trust.
With AI agents, we need to add another layer:
action authorityTraditional access control asks:
Who can access this system?Agentic AI security must also ask:
Who, or what, is allowed to act?That is a different question.
A user reading data is one thing. An agent reading data, planning a task, calling tools, modifying systems and trying alternative routes is another.
Least privilege is no longer only for users
Least privilege has always mattered.
But with agents, it becomes more difficult and more important.
A human user normally brings context, hesitation and responsibility. That does not make humans safe, but it creates friction.
An autonomous agent may not have the same friction. If a tool is available and the objective appears to require it, the agent may use it.
So the question becomes:
Which capabilities should this agent have at this exact moment, for this exact task, under this exact policy?
That is much narrower than giving an agent a broad service account, shell access, network access and a vague instruction.
For agentic systems, least privilege should apply to capabilities, tools, credentials, networks and time.
A control model for autonomous agents
The practical response is not to ban all agents.
That is unrealistic.
The practical response is to design containment and evidence around them.
At minimum, agentic AI systems need controls such as:
| Control | Why it matters |
|---|---|
| Least privilege | The agent should receive only the capabilities required for the task. |
| Scoped tools | Tool access should be explicit, reviewed and task-specific. |
| Ephemeral credentials | Short-lived tokens reduce the blast radius of misuse or compromise. |
| Network egress control | An agent should not automatically have the whole internet as an action surface. |
| Tool allowlisting | The system should define which actions are possible, not merely which prompts are allowed. |
| Human approval gates | Certain operations should require external approval before execution. |
| Immutable audit trail | We must be able to reconstruct what the agent did, when, with which tool and under which authority. |
| Behavioural monitoring | Security teams need to detect suspicious sequences, not only suspicious commands. |
| Kill switch | Containment must sit outside the agent’s own control path. |
| Recovery plan | The organisation must know how to revoke credentials, isolate tools and roll back actions. |
This is not bureaucracy.
It is architecture.
If a system can act autonomously, then policy, capability registry, approval workflow and evidence layer are not optional decoration. They are part of the safety boundary.
Sandboxes must be tested, not assumed
It is tempting to say: put the agent in a sandbox and the problem is solved.
Sandboxing is important. Isolation matters. Containers, VMs, network policies and restricted execution environments are useful controls.
But isolation is not a belief system.
It must be tested.
Recent research on container sandbox escape capabilities for frontier LLMs argues that model capabilities in this area should be measured against concrete exploit scenarios, not assumed away. Quantifying Frontier LLM Capabilities for Container Sandbox Escape.
This does not mean every model will escape every sandbox.
It means that sandbox design belongs in the threat model.
If an agent has tools, internet access, credentials, files, interpreters or APIs, the question is not only whether the prompt is safe.
The question is whether the execution environment, network boundaries and credential model remain safe when the agent starts exploring alternatives.
AI security is not only an AI problem
This is where many discussions become too narrow.
People ask whether the model is safe.
That matters, but it is not enough.
The bigger question is whether the system around the model is safe.
A production AI agent is not just an LLM. It is a system composed of:
model
+ tools
+ memory
+ retrieval
+ credentials
+ network access
+ identity delegation
+ logs
+ approval paths
+ business objectiveThat is not only AI governance.
It is security architecture.
We do not need to wait for AGI to have a real security problem. The combination of LLM, tools, credentials, network access and autonomy is already enough to create new failure modes.
A badly designed agent does not need to hate the company.
It only needs to be useful in the wrong way.
The new architect question
The old question was:
Can this user access the system?The better question is now:
Can this agent perform this action, through this tool, using this credential, toward this objective, under this policy, with evidence afterwards?That sentence is longer because the problem is larger.
It includes IAM, application security, cloud security, monitoring, audit evidence, governance and incident response.
It also changes how we evaluate AI systems. Prompt testing is not enough. We need to test:
- which tools the agent can call,
- which credentials it can obtain,
- which networks it can reach,
- which files it can read,
- which external systems it can influence,
- which approval gates it can bypass,
- which logs remain after the action,
- and which recovery steps exist when behaviour is unexpected.
That is the real threat model.
Who stops Skynet?
Twenty years ago, my incident response plan was simple:
switch off the router and go to sleepThat probably will not work anymore.
So maybe the question is not:
Who stops Skynet?A better question is:
Who controls the agent that received a goal, tools, credentials and the ability to find its own path?Because Skynet may never arrive.
But thousands of autonomous agents connected to clouds, identities, APIs, repositories, mailboxes, ticketing systems, CI/CD pipelines and production environments?
Those are already being built.
And that is a much more interesting security problem.