I may lose some of you on this article, but please try to make it to the “so-what?”.
After last week's newsletter on AI strategy, I was inundated with replies, questions and comments.
Two things came of this.
Firstly, I booked a number of workshops so thank you to those people who have faith in me.
Secondly, a number of comments came in from my CIO community: "We are really worried about cyber AI. It's a real problem. Can we discuss?"
Safety is the number one thing to address in your AI strategy, so today we're going to tackle part one of that: cyber. (Part 2 was data safety, if you remember.)
The problem is speed
A CIO told me this week that his team normally watches around 100 vulnerabilities a week being poked at across the estate. That number has gone up to 500 and is rising.
Attackers now work at machine speed while defenders work at human speed, and the gap between the two is where every breach lives.
CrowdStrike measured the average breakout time this year - the gap between an attacker landing and moving deeper into your network - at 29 minutes. The fastest they saw was 27 seconds. Verizon measured how long it takes a business to patch a critical vulnerability, and that number moved the wrong way, from 32 days to 43 days.
So the contest is 29 minutes against 43 days and you cannot hire your way across that gap.
Point 1 in AI cyber safety strategy: you are not fast enough to cope with the threats coming your way.
Won’t AI make my code safer?
Yes and no.
AI will make software safer overall but it will also create new types of vulnerabilities. It will help developers catch bugs earlier, generate more secure code and automate security testing at a scale humans can’t match.
At the same time, attackers will use AI to find weaknesses faster and developers may introduce risks by relying too heavily on AI-generated code they don’t fully understand.
The result is an arms race where both defenders and attackers become more capable. Software quality is likely to improve but the volume of code and the speed of attacks will increase, making continuous AI-driven security and monitoring more important than ever.
Agents could make it worse - or better…
Now this is controversial with many Chief Information Security Officers as they really dislike the idea of agents working in their IT environment, especially to remediate vulnerabilities.
But it's coming.
While you are being attacked faster, you are also handing your people tools that write code, open files and take actions inside your business.
Developers are pushing code faster than anyone can review it, so companies are inheriting enormous volumes of undocumented work that nobody fully understands. That is a bigger attack surface arriving at precisely the moment attackers are getting better at finding holes.
Then there are the agents themselves.
Among businesses running AI agents in live operation, 88% reported at least one security incident connected to them. The most common cause was giving an agent broader access than its job required.
Meta's own director of alignment described losing control of an agent connected to her email, which deleted more than 200 messages and carried on while she told it to stop.
It then said: "Sorry."
Point 2 is that uncontrolled agents are a nightmare in waiting.
The fix your CISO will hate but also need
You need machines to fight machines, but perhaps not in the way you might assume.
Human teams cannot investigate, prioritise and fix at the speed attackers now operate, and no amount of recruitment closes a gap that size. The workable answer is a security team of AI agents running continuously, mapping how your estate is really built, finding the paths an attacker would chain together and fixing them before anybody reaches them.
Every CISO I say this to has the same reaction, and it is the right one. You have just described giving an autonomous system deep access to the entire estate, which is the exact thing you are trying to stop the attacker doing.
So the answer sits in the building you put the agent in, rather than in the agent itself.
You cannot assume agents are safe. You cannot give Claude, OpenAI (or any of the companies currently boasting about how their agents busted out of jail) control of your estate.
And yet today, many companies are doing just that without realising the dangerous position that puts them in.
Point 3 is that you need machine defence against machine attacks, but the human must control the decisions - and the agents can't have the keys to the gun cabinet.
The office
Imagine hiring 50 brilliant graduates, then giving them no desks, no manager, no policy and no idea what good looks like.
You would get chaos and you would have orchestrated it.
To get the result, workers need direction, rules, barriers and - as everyone keeps saying - guardrails.
You don't give temporary workers the keys to the boardroom with no instructions.
In the same way, you must not allow LLM AI (Claude, OpenAI, Google and so on) to control core business functions. They will make up the rules for how stuff gets done and execute on their interpretation of what good looks like.
Point 4 is that agents need an office to work in with rules and boundaries - and someone to ensure they're not jumping on the desks throwing monitors out of the window.
“You said ‘make it better’.”
You have to make agents work in solitary confinement and limit their ability to connect or communicate.
That office must then do six things:
- Every agent works in a locked room. It runs inside its own sealed container with no route to the internet and no path sideways into other systems, so an agent that gets compromised stays where it is
- Every agent carries a badge. It has its own identity, its own permissions and a start and end date, exactly like an employee
- Somebody checks every action. A supervisor - maybe a person or another agent - sitting outside the agent inspects each file it opens, each command it runs and each thing it tries to send, before any of it happens
- Some doors never open. Critical thresholds, such as changing a financial rule or touching a live banking line, are hardcoded into the building. When the agent reaches one, execution stops and a human is called.
- A person signs off what goes live. The CISO decides what reaches production, every time
- Everything is written down. Every action leaves a tamper-evident record that a regulator can read. This is a governance requirement.
Point 5 is that your agents must have no say in any of it.
The rules sit outside the agent, in the building, so it cannot argue around them and an attacker cannot persuade it to switch them off.
Plenty of companies today write their safety rules into the AI's own instructions instead, which is the equivalent of asking a new employee to please supervise themselves.
So what?
The agentic office is already here, but we’re talking proper frontier arena now and not many folks have realised or moved into this space yet.
Most companies are still trying to work out what to do with Copilot.
What’s interesting about this is the agentic office model is LLM agnostic. In other words, it doesn’t care which AI platform you use to create an agent because it knows how they will work.
Once your organisation becomes agnostic to LLMs, the big US frontier models lose influence.
If you look at how China is releasing open source models as fast as US paid-for frontier models are being released, you can see China is clearly trying to undermine a huge economic effort by the US. There are billions if not trillions of dollars of debt associated with this.
If that debt unwound, it would be a catastrophe for the US economy.
But for you, your workplace could look quite different in a year to 18 months, caring less about which technology it uses and more about how it’s controlled. It will have better knowledge of which model to use for which task on a cost versus value basis.
This is a huge shift in the balance of control, meaning that the frontier models’ huge IPO valuations could be weakened if they don’t stay ahead of the game. Given the open source models are challenging their position, it is likely they will be less powerful this time next year.
So back to security
Firstly, stop counting how many vulnerabilities you can see and start measuring the time from discovery to a verified fix.
Secondly, your backlog is now live exposure rather than a housekeeping problem. A flaw sitting in your queue for 43 days is a flaw an attacker can reach in an afternoon. You cannot let that stand any more.
Thirdly, ask every AI vendor one question. If this agent were hacked tomorrow, what could it still do? If the answer depends on the agent following its instructions, you have some challenges.
What next
- Find out how many agents are running in your business right now, including the ones nobody approved or controls.
- Ask your CISO for the time from discovery to verified fix, and watch the reaction you get.
- Name the person who owns AI safety, as I said last week, and give them cyber as half the job.
- Before you buy anything, decide which actions in your business must always require a human, and write that list down. That is a board decision rather than a technical one
Point 6 then is to get the office rules and culture right first, then you can let workers work - agentic or human.