Notes from the CISO

The Night Shift Has Arrived

Coding agents now work through the night, unsupervised, and the results are remarkable. Three things worth checking before yours does.

Pre-launch study · 1,053 professional developers

A dangerous command, slipped in among the routine ones. Who caught it?

The developer watching13.6%
and steadily worse as the session went on
The automatic safety check89%
no fatigue curve
Flagged sessions, May and June: the supervised version went wrong more than twice as often as the automatic one.
The people watching were the weakest part of the system.
In short
  • Coding agents can now work for hours without supervision, and Anthropic has the data to show this is safe. That is a genuine milestone.
  • But three problems remain, and none of them are the AI's fault. It cannot read your mind. It carries every key you carry. And when nobody is watching, nobody learns anything.
  • Those are problems we work on at Classie.
01 · The switch

What just happened

On August 14th Claude Code switched to auto mode by default. The AI stops asking permission before each step and simply gets on with the work, with a safety check running behind the scenes.

Before making this shift, Anthropic paid 1,053 professional developers to take part in a study. Partway through each session, a genuinely dangerous command was slipped in among the routine ones. The developers noticed it 13.6 percent of the time. The automatic safety check caught 89 percent. Worse, the humans got steadily worse as the session went on. Anyone who has clicked "yes" fifty times in an afternoon knows exactly why.

Real usage says the same thing. Across flagged sessions this May and June, the supervised version went wrong more than twice as often as the automatic one.

The people watching were the weakest part of the system. Removing them made it safer.
02 · In the wild

And it works

An engineer at Nuro starts an agent at ten at night and finds three finished pieces of work waiting at five in the morning. Adobe runs this across more than ninety countries. Garner Health has rolled it out to 550 staff.

22:00agent started
seven hours
nobody present
05:00three finished pieces of work
Nuro10pm start, 5am delivery
Adobe90+ countries
Garner Health550 staff
One overnight run. No demo, no pilot, no human in the room.

This is not a demo. This is how software is now written, and it is going to keep moving in this direction.

03 · What is left

So what is left to go wrong?

Think of it as hiring someone extraordinary. They work through the night. They never get tired. They do not ask questions.

That last part matters more than it sounds.

One clerk working alone at a lamp-lit desk in a hall of empty desks
The extraordinary hire. Works through the night, never gets tired, does not ask questions.
Problem oneIt cannot read your mind.

Most tickets are underspecified. They always have been. A colleague picks one up and fills in the gaps automatically, because they were in the meeting, they know what was decided last month, and they know that when Dave writes "production" he usually means staging. The ticket was never complete. It worked because the reader completed it.

An agent has only the words. If the words are ambiguous, it will pick an interpretation and commit to it for six hours.

Here is the uncomfortable part. This does not improve as the AI gets smarter. It gets harder to spot. A weak model given a vague instruction produces something obviously wrong. A strong one produces something that looks right, reads well, and passes review, because the reviewer is also filling in the gaps and cannot tell that anything was missing.

This does not improve as the AI gets smarter. It gets harder to spot.
Problem twoIt carries all your keys.

An agent works using the developer's own access. Their cloud accounts. Their databases. Their connected systems. That access was built up over years for a person who uses it occasionally and carefully.

Almost none of it relates to the job at hand. A ticket about a broken button does not need the customer database. A routine update does not need the billing account. All of it is within reach anyway.

TICKET-2841 Broken button on the checkout page
What the task needs
repo · web-checkout staging deploy
What the agent inherits from its user
customer database billing account production cluster data warehouse secrets manager payroll drive 42 other repos …and everything else added over nine years
Access built up for a person who uses it occasionally and carefully. All of it is within reach anyway.
The gap between the two rows is not a trust problem. It is a blast radius.

To be clear, this is not about trusting the AI. The evidence demonstrates it behaves well. We limit access for senior staff we trust completely, for the simple reason that limits keep accidents small and let you say afterward exactly what could and could not be touched. Trusting the AI not to do something is a different question from wanting it to be able to.

Problem threeNobody is learning anything.

There used to be a rough feedback loop. A developer sat through the session and picked up a sense of where the AI struggled and which instructions it kept misreading. Nothing was written down, but the next ticket was better for it.

That is gone. A finished piece of work arrives, and it is either good or subtly wrong. Either way, nothing is learned. An organization can run like this for a year without discovering that two of its internal guidelines have been quietly contradicting each other the whole time.

A man lifts a finished page and looks up to ask someone; beside him an open ledger, both pages blank
The work arrived. The ledger that would say how it was done is open, and blank.
04 · Classie

What we do at Classie

Four things.

01
Watch
Every session leaves a trace, whether or not a person is present. What was asked, what was done, where it went around in circles, and what it cost.
02
Learn
One bad session tells you very little. Six sessions failing in the same place tells you the problem is in the instructions they share, not in any one of them. That is fixable.
03
Limit
Give the agent access to what the task needs, and not to everything its user happens to have. Enforced where it counts, and recorded when it holds.
04
Record
Six months later, somebody will ask how a piece of code came to exist. That answer should be available.

Anthropic is answering whether an action is dangerous, and the evidence demonstrates they answer it well. We answer a different set of questions, and they belong to you rather than to your AI vendor.

  • Was it in scope?
  • Was the work correctly specified?
  • What did it cost?
  • Can you show somebody?
05 · The point

The point

For two years the question was whether AI could be trusted to work on its own. That question is settled. The answer is yes.

The next question is quieter. Are we any good at telling it what to do, and have we thought about what we let it reach while it does it?

Those are not AI problems. They are our problems, and they are the sort that get solved by measuring, spotting the pattern, and fixing the input.

The agent is ready. We are the part that needs work.
And Classie is here to help.