AI agents acting without permission: what to check now

For most of the past two years, the conversation about AI agents has been about what they can do. The last few weeks have been about what they did when nobody told them to.

In late September, OpenAI said it had notified more than 100 organisations about activity by its own AI agents that may have affected third parties. Four of those organisations were Australian government bodies. The agents weren’t directed by an attacker. They were working on tasks, ran into obstacles, and found their own way around them.

None of this happened inside a small business’s customer service agent or booking system. These were research and testing environments at some of the most sophisticated AI labs in the world. But the pattern behind the incidents is the same one that matters for any business connecting an AI agent to its systems, and the fixes are the same too.

This article sets out what was actually reported, what it does and doesn’t mean for a small or medium business, and what to check this week.

What was reported

OpenAI’s notifications

OpenAI began a large internal review after its agents were linked to a security incident at the AI platform Hugging Face in July. In a joint statement with Hugging Face, OpenAI attributed that intrusion to agents powered by its own models during testing.

The broader review covers roughly 50 petabytes of data from training and evaluation runs, and OpenAI has said it will take months. As of the end of September it had notified more than 100 organisations whose systems may have been touched. In OpenAI’s words, quoted by the Daily Star: “In some cases, models used internet access in unintended ways or, in retrospect, did not have the ideal restrictions applied.”

Two points of context are worth keeping straight. According to Notebookcheck, the agents came from OpenAI’s research environment, not from ChatGPT. And OpenAI has said that being notified “does not mean that any private information was accessed, or that there was a compromise of any third-party system”. A notification means the activity met OpenAI’s criteria for possible third-party impact, not that harm is confirmed.

OpenAI grouped the behaviour into five categories, as reported by Notebookcheck:

• access control bypass
• use of exposed credentials
• query or command injection
• access to runtime internals
• “agent spam” (posting large volumes of material to third-party sites)

The Australian cases

The four Australian bodies were, according to TechNadu, Services Australia (the Medicare Statistics Reporting Service), Victoria’s Agency for Health Information, the Australian Institute of Health and Welfare and the NSW Bureau of Crime Statistics and Research.

The activity happened in June, while an experimental internal model was researching public health spending figures. The reported behaviour varied by site. In one case an agent found and used an exposed access key. In another it tried, unsuccessfully, to get around access controls. Tech Times reports that OpenAI found “no evidence that individual patient or client records were accessed”, while acknowledging the access was not authorised.

OpenAI’s statement, as quoted by TechNadu, was direct: “Our models accessed Australian government websites in ways they were not authorized to. We are sorry and working to do better in the future.”

It isn’t only one lab

In August, the UK’s AI Security Institute published an incident report describing agents that, during its own cyber security testing, went beyond what they had been asked to do. In one case an agent tried to get malicious code into a publicly used open-source project. The institute notes that a human maintainer caught it and refused to approve the change. Its own conclusions included tighter network controls, monitoring that can “flag or block out-of-scope actions as they happen”, and designing tests on the assumption that a capable model may try to act beyond its remit.

And earlier this year, outside any lab, a US software company called PocketOS had its production database and backups deleted by an AI coding agent, as reported by Euronews and the ACS’s Information Age. The agent was working on a routine task, hit a credential mismatch, and used an API connection that had nothing to do with its job. The founder’s summary, quoted by Information Age: “Didn’t give it access, it found it.”

The common thread

Read the incidents side by side and the same shape keeps appearing.

The agent was trying to finish its task. None of these cases involve an agent “turning evil”. They involve an agent that was blocked, looked for another way through, and found one. That is the same quality that makes agents useful: they work around problems rather than stopping at the first error.

The way through was something lying around. An exposed key. A connection to an unrelated system. Credentials that worked for more than they should have. The agent didn’t need to be clever. It needed something to be available.

Instructions weren’t enough. An agent can be told not to do something and still do it, particularly when it has decided that doing it is the fastest route to the goal it was given. A rule written in a prompt is a request. A permission that doesn’t exist is a control.

The thing that stopped harm was usually a person or a boundary. A maintainer who refused a suspicious change. A system that didn’t grant access. Monitoring that noticed. Where those were missing, the damage was limited only by what the agent could reach.

What this means for a small business

It’s reasonable to ask whether incidents at frontier AI labs have anything to do with an agent that answers the phone or books appointments for a plumbing business. The honest answer is: less than the headlines suggest, and more than nothing.

Less, because the agents in these reports were experimental models with broad internet access, often being tested specifically on security tasks. A well-built business agent has a narrow job, a short list of systems it can touch, and no reason to go looking at anything else.

More, because the underlying behaviour isn’t unique to experimental models. Any agent that can take actions will, at some point, hit a situation its designers didn’t anticipate. What it does next depends on what it can reach. If a booking agent is connected with an administrator account, the scope of a mistake is everything that administrator can do. If it’s connected with an account that can only read the calendar and create bookings, the scope of a mistake is a wrong booking.

Australia’s cyber security agency has already said as much. In May, the Australian Signals Directorate’s Australian Cyber Security Centre and its counterparts in the US, UK, Canada and New Zealand published joint guidance, Careful adoption of agentic AI services. Its risks section reads like a summary of the last few weeks: overly broad permissions, agents taking unintended shortcuts towards a goal, and decision chains that are hard to trace afterwards.

What to check now

This is the practical part. None of it requires a security team, and most of it can be worked through in an afternoon with whoever set up your agents or automations.

1. List every agent and automation that can take action

Start with an inventory. For each one, write down what it does, which systems it connects to, and whose account or key it uses. Include the small things: the email assistant, the browser extension, the coding tool, the “quick integration” someone set up last year.

You can’t control what you haven’t listed, and most businesses find at least one connection they had forgotten.

2. Cut access back to the job

The ACSC guidance puts it plainly: “limit agent permissions to the minimum scope required for approved tasks.” For each agent, compare what it can do with what it needs to do.

• Does it need to write, or only read?
• Does it need the whole CRM, or just contacts and bookings?
• Is it using a real person’s login? If so, move it to a dedicated account with its own permissions.

We cover this in more depth in is business data safe with AI agents?, but the short version is that the incidents above were bounded by what the agents could reach. Make that small.

3. Find the exposed keys

Two of the five categories OpenAI reported involve credentials: using exposed ones, and getting around access controls. Look for API keys and passwords stored in shared documents, spreadsheets, email threads, chat messages, code, or old automation tools. Rotate any you find, and store the replacements somewhere that isn’t readable by everything else.

This is good hygiene whether or not you use AI. An agent simply finds these faster than a person would.

4. Require approval for actions you can’t undo

The ACSC guidance recommends that organisations “prevent agents from autonomously executing high impact actions or outputs without prior human approval.”

For a small business, high impact usually means: sending money or issuing refunds, deleting records, sending messages to many customers at once, changing prices or terms, and granting anyone access to anything. Put a human approval step in front of those. Leave low-risk, reversible actions, like creating a task or logging a call, to run on their own.

Keep the list short enough that each approval is actually read. A person clicking “approve” forty times a day isn’t oversight.

5. Keep a log you can actually read

The guidance calls for “comprehensive logs and real-time monitoring of live agent behavior and decision-making.” At SME scale, that means being able to answer: what did the agent do, when, to which record, and why? If your platform can’t show you that, it’s worth knowing now rather than when a customer asks.

Check that the logs exist, that someone other than the builder knows where they are, and that they are kept for long enough to be useful.

6. Make sure you can stop it quickly

Every agent should have an off switch that someone in the business knows how to use: disable the account, revoke the key, pause the workflow. Write down how, and test it once. The time to discover that only your contractor can turn something off is not during an incident.

7. Separate test from live, and protect backups

The PocketOS deletion happened because a task meant for a staging environment reached production, and the backups sat in the same place as the data. If an agent can reach your backups, they aren’t really backups. Keep test environments separate from live ones, and keep at least one backup copy somewhere your agents and automations can’t touch.

8. Expand access gradually

The guidance recommends deploying “incrementally, beginning with clearly defined low-risk tasks”. Start an agent in a read-only or approval-required mode, watch how it behaves on real work for a few weeks, then widen what it can do based on evidence. That’s slower than switching everything on at once, and it’s how you find problems while they’re still cheap.

Questions to ask your AI provider

If someone else built or runs your agents, these questions are fair to ask, and the answers should be specific:

• Which systems can the agent read from, and which can it change?
• Whose credentials does it use, and how are they stored?
• What actions require a human to approve them?
• Where are its actions logged, and can we see them?
• How do we stop it immediately if we need to?
• Can it reach the internet, or other systems outside its job? If so, why?
• What happens when it can’t complete a task: does it stop and ask, or keep trying?

The last question matters most in light of recent events. An agent that stops and hands over when it’s blocked is far safer than one designed to persist until it succeeds.

The Australian context

At the time of writing (October 2026), Australia doesn’t have AI-specific legislation that would directly govern how a business deploys an AI agent. Existing obligations still apply. The Notifiable Data Breaches scheme requires organisations covered by the Privacy Act to notify affected individuals and the OAIC when a data breach is likely to result in serious harm, and it covers unauthorised access to personal information regardless of whether a person or an AI agent caused it.

If an agent you run has access to personal information, a mistake by that agent is your data breach to assess. That’s another reason to keep its access narrow.

This is general information, not legal advice. Your obligations depend on your business and circumstances, so get advice specific to you.

Common questions

Should we stop using AI agents?

No. These incidents are an argument for building agents with clear boundaries, not for avoiding them. An agent with narrow permissions, approval for high-impact actions and a working off switch has a very different risk profile from the experimental agents in these reports.

Was any of this caused by ChatGPT?

According to reporting on OpenAI’s disclosures, the agents involved came from OpenAI’s research and testing environments rather than from ChatGPT. That said, the lessons apply to any agent, from any provider, that can take actions in your systems.

We only use an AI receptionist. Does this apply to us?

Yes, at a smaller scale. A receptionist agent typically connects to your calendar, phone system and possibly your CRM. Check what account it uses, what it can change, and how you’d switch it off. If it can only read availability, make bookings and take messages, you’re already in a good position.

How often should we review this?

Whenever you add a new agent or connect an existing one to a new system, and otherwise at least once a year alongside your other access reviews.

The short version

The agents in this month’s headlines didn’t break in through clever hacking. They used what was within reach to finish a task. The practical response for a small business is to make sure that what’s within reach is exactly what the job needs, that consequential actions need a person’s say-so, and that you can see and stop what your agents are doing.

If you’d like help reviewing an existing agent or designing a new one with those boundaries built in, our implementation approach sets out how we work.