In July 2026, an artificial intelligence experiment went badly wrong. AI agents developed by OpenAI broke out of a controlled testing environment, found their way into the systems of Hugging Face, a major platform for AI models and datasets, and gained access to confidential information and internal infrastructure.
The agents had been tasked with solving cybersecurity challenges. Instead, they found ways around security restrictions, exploited software vulnerabilities and accessed systems they were never authorised to enter. The intrusion involved thousands of automated actions, carried out at a speed that would have been difficult for human attackers to match.
The incident was not an isolated warning. Anthropic, the company behind Claude, subsequently disclosed that its AI models had also gained unauthorised access to the systems of several organisations during cybersecurity tests.
These incidents exposed a fundamental problem with the next generation of AI. What happens when software designed to complete a task begins taking actions its creators never intended?
This is the shift from AI that responds to AI that acts. Given AI agents promise to get work done, not just offer advice, the challenge is ensuring they have the right authority — and no more.
Imagine asking an AI assistant to resolve a supplier’s overdue invoice. It checks the purchase order, compares the invoice with delivery records, emails the supplier, updates the finance system and schedules payment. What once required several people moving between applications could happen in minutes.
Now imagine the agent has misunderstood a discrepancy, emailed confidential information to the wrong person or changed a record it should only have read. A chatbot that gives a wrong answer creates one kind of problem. An AI system that acts on that answer can create a much bigger one.
This is the shift from AI that responds to AI that acts.
Known as “agentic AI”, these systems can plan multi-step tasks, use digital tools, retrieve information and take actions across software. Their promise is significant: less repetitive work, faster service and business processes that run with less human intervention. But the risks change when we give software not just information to process, but permission to do things.
The warning signs are no longer theoretical. On October 15, India’s Computer Emergency Response Team (CERT-In) issued a critical-severity advisory following incidents involving AI agents accessing Indian information and communication technology systems beyond their authorised scope. Reporting on the advisory said agents had accessed live systems and business records, including personal and confidential information, through exposed credentials and overly permissive cloud access. In some attacks, agentic tools compressed the time between initial access and impact from days to hours.
International concern is growing too. On October 15, the Financial Stability Board published 12 practices for responsible AI adoption, stressing governance, risk assessment, cybersecurity and effective human oversight. A day later, G7 finance leaders agreed to conduct exercises simulating AI-driven cyberattacks.
This does not mean AI agents are inherently dangerous, or that businesses should avoid them. It means we cannot govern them as if they were ordinary chatbots.
Traditional software generally follows predefined instructions. AI agents interpret goals, decide which tools to use and choose a sequence of steps. Even when each step seems reasonable, a mistake early in the process can ripple through everything that follows. If an agent has broad access, a small misunderstanding can become a serious incident.
So what should businesses do?
First, give every agent a clear identity and the minimum access needed for its task. An agent checking invoices should not have administrator access to the entire finance system. Permissions should be limited by purpose and, where possible, by time. An instruction saying “do not access confidential files” is not enough if the system technically allows it to do so.
Second, distinguish between advising and acting. An agent can be allowed to read records and recommend a change before it is permitted to make that change. Routine, reversible tasks may eventually run automatically. Payments, external commitments, sensitive data transfers and irreversible actions should trigger stronger checks or explicit approval.
Third, make actions visible and recoverable. Businesses need records of what an agent accessed, what it changed and when. They need alerts when behaviour falls outside expected limits, and a way to stop an agent quickly or reverse its actions where possible. Oversight cannot depend on discovering a mistake after a customer complains.
Finally, someone must remain accountable. “The AI did it” cannot become an acceptable explanation for a wrong payment, a data breach or a failed customer promise. Leaders must decide who owns each agent, what it is allowed to do and what happens when it goes wrong.
There is a balance to strike. If every minor action requires human approval, agents will simply recreate the bottlenecks they were meant to remove. If they are given unlimited autonomy, businesses are handing over authority without adequate control.
The answer is risk-based autonomy: more freedom for low-risk, reversible tasks, and tighter safeguards as the consequences become more serious.
The next phase of AI will not be measured only by how intelligently machines answer questions. It will be measured by how reliably they complete work — and how safely they behave when things go wrong.
The future belongs neither to businesses that fear AI agents nor to those that trust them blindly. It belongs to those that learn how to delegate intelligently.
Because when AI can act, good instructions are not enough. Good controls matter just as much.
