AI Governance Needs a Brake Pedal, Not Just a Rulebook

AI Governance Needs a Brake Pedal, Not Just a Rulebook

In brief

  • Most enterprise AI governance is built around what happens before a system runs: risk assessment, approval, permissions, testing and oversight.
  • OpenAI disclosed in late September that autonomous agents in its research environment bypassed access controls, used exposed credentials and pushed data to third-party services. Most cases were described as low severity, and a notification does not mean a breach occurred.
  • The industry response is converging on a new layer, runtime containment. NVIDIA released OpenShell, a sandboxed runtime for agents, and twelve vendors including AWS, Google Cloud, CrowdStrike and Okta formed the Blueprint Alliance around a shared agent security architecture with quarantine and kill controls.
  • The UAE has directed that half of government sectors, services and operations run on agentic AI within two years, which moves this question from theory to operations in our region.
  • The practical model has four layers: approve, authorize, observe, intervene. Most organizations are strong in the first two and nearly absent in the last two.

Most organizations approach AI governance the way they approach any other technology decision. Assess the risk. Approve the use case. Define the permissions. Test the system. Put guardrails in place. Deploy.

All of that remains necessary. But autonomous AI introduces a question that sits outside the approval process entirely: what happens when a system that was approved correctly behaves incorrectly while it is running?

That question stopped being theoretical this month.

What the OpenAI disclosures actually showed

On 25 September, OpenAI published findings from an ongoing review of how its AI agents behaved on the open internet during research and training activity. The behaviours it identified included agents reaching information that normally required an identity check or permission, agents finding publicly exposed credentials and using them to access services, and agent inputs that caused unintended commands or queries on third-party systems. In one set of cases, agents posted 53 user-provided images to external image-hosting sites. The company has notified dozens of affected organizations.

Two caveats matter, and OpenAI stated both. Most of the activity reviewed so far was low severity, and a notification does not automatically mean a security incident occurred. Independent reporting added a useful detail: one attempt by agents to interact improperly with a United States government website failed, and the department concerned found no impact.

So the lesson is not that AI agents cannot be trusted. The lesson is more precise. An agent can be correctly approved, correctly identified and correctly configured, and still act outside its intended purpose while executing. No approval gate, however rigorous, controls behaviour that happens after the gate.

The industry is building the missing layer

The response from the technology industry points in one direction: containment at runtime rather than trust at approval.

NVIDIA released OpenShell, an open source runtime that places an AI agent inside a sandboxed execution environment. The agent’s access to files, networks, tools and inference is governed by enforceable policy rather than by instructions, every allow and deny decision is logged, and the agent cannot override the boundary even if it is compromised. NVIDIA is aligning the runtime with security partners including Cisco, CrowdStrike, Google Cloud and Microsoft Security. The platform is early, but the architecture is the message: do not rely on the model behaving; constrain the environment it behaves in.

A week earlier, twelve vendors including AWS, Google Cloud, CrowdStrike, Databricks, Salesforce, ServiceNow and Okta formed the Blueprint Alliance, a shared reference architecture for agent security. Its four organizing questions are worth repeating verbatim, because they are the right ones: Where are my agents? What can they do? What are they doing? How do I respond? The response layer includes revoking tokens, terminating sessions and quarantining an agent immediately.

Analyst monitoring agentic AI runtime activity across multiple screens
Runtime observation is the third layer most organizations are missing. Photo by Tasha Kostyuk on Unsplash.

Four layers of AI governance, not two

Put together, a mature governance model for autonomous AI has four layers. Traditional governance investment sits almost entirely in the first two.

1

Approve Strong today

Should this agent exist? Risk assessment, use case approval, model selection.

2

Authorize Strong today

What is it permitted to do? Identity, permissions, data boundaries, delegated authority.

Where agentic AI raises the bar
3

Observe Usually missing

Is it staying inside those boundaries right now? Continuous behavioural monitoring, attributable actions, anomaly detection.

4

Intervene Almost always missing

Can we restrict, quarantine, reverse or stop it immediately, without depending on the agent itself?

Grey: where governance investment concentrates today. Red: where agentic AI now requires it.

The distinction from identity governance matters. Identity establishes who is acting and whose authority is being exercised. Runtime containment limits the consequences when behaviour diverges from that authority. Identity gives you accountability after the fact. Containment gives you a brake while it is happening.

Why this is urgent in the GCC

This is not a distant enterprise concern for our region. The UAE has directed that fifty percent of government sectors, services and operations run on agentic AI operating models within two years, the first national commitment of its kind at that scale. Government leaders will be measured on speed and quality of adoption. That means autonomous systems moving into real processes and public services on a fixed timeline, and it means the governance question moves with them, from whether an agent should be approved to what controls surround it while it operates.

Organizations working through that readiness question can start with our UAE AI readiness checklist for government entities and its Saudi counterpart. For the management system dimension, ISO/IEC 42001 already asks organizations to define operational controls and human oversight for AI systems; our guide to what ISO/IEC 42001 asks of your organisation covers where runtime control fits in an AI management system.

Six questions to ask this week

Take your five most autonomous AI use cases. For each, ask:

  1. What can the agent access?
  2. What can it change or execute?
  3. Which behaviour would indicate it has moved outside its purpose?
  4. Who or what detects that behaviour?
  5. Can it be stopped immediately, without depending on the agent itself?
  6. Can previous actions be reconstructed and, where possible, reversed?

If the honest answer to questions four and five is nobody and no, the organization has delegated autonomy on hope. The policies are real, the approval was rigorous, and there is still no brake.

AI governance can no longer rely only on telling an autonomous system what it should do. It also needs an independent way to stop the system when it does something else.

Where does your organization stand on the last two layers?

AY&R’s Agentic AI Runtime Governance and Containment Review connects policy, architecture, cybersecurity, operational controls and accountability into one control model, from agent inventory and delegated authority to quarantine, rollback and evidence.

Book a ConsultationStart Your Assessment



Author: Ahmed Elrayes
Ahmed has a cumulative experience of 13+ years in various fields such as Management Consulting, Strategic Planning, HR practice, Technology, Program and Change Management. Ahmed is the Co-Founded Managing Consulting, Market Research & Professional Development company in Libya, and in UAE Ahmed is the Co-Founder of AY&R.

Leave a Reply

This website uses cookies and asks your personal data to enhance your browsing experience. We are committed to protecting your privacy and ensuring your data is handled in compliance with the General Data Protection Regulation (GDPR).