AI governance shifts toward provable control

AI Governance: Managing Authority in Autonomous Systems

As artificial intelligence agents evolve from experimental tools to essential components of production systems, the governance frameworks within enterprises face increasingly complex challenges. Organizations must go beyond merely understanding what actions an AI agent performed. They now need to prove the authorization behind those actions, clarify why an agent was permitted to take specific actions, and ensure that this authority remains valid as tasks traverse a network of agents and systems.

This challenge intensifies in regulated industries, where requirements for security, auditability, and data sovereignty often limit the use of external software-as-a-service control planes.

In the latest episode of theCUBE Research’s AppDevANGLE podcast, discussions with Sudeep Goswami, CEO of Traefik Labs Inc., and Andreas Prins, who oversees sovereignty strategy at SUSE Group, highlighted the necessity for AI governance frameworks to adapt as autonomous agents begin to make decisions and delegate responsibilities within increasingly intricate enterprise ecosystems.

Goswami noted, “When you have an agent that is handing a task to another agent, that authority should shrink and not leak out.” This raises critical questions about the retention of authorizations as tasks shift between full autonomy and initial human input.

From Visibility to Provable Authorization

Traditionally, enterprise observability has concentrated on reconstructing events post-incident through logs, traces, and dashboards to identify failures within involved systems. However, the emergence of agentic AI raises new governance concerns.

In an autonomous system, authority may initially be granted by a human but could then be delegated across multiple agents, applications, and systems. Each transition has the potential to alter the scope and validity of what is considered authorized.

This highlights that relying solely on identity and credentials is inadequate. Goswami posed a critical question: “Just because an agent has some credentials, is that agent allowed to make this specific action right now, given the surrounding context?” He emphasized that traditional credentialing does not encompass this complexity.

The challenge parallels physical access control; for instance, having an employee badge may allow entry into a building but does not grant permission to execute financial transactions or access sensitive systems. Consequently, AI governance must evolve to a contextual model, where policies determine an agent’s capabilities based on identity, task, environmental conditions, and the delegation chain surrounding each request. This evolution necessitates a shift from retrospective governance to real-time authorization and verification.

The Impact of Agent Delegation on Accountability

The situation becomes more intricate when agents start to delegate tasks to one another. Traditional enterprise access models have typically been designed around human users and predefined service accounts. The introduction of agentic systems creates a dynamic network of machine-to-machine interactions, where authority can swiftly shift across orchestrators, subagents, APIs, and tools.

Prins likened this transformation to the advancements seen in continuous integration and continuous delivery systems, where organizations transitioned from manual processes to automated pipelines. This shift required codifying approvals and security checks directly into the development lifecycle. Similarly, agentic AI may necessitate a rethinking of how governance and authorization are articulated.

He noted that as enterprise agents multiply, understanding their roles and functions becomes increasingly complicated. In a recent discussion, an executive revealed that an engineering team had created approximately 8,000 agents. “If you’re unaware how many are created, you are also unaware what they’re doing and what their function is,” Prins pointed out.

This proliferation of agents could outstrip traditional governance models, leading enterprises to eventually require robust controls akin to software supply chain management tailored for agents, focusing on explicit identity, delegated authority, policy enforcement, and actionable insights into agent behavior.

Enforcing Policies in Context

Defining governance policies is only part of the solution. Enterprises also need effective mechanisms to enforce these policies where agents interact with applications and infrastructure. This positions the gateway layer as a critical component in modern governance architectures.

Goswami emphasized that AI governance systems should accurately capture both permitted and denied actions. Transparent accounts of both successful task completions and blocked unauthorized activities are essential to demonstrate the efficacy of security measures. “You want to be able to showcase proof that your guardrails are working,” he noted, highlighting the need for context-aware enforcement.

The enforcement mechanism can also feedback into governance policy formation. If agents frequently attempt actions that are denied, it underscores potential design flaws in workflows or inadequacies in the policy framework.

This interconnected governance model challenges enterprises to marry policy definition, enforcement, and evidentiary processes into a cohesive system rather than treating them as isolated functions.

The Limitations of Audit Logs

Audit logs have traditionally been the cornerstone of enterprise compliance, but the rise of autonomous AI poses a new trust dilemma: the systems generating this evidence may also have direct control over it. Goswami likened the scenario to a vehicle’s odometer, where the only party reporting mileage may also have the capability to modify it, thereby nullifying any independent verification of its accuracy.

Such vulnerabilities can extend to audit logs from applications and vendors, which can be manipulated without external oversight. “They can be tampered with, and there’s no third-party way to know when and how it was tampered,” Goswami stated.

To bolster trust models, cryptographic evidence can enhance the traceability of changes. By not solely relying on traditional logging, organizations can cryptographically capture decisions made, authorization transactions, and actions taken by agents.

However, cryptography alone cannot resolve the issue entirely. Goswami argued that a robust verification system is crucial for ascertaining whether evidence has been altered post-creation. “You need the logging capability at a cryptographic level, but then you also need a third-party verification mechanism to be able to check against it,” he said.

This comprehensive approach transforms observability data into forms of verifiable evidence, particularly critical in highly regulated sectors where organizations must demonstrate the integrity of agent actions and records.

Embedding Sovereignty into Governance Architecture

The complexity of governance escalates when enterprises cannot rely on external control frameworks. Research shared during the podcast indicated that 47% of surveyed organizations operate within mixed connected and disconnected environments, while 11% specifically deploy generative AI in on-premises or air-gapped infrastructures.

This landscape places sovereignty beyond geographical compliance considerations. Organizations in sectors like defense, healthcare, and financial services must fully control their models, governance protocols, and verification infrastructures.

Goswami remarked, “The moment you become dependent on a third-party SaaS service that you don’t control, all bets are off.” Prins elaborated on sovereignty as a risk-based decision that encompasses both the AI models and the infrastructure utilized.

Organizations may opt for closed frontier models, open-weight models, or open-source frameworks, each with its unique trade-offs concerning transparency, usability, and control. The choice of deployment environment also matters; companies might access models via SaaS providers or operate them within self-managed infrastructures. According to Prins, the stakes for customer-controlled infrastructure rise as workload sensitivity increases: “The more regulated, the more control you should have.”

Building a Sovereign AI Ecosystem

The conversation underscored the impracticality of achieving sovereignty through a singular platform. A full enterprise AI stack encompasses various components, including models, CPUs, GPUs, Kubernetes infrastructure, gateways, policy engines, observability systems, and evidence layers. Each of these elements plays a role in confirming an organization’s capacity to manage and verify its AI environment.