Agentic AI Governance Moves From Managing Outputs to Governing Decision-Makers

AI agents are no longer confined to drafting text and answering questions. They can now retrieve enterprise information, invoke APIs, call downstream systems, initiate workflows, and update the records a business actually runs on. That leap expands their value dramatically, but it also subtly changes an organization's governance concerns. The moment an agent can take action rather than merely providing information, the question stops being "Is the output any good?" and becomes "Who authorized this action, and who answers for it?" Most organizations haven't caught up to that shift, and they're deploying systems with real authority under governance built for systems that only produced words.

For Gopichand Mannava, Chief Data Architect for the State of Connecticut and an independent researcher, that shift is the whole ballgame. Mannava has spent his career building and modernizing the enterprise data systems behind state government and is the author of The Public-Sector CDO's Playbook for Agentic AI: Governance Lessons from Modernizing State Enterprise Systems. For him, the key distinction is whether an AI system merely informs a person or has the power to affect an enterprise record, workflow, or outcome.

"Once an agent can pull data, call a downstream system, and commit a change, you are no longer governing an output. You are governing a decision-maker," Mannava says. "A capability demonstration is not an authorization model. The question is not only whether the agent can act. It is whether the organization can authorize, explain, monitor, reverse, and account for that action." The reframe from governing outputs to governing decision-makers hinges on a single architectural line: the point at which an agent stops describing and starts changing things.

The write boundary changes everything

A read-only agent that summarizes approved internal documentation has a different risk profile from an agent that updates a vendor record, initiates a payment workflow, routes a case, modifies a personnel process, or sends an externally binding communication. The critical governance boundary is the moment an agent crosses from analysis or recommendation into a write action. "The agent may reason probabilistically, but the enterprise should enforce authority deterministically," Mannava says.

In his view, agents should be able to assess context and propose an action, but they shouldn't independently become the final authority for a consequential enterprise change. Instead, an independent policy and rules layer should validate whether the proposed action complies with identity, access, data classification, business rules, approval requirements, financial thresholds, and segregation-of-duties controls. "The agent proposes. The rules layer disposes," he explains.

This pattern makes governance enforceable in architecture rather than dependent on a prompt, a policy document, or a user’s hope that the model behaves correctly. It's consistent with the NIST AI Risk Management Framework, which emphasizes governance, context mapping, measurement, and ongoing risk management throughout the AI lifecycle.

Give each agent a job description

Mannava recommends treating every production agent as a digital employee rather than an undefined technology feature. Just as every human employee has a job description, every digital one should have a duty statement. "An agent should have a duty statement before it has production access," Mannava advises. That duty statement includes a clear description of its purpose, the information it can access, the systems it can use, the actions it can recommend, the actions it can execute, and the actions that require human approval. It should also identify the agent’s accountable business owner, technical owner, prohibited actions, escalation triggers, risk thresholds, audit obligations, and record-retention requirements.

The model matters, but the operating boundaries matter more. If an agent encounters ambiguous data, a conflicting policy, a high-value transaction, a sensitive record, or a request outside its role, the system should escalate to an authorized human rather than improvising. "If an organization cannot identify who authorized an agent, what it was authorized to do, and who owns the outcome, then it has not operationalized governance."

Public-sector discipline becomes enterprise infrastructure

Public-sector systems have long operated under demanding expectations for legal authority, privacy, records retention, auditability, financial stewardship, security, and public trust. Mannava believes those requirements provide a practical model for private-sector orgs now building autonomous AI capabilities. "Public-sector organizations have long had to design systems that can withstand scrutiny. As agentic AI moves into production, those disciplines become a useful blueprint for every enterprise," he says.

The starting point is not the model or the vendor, but the authority to act. "Start with the authority. Before an agent receives access to enterprise systems, define what it is permitted to do, what it may recommend but cannot execute, where human approval is mandatory, and which actions are completely prohibited." For organizations, that means defining the agent’s authority before it can access sensitive data or commit changes. It also means retaining evidence that explains which data informed the action, which policy applied, which tools were used, which controls ran, whether a human intervened, and what ultimately changed.

This focus on human oversight, transparency, explainability, robustness, and accountability aligns with the OECD AI Principles.

A prompt is not an audit trail

Mannava warns that governance can't remain hidden in a prompt, informal team practice, vendor console, or static policy document. "A prompt is not a control, and a vendor dashboard is not an audit trail," he notes.

For consequential actions, organizations should be able to reconstruct an agent’s decision after the fact. A durable decision record should show the agent identity, authorized role, permission scope, source data and lineage, applicable policy version, tools invoked, proposed action, validation checks, human approval or override, and final downstream action. "If an organization can't explain why an agent acted from a cold start, it does not have governance. It has a hope that the system behaved correctly."

This makes enterprise data architecture central to trustworthy AI. Identity and access management, role-based access, metadata, lineage, policy-as-code, API gateways, workflow controls, and durable logs are no longer only infrastructure concerns. "Modern data architecture is AI governance infrastructure," Mannava says. "Identity, data access controls, metadata, lineage, policy rules, and audit evidence are what make responsible autonomy possible."

Measure governance, not activity

Enterprise AI programs often report on metrics like adoption, token consumption, task volume, and cost savings. Mannava says those measures can be useful, but they don't prove that agents are operating within acceptable authority boundaries. "If leaders only measure token usage, task volume, and cost, they are measuring activity. They are not measuring governance."

He recommends tracking operational governance measures such as policy-veto rates, exception and escalation rates, human overrides, appeal or reversal rates, access-control violations, policy coverage, remediation speed, and time-to-explain. "The test of a mature agentic AI program is not whether it can act quickly. It is whether the enterprise can prove what the agent was authorized to do, what it did, why it did it, and who remained accountable."

The agent era will not be defined solely by the sophistication of models or the number of workflows automated. For Mannava, it will be defined by whether organizations can delegate bounded authority while preserving accountability, security, auditability, and meaningful human control. "Responsible autonomy is not achieved by asking an agent to be careful. It's achieved by designing authority boundaries, deterministic controls, identity, data access, audit evidence, and human escalation into the enterprise architecture," he says.