AI Agent Governance Runs Before the Tool Call

AI agent governance after the tool call is audit paperwork, not control.

Even small actions have side effects: sending an email, updating a customer's details, approving a refund, updating a Salesforce opportunity, or even simply spending money from a wallet. Each of these actions has a useful side effect and a less useful side effect. If the runtime does not have enough context to prevent the less useful side effect from happening then there is value in auditing what happened.

Gartner predicted in May that uniform governance applied to AI agents with different autonomy levels and access rights would cause either over restriction or under restriction. In its forecast for 2027, Gartner is predicting that by 2027, 40% of enterprises will either demote or decommission autonomous AI agents after discovering governance related gaps while running them in production.

The fact that a read-only summarizer and an agent that modifies account data both go through the same review process for governance is already bad enough. But that a drafting assistant agent and a production operations agent would both be subject to the same approval rules is simply inane. Likewise, an agent that recommends a refund and an agent that actually issues the refund should be in different runtime envelopes.

The action boundary is the governance boundary

Enforcing agentic AI governance at the action boundary is cleanest when the model is proposing an action. The runtime can then decide what to do with that proposed action. The runtime knows about the user, the thread, the run, the attempt, the name of the tool that was proposed, the arguments for that tool, the prior state of the runtime (i.e. the runtime's store), the approval state of the runtime (i.e. what agents have been approved for what), and the credentials that are available to the runtime. So until the model has had a chance to run and cause a side effect, the action boundary is a clean place for governance.

On agentic AI governance, Focused has already written about AI agent governance in the execution path. The bigger problem is the form that this control takes and in particular how identity, policy, evals, traces, approvals, and recovery can be compiled into a single decision at the point of execution.

A useful contract then can answer the simplest of questions about the execution of actions by an agent, as follows: 1) May this agent call this tool for this user? 2) Was this action part of a workflow that granted a permission envelope for this agent to act? 3) Does this action comply with the account/region level constraints? 4) Is approval required for this action? 5) What is the appropriate recovery for this action that violates a soft constraint?

Without a runtime decision however the governance of a system is just a Slack archeological dig: people search for a prompt, the tool schema for that prompt, a human approval, the corresponding trace, the Jira ticket, the diff of the last deployment and the change of a metric which caused all this trouble in the first place. Everybody pretends the system had a control plane because all the artifacts were somewhere.

Existence is not enforcement.

Behavioral contracts give governance a runtime shape

There is increasing academic research aimed at governing agentic AI, much of it focusing on contracts for such agents. Recently a paper has been published on Agent Behavioral Contracts, a contractual representation of an agent's behavior, decomposed into Preconditions, hard Invariants, soft Invariants, hard/soft Governance Constraints and Recovery mechanisms, with results from 1,980 sessions (average of 88-100% hard constraint compliance, with "drift" bounded by 0.27 below which cases are considered to be within tolerance, all with average per-action processing time of under 10 ms). The contracts surfaced 5.2 to 6.8 soft violations per session that would have otherwise gone undetected by uncontracted baselines.

A behavioral contract translates trace into evidence. The trace of a contracted agent can become a list of records that links to change records and permission envelopes, the machinery already used by the organization in production rollout. A read-only summarizer is a different thing from a production accountable autonomous AI agent. A drafting assistant is different from a production operations agent. A refund recommender and a refund issuer each fall within a different runtime envelope.

Runtime context is the missing input

Runtime governance is closest to the tool layer. The documentation for the LangChain runtime describes the create_agent method for LangGraph's runtime, which has context, store, stream writer, execution info and server info available to tools and to middleware. The docs expose user context (e.g. authenticated user id, user email), store access, thread id, run id, attempt number, server info (e.g. server id, version), and related runtime details through the ToolRuntime surface. That is exactly the data a governance decision needs.

Guards can be enabled at the agent level to restrict the start and end of execution, as well as calls to models and tools within a runtime. There are pre-built and pluggable middleware including PII removal as well as human-in-the-loop approval for sensitive commands that require human intervention to approve before execution. The docs put enforcement around execution.

And by the way, Open Policy Agent already decouples policy decision-making from enforcement and has a big user base around Kubernetes admission control, API gateways, service authorization, CI, and infrastructure policy.

Agent governance needs to go down a similar route with agent-specific inputs: the policy decision function needs to know the proposed action for the tool that the agent is about to invoke for that user. It needs to know the workflow info to determine the permission envelope for that invocation. It needs to know account and region constraints for that action, and whether that action requires approval. Finally, the decision function should return more than simple allow/deny: it should return a receipt for the action that was allowed (e.g. all change history for that record), it should block the action (e.g. reject the payment), it should attempt to recover from the action (e.g. retry the remediation after a delay), or it should escalate the action to a human and resume that same thread after approval by that human.

Proportional governance beats one big gate

Uniform governance looks clean in a committee deck. All agents are reviewed the same way. They go through the same checklist. They inherit the same constraints. That is how low-risk AI agents become stupid and high-risk AI agents become dangerous: the first get buried in process, while the second get controls written for a chatbot.

A proportional model for governance starts with the agent's level of autonomy, and its potential blast radius (i.e. the amount of damage it can cause if left unwatched or unleashed from its usual boundaries). Observers need agents with scoped read access, possibly with credentials or scope set programmatically and logged. Agents that can be tested. Advisors are dangerous because their outputs are taken at face value by other agents and by humans, so they must be checked for quality, with various test cases. Agents that Approve actions for other agents have the greatest potential for good and evil: they can be programmed to Approve in the absence of a human, but their decisions must then be subject to human review and approval in turn, with audit trails of their decisions and incident reports for when things go wrong. Finally, there are fully Autonomous agents, whose behavior must be monitored in real time with hard invariants, with provisions for recovery from failure, circuit breakers, owner-based escalation of problems to other agents or teams, and continuous monitoring of their performance.

There are also references here to the important concept of workload identity. The contract cannot reason about authority if the tool call arrives as a generic API key sitting in a prompt or environment variable. It needs workload identity, user or actor identity, a scoped credential, and a record of why that credential was usable for this action. Without those pieces, people are back to hoping a black box behaves.

Skills, MCP tools, and imported agent capabilities are executable artifacts too. Executable agent artifacts should carry provenance, version, permission scope, eval evidence, and rollback behavior. The runtime should be able to inspect those artifacts before letting them act.

The receipt is the product of governance

The receipt, as it pertains to governance, is close to side-effect receipts produced before the primary side effect of a command's execution. The timing matters. The governance receipt records the policy decision before the mutation runs.

As noted above, a receipt of a governance decision should contain the necessary information to allow someone to reproduce and review the trace that resulted from the acceptance of the proposed action and to know where to look for the changed trace (and thus, the trace resulting from the deployment as well as the change record and incident channel for the deployment as well). It should include: 1) a description of the proposed action, 2) the contract version(s) used to evaluate the action, 3) the runtime input(s) used by the contract to make its decision, 4) the policy branch(es) and approval/recovery path(s) executed by the contract to arrive at a decision, and 5) a link to the resulting trace. The resulting receipt should enable someone to review the acceptance of the action and see the resulting change in the trace.

Benchling's LangChain interview is a good production reminder here. Their team runs multiple model providers on scientific tasks because provider families make different mistakes, and they review production traces with a rotating fire chief. Trace review is part of the operating model, not a dashboard someone checks when everything catches fire.

Honeycomb's Canvas feature turns the existing investigation interface into a collaborative workspace where teams can pin queries, charts, findings, recommended actions, hypotheses, and text summaries. Auto-investigations can be triggered from existing triggers and anomalies. That shared investigation space is much more powerful when the trace being investigated includes governance receipts instead of just the span for the tool that went wrong.

But when things go sideways the receipt will provide valuable information about what happened. The customer record was updated when it shouldn't have been. A payment was issued twice. The AI generated email was outside of policy. The autonomous operations closed loop kept trying to remediate a problem with a bad solution. The question will not be what was the approval process for the use of AI. It will be what decision in the runtime allowed the AI to take that action.

Own the contract like production code

AI agent governance is becoming a software artifact: it has versions, it has owners, it can be tested, it can be deployed as part of a larger workload, it can be rolled back, it has audit evidence, it has release processes and incident processes, etc. Treat it like production code.

Contracts are produced for the agent runtime, and thus get treated similarly to other tool definitions. They go through review. They get tests for representative actions, regression cases for bad actions, fixtures for policy inputs, and trace assertions for receipts. They can be reverted when a contract blocks healthy work or allows unsafe work to continue.

This is less glamorous than an enterprise AI governance platform slide. Good. Glamour is a terrible control surface.

Everything else is paperwork unless it changes a decision before a side effect occurs.