Designing Systems for AI Agents, Orchestration Layers and Agent Identity
Vasim Gujrati
Solutions Architect, AI & Platforms, Unico Connect
In this article
Quick Answer
AI agents fail in production because the system around the model is not built for autonomy, even when the model itself is fine. Three architectural pieces matter most. You need an orchestration layer that coordinates agent interactions, a scoped identity for each agent tied to its specific task, and structured tool interfaces that are predictable enough for agents to use without supervision. Teams that get these three right ship agents that scale beyond a single workflow.
Why "Bolt AI on Top" Fails
The first wave of enterprise AI was assistive (a chatbot grafted onto a CRM, a summariser bolted onto a ticketing queue). The second wave is agentic, meaning software that interprets intent and acts across multiple systems without continuous human input. The architectural assumptions shift completely with it.
Malay Parekh, our CEO, put it this way in DesignRush News in March 2026.
"We are moving from software that waits for input to software that can understand intent and act on it... AI agents need systems that are structured, predictable, and easy to interpret."
The systems that waited for input were optimised for human callers, with forms, GUIs, exception messages that read well to a person and retry behaviours designed around user patience. Agents do not tolerate any of that. If you build for agents, humans benefit too, while a system built only for humans makes agents fail expensively.
Piece 1, The Orchestration Layer
An orchestration layer is the controller that decides which agent runs when, hands off state between agents, prevents duplicated work, and enforces global guardrails (cost ceilings, time budgets, escalation rules).
In 2026, the dominant primitives are these.
- LangGraph gives you an explicit graph of nodes (agents) joined by edges (handoffs), with state propagated between them.
- OpenAI Agents SDK provides a runtime that handles tool dispatch, though you still have to manage state at the application level.
- Custom orchestration makes sense when domain logic dominates and the abstractions you get off the shelf become a tax. We have built it for clients in logistics and fintech where the agent workflow maps directly to a business process flow chart.
Which one you pick matters less than the discipline of having one. Codebases that scatter agent invocations across services pile up state management bugs, and those bugs compound. Putting orchestration in one layer also puts observability and cost tracking in one place, and both matter the moment finance asks how much the agents cost last month.
Piece 2, Agent Identity and Scoped Access
Treat each agent as a service principal with its own identity, distinct from the user it acts on behalf of. Without that separation, access becomes impossible to control or audit.
Concretely, that means three rules.
- Give each agent its own credential (token, key, OIDC subject) with permissions tied to its specific job, which is least privilege applied to agents. The "draft response" agent gets no write access to the production database, and the "send email" agent gets no permission to read all conversation history.
- When an agent acts on behalf of a user, make that delegated authority explicit, so the user identity travels through the call chain alongside the agent identity and the audit logs capture both.
- Scope tool calls per agent. An agent registered to handle "order intake" cannot call the "refunds" tool, even if both tools exist in the same registry. Static, declarative scoping prevents accidental cross-domain access.
Procurement and compliance teams care about this part most. Standard access management controls assume access is scoped per identity, so AI agents that all share one root credential fail the first audit against those controls.
Piece 3, Structured, Predictable Tool Interfaces
Agents call tools the way developers call functions. When the function signature is well documented, predictable and idempotent, the agent succeeds at a high rate. When the function is loosely typed, fails silently some of the time, or has hidden coupling, the agent fails in ways that make the model look "wrong."
In practice, a "structured" tool interface looks like this.
- JSON schema for every tool input and output. A markdown description of the tool does not count.
- Explicit error semantics. The tool returns either a structured success result or a structured error with a category, and never throws an exception into the void.
- Idempotency tokens for any tool with side effects. The agent will retry, and the system must not charge twice when it does.
- Latency budgets declared per tool, so the orchestrator can fall back or escalate when a tool blows its budget.
Model Context Protocol (MCP), the open standard Anthropic introduced in late 2024, codifies a lot of this. Even if you do not adopt MCP directly, the shape of an MCP server is a useful target for any agent-tool interface in your stack.
Three Practical First Steps
If you are designing agent-ready architecture in 2026 and have nothing in place yet, start with these.
- Map the workflow as a graph. Before writing any code, draw the agent workflow as a state machine, with agents or tools as the nodes and transitions as the edges. If the graph has more than ~10 nodes, split it into sub-workflows.
- Stand up one orchestration primitive. Pick LangGraph or your custom equivalent and run one workflow through it. Get observability in place before you add the second workflow.
- Issue scoped credentials per agent. Do it even if you only have two agents to start, because the pattern is cheap to establish early and expensive to retrofit.
Skip any of these and the system works fine at small scale, then breaks unpredictably at production scale.
What Is Coming Next
"In the near future, agent-to-agent communication could become as standard as API-to-API communication is today." — DesignRush News, March 2026
Most enterprise AI deployments today use a single agent. Over the next 12 to 18 months the shift will be toward multi-agent systems, where specialised agents collaborate on complex workflows. Orchestration, scoped identity and structured tools are the foundation for that shift, and teams that build them now will move significantly faster when the multi-agent wave lands.
Frequently Asked Questions
What is an orchestration layer in AI agent architecture?
An orchestration layer is the controller that decides which agent runs in which order, manages state between agents, and enforces global rules like cost limits and escalation policies. LangGraph and custom orchestrators are common implementations. When a system has no orchestration layer, agent invocations get scattered across services and become impossible to observe or govern.
Why do AI agents need their own identities?
Each agent should have its own credential with permissions scoped to its specific task, separate from the user it acts on behalf of. That separation makes access auditable, prevents accidental cross-domain actions, and follows the unique identification and least privilege controls in frameworks such as NIST SP 800-53. Without per-agent identity, one compromised agent can act with the full permissions of the system.
What is MCP and should I use it for agent tools?
Model Context Protocol (MCP) is the open standard Anthropic created for connecting AI agents to tools and data sources, now hosted by the Agentic AI Foundation under the Linux Foundation. Think of it as USB-C for AI integrations, with one standard interface serving many tools. Adopting MCP directly is optional, but designing your tool interfaces in the shape of an MCP server (typed inputs, structured errors, idempotency tokens) makes the architecture easier for agents to work with either way.
How is agent orchestration different from microservice orchestration?
Microservice orchestration assumes deterministic services that succeed or fail in known ways. Agent orchestration handles probabilistic services where the same input can produce different outputs, and where retries, escalation, and human-in-the-loop checkpoints are first-class concerns. The state machine looks similar, but the failure handling is more sophisticated.
Where should I start if I have one agent in production today?
Map the workflow as a graph before adding the second agent. Then stand up a real orchestration primitive (LangGraph or custom) and issue scoped credentials to your one agent today, while the pattern is still cheap to establish and before it becomes painful to retrofit. Add the second agent through the same orchestration layer once those are in place.
This article expands on the remarks Malay Parekh made in DesignRush News, March 2026. Unico Connect builds production-grade AI agents and agent platforms for SaaS and enterprise clients. See our Agentic AI service and Claude Code for Teams.
