Designing Systems for AI Agents: Orchestration Layers and Agent Identity
Malay Parekh
CEO & Director, Unico Connect
Quick Answer
AI agents fail in production not because the model is wrong but because the system around the model is not built for autonomy. Three architectural pieces matter most: an orchestration layer that coordinates agent interactions, scoped agent identities tied to specific tasks, and structured tool interfaces predictable enough for agents to use without supervision. Teams that get these three right ship agents that scale beyond a single workflow.
Why "Bolt AI on Top" Fails
The first wave of enterprise AI was assistive — a chatbot grafted onto a CRM, a summariser bolted onto a ticketing queue. The second wave is agentic — software that interprets intent and acts across multiple systems without continuous human input. The architectural assumption shifts completely.
As I shared in DesignRush News in March 2026:
"We are moving from software that waits for input to software that can understand intent and act on it... AI agents need systems that are structured, predictable, and easy to interpret."
The systems that waited for input were optimised for human callers. Forms, GUIs, exception messages that read well to humans, retry behaviours designed around user patience. Agents don't tolerate any of that. Build for agents and humans benefit too. Build only for humans and agents fail expensively.
Piece 1 — The Orchestration Layer
An orchestration layer is the controller that decides which agent runs when, hands off state between agents, prevents duplicated work, and enforces global guardrails (cost ceilings, time budgets, escalation rules).
In 2026, the dominant primitives are:
- LangGraph — explicit graph of nodes (agents) with edges (handoffs), state propagated between them.
- OpenAI Assistants + tool calling — runtime that handles tool dispatch, but requires application-level state.
- Custom orchestration — appropriate when domain logic dominates and off-the-shelf abstractions become a tax. We have built custom orchestration for clients in logistics and fintech where the agent workflow maps directly to a business process flow chart.
The choice between them is less important than the discipline of having one. Codebases that scatter agent invocations across services accumulate state-management bugs that compound. Centralising orchestration into one layer also centralises observability and cost tracking — both of which matter the moment finance asks how much the agents cost last month.
Piece 2 — Agent Identity and Scoped Access
Treat each agent as a service principal. It needs its own identity, distinct from the user it acts on behalf of. Without that separation, access becomes impossible to control or audit.
Concretely:
- Each agent gets its own credential (token, key, OIDC subject) with permissions tied to its specific job. The "draft response" agent does not get write access to the production database. The "send email" agent does not get permission to read all conversation history. Least privilege, agent edition.
- Delegated authority is explicit. When an agent acts on behalf of a user, the user identity travels through the call chain alongside the agent identity. Audit logs capture both.
- Tool calls are scoped per agent. An agent registered to handle "order intake" cannot call the "refunds" tool, even if both tools exist in the same registry. Static, declarative scoping prevents accidental cross-domain access.
This is the part that procurement and compliance teams care about most. Standard access-management controls assume per-identity scoping. AI agents that all share one root credential fail that audit the first time it happens.
Piece 3 — Structured, Predictable Tool Interfaces
Agents call tools the way developers call functions. If the function signature is well-documented, predictable, and idempotent, the agent's success rate is high. If the function is loosely typed, sometimes succeeds and sometimes fails silently, or has hidden coupling — the agent will fail in ways that look like the model is "wrong."
What "structured" means in practice:
- JSON schema for every tool input and output. Not a markdown description. A schema.
- Explicit error semantics. The tool returns either a structured success result or a structured error with a category. No exceptions thrown into the void.
- Idempotency tokens for any tool with side effects. The agent will retry. The system must not double-charge.
- Latency budgets declared per tool. The orchestrator can fall back or escalate when a tool blows its budget.
Model Context Protocol (MCP) — Anthropic's open standard from late 2024 — codifies a lot of this. Even if you do not adopt MCP directly, the shape of an MCP server is a useful target for any agent-tool interface in your stack.
Three Practical First Steps
If you are designing agent-ready architecture in 2026 and have nothing yet:
- Map the workflow as a graph. Before writing any code, draw the agent workflow as a state machine. Nodes are agents or tools; edges are transitions. If the graph has more than ~10 nodes, split into sub-workflows.
- Stand up one orchestration primitive. Pick LangGraph or your custom equivalent. Run one workflow through it. Get observability before adding the second workflow.
- Issue scoped credentials per agent. Even if you only have two agents to start. The pattern is cheap to establish early and expensive to retrofit.
Skip any of these and the system works fine at small scale and breaks unpredictably at production scale.
What's Coming Next
"In the near future, agent-to-agent communication could become as standard as API-to-API communication is today." — DesignRush News, March 2026
Most enterprise AI today is single-agent. The shift over the next 12–18 months will be toward multi-agent systems where specialised agents collaborate to handle complex workflows. The architectures described here — orchestration, scoped identity, structured tools — are the foundation for that. Teams that build those foundations now will move significantly faster when the multi-agent wave lands.
Frequently Asked Questions
What is an orchestration layer in AI agent architecture?
An orchestration layer is the controller that decides which agent runs in which order, manages state between agents, and enforces global rules like cost limits and escalation policies. LangGraph and custom orchestrators are common implementations. Without one, agent invocations get scattered across services and become impossible to observe or govern.
Why do AI agents need their own identities?
Each agent should have its own credential with permissions scoped to its specific task — separate from the user it acts on behalf of. This makes access auditable, prevents accidental cross-domain actions, and is required by most enterprise security frameworks. Without per-agent identity, one compromised agent can act with the full permissions of the system.
What is MCP and should I use it for agent tools?
Model Context Protocol (MCP) is Anthropic's open standard for connecting AI agents to tools and data sources. Think of it as USB-C for AI integrations — one standard interface, many tools. Even if you don't adopt MCP directly, designing your tool interfaces in the shape of an MCP server (typed inputs, structured errors, idempotency tokens) makes the architecture more agent-friendly.
How is agent orchestration different from microservice orchestration?
Microservice orchestration assumes deterministic services that succeed or fail in known ways. Agent orchestration handles probabilistic services where the same input can produce different outputs, and where retries, escalation, and human-in-the-loop checkpoints are first-class concerns. The state machine looks similar; the failure handling is more sophisticated.
Where should I start if I have one agent in production today?
Map the workflow as a graph before adding the second agent. Stand up a real orchestration primitive (LangGraph or custom). Issue scoped credentials to your one agent today — the pattern is cheap to establish now and painful to retrofit later. Then add the second agent through the same orchestration layer.
This article expands on Malay Parekh's remarks in DesignRush News, March 2026. Unico Connect builds production-grade AI agents and agent platforms for SaaS and enterprise clients. See our Agentic AI service and Claude Code for Teams.
