LlamaIndex Agent Framework

1. Vendor and Product Reality

LlamaIndex began in late 2022 as GPT Index, a focused open-source library by Jerry Liu for indexing private data into vector and structured stores so that large language models could reason over it. Within two years it became one of the two reference data frameworks for LLM applications — alongside LangChain — and the commercial entity LlamaIndex Inc. raised institutional venture funding to build the hosted layer that surrounds the open-source core. The framework now sits inside a meaningful share of production retrieval-augmented-generation (RAG) deployments at enterprises, AI-native startups, and internal platform teams, with millions of monthly downloads and a developer community whose conventions effectively define the idiom of LlamaIndex application code.

The open-source core is organized around a small set of first-class abstractions: documents and nodes as the unit of indexed content; indices (vector, summary, tree, knowledge-graph, keyword) as retrieval substrates; retrievers as the read interface to indices; query engines as the composition of retrieval, post-processing, and response synthesis; and — most importantly for this analysis — agents as the multi-step reasoning surface that consumes query engines as tools. LlamaParse, the company's commercial document-ingestion service, supplies high-fidelity parsing of complex documents (tables, figures, multi-column PDFs) that feeds the indexing surface. LlamaCloud is the hosted commercial layer offering managed indices, managed parsing, and increasingly managed agent deployment, billed under enterprise terms.

The current agent architecture decomposes execution into AgentRunner and AgentWorker — a deliberate separation introduced to give the framework a stable public API while permitting reasoning strategies (ReAct, OpenAI function-calling, structured planner, multi-step query planning) to evolve as worker variants. AgentRunner owns the task lifecycle: it accepts a user input, manages the persistent task state, holds the memory object, dispatches the worker for each step of the reasoning loop, and surfaces intermediate results. AgentWorker owns the per-step reasoning: it consumes the task state, calls the LLM with the appropriate prompt, parses the output into a tool call or final response, executes the tool, and returns a step result. ReAct, OpenAI function-calling, and structured-planner agents are all expressed as worker variants over a common runner interface, and query engines plug in as tools — giving agents direct access to the retrieval and synthesis stack the framework was originally built around.

Adoption is broad and increasingly enterprise. LlamaIndex sits inside production deployments at financial-services firms, large technology companies, regulated healthcare and legal organizations, and a long tail of internal platform teams. The agent abstractions, more than the original index abstractions, are the entry point for teams building multi-step LLM workflows over enterprise data, and the LlamaCloud commercial roadmap is increasingly oriented around hosted agent deployment with multi-tenant isolation, observability, and governance hooks. The product is real, the adoption is real, and the commercial trajectory is plausibly toward becoming the agent-runtime layer of the enterprise AI stack.

2. Architectural Gap

The AgentRunner / AgentWorker decomposition is a clean execution-time abstraction, but it is not a schema. Agents in LlamaIndex are configurations: a system prompt, a tool list, a memory object, a worker class, optionally a callback manager and a verbosity flag. There is no first-class semantic object that defines what an agent is in a structurally validated, credentialed way — what state it may hold, what mutations it may emit, what other agents it may compose with, what tools it may invoke under which conditions, and under whose authority any of those operations are admissible. The agent's identity is its Python object; its admissible behavior is whatever the worker code happens to do at runtime; its memory is a free-form sequence of messages with at best ad-hoc constraints.

The practical consequence shows up at three boundaries. First, agent-to-agent composition is ad hoc; one agent invoking another is implemented as a tool call where the inner agent is wrapped behind a tool interface, but there is no structurally validated handoff — no schema declaring what the calling agent may pass, what the called agent may return, what credentials flow with the call, what the calling agent is permitted to do with the result. Multi-agent topologies — planner-and-specialists, critic-and-worker, supervisor-and-team — are constructed by convention, with correctness enforced by careful prompt engineering and developer discipline rather than by structural validation. Second, mutations to agent state — memory updates, tool-result writes, planner revisions, scratchpad changes — are not schema-bound, so there is no built-in way to prove that a given mutation was admissible at the moment it occurred. A misbehaving worker can write arbitrary content to memory; an upstream prompt-injection can manipulate the planner; nothing in the framework rejects a mutation that violates the agent's intended invariants because no invariants are declared. Third, multi-tenant deployments must implement credentialing outside the framework, typically in a wrapping service that gates which user can invoke which agent, because the agent object itself does not carry credential-scoped schema; the framework treats authorization as the surrounding application's problem.

None of this prevents shipping useful agents. LlamaIndex agents demonstrably ship at scale and produce real business value. What it prevents is shipping agents that are auditable and composable across organizational boundaries without significant external scaffolding. Enterprise procurement of agent frameworks is increasingly gated on questions the framework cannot itself answer: can you prove what state this agent was in at the time it took this action; can you prove that a memory write was admissible under the policy in force at that moment; can you compose this agent with another team's agent without trusting the other team's prompt engineering; can you isolate tenant A's agent state from tenant B's not by deployment hygiene but by structural property of the agent object. Today those questions are answered by deployment wrappers, observability platforms, and governance overlays — none of which are properties of the agent itself.

3. What the AQ Agent-Schema Primitive Provides

The agent-schema primitive defines cognition-compatible semantic agent objects: typed, structurally validated, credentialed, with schema-bound mutation as a first-class operation. Each agent is an instance of a schema that declares its admissible state, its admissible transitions, the credentials under which those transitions may be invoked, the tools it may use under each credential, and the composition contracts it offers to other agents. The schema is a formal artifact, not a prose document — it is machine-checkable, versioned, and binds the agent's runtime behavior to the contracts it advertises.

Structural validation runs on every state transition, not only at construction. A memory write that violates the schema is rejected at the boundary; a planner revision that exceeds the schema's admissible plan space is rejected; a tool invocation outside the schema's admissible tool set is rejected. The validation is structural in the sense that it is a property of the agent's type, not of an external policy engine watching the agent. Credentialed schema means the schema itself is parameterized by authority — an agent operating under one credential exposes a different admissible surface than the same agent under another credential. A finance-analyst agent under a tier-one credential may invoke the trading tool; the same agent under a tier-three credential may not, and the schema rejects the invocation rather than relying on an external gate. Schema-bound mutation means that any change to agent state, including memory writes, planner revisions, tool-result integration, and scratchpad updates, must be admissible against the schema or it is rejected at the boundary. The agent's history is therefore a sequence of schema-validated mutations, which is the structural property auditability requires.

Agent-to-agent composition becomes structurally validated handoff. The schema declares the agent's input contract, output contract, and the credentials it consumes and emits when invoked by another agent. A planner agent dispatching to a specialist agent passes a typed handoff that the specialist's schema accepts or rejects, with credential propagation following the schema's declared rules. Multi-agent topologies become composable in the type-theoretic sense — you can verify that a topology is well-formed by checking schemas, not by running it and watching what happens.

The result is an agent object that is cognition-compatible — usable by reasoning loops in the AgentRunner / AgentWorker idiom that LlamaIndex developers already know — while also being structurally governable. The primitive does not replace LLM reasoning; it gives the reasoning a typed substrate to operate on. The inventive step is the combination of cognition-compatibility (agents remain reasoning surfaces, not rigid state machines) with structural validation, credentialed schema, and schema-bound mutation as a single object — the missing architectural element that current agent frameworks treat as an external concern.

4. Composition Pathway

LlamaIndex's existing AgentRunner remains the execution engine. The AgentWorker remains the per-step reasoning unit. Query engines continue to act as tools; retrievers, response synthesizers, indices, and the broader LlamaIndex ecosystem continue to function unchanged. What changes under composition with agent-schema is that the agent identity and state become schema instances rather than free-form configurations. The runner's task lifecycle calls become schema-validated transitions; the worker's tool calls become schema-bound mutations whose admissibility is checked against the agent's credentialed schema before they are dispatched.

The integration surface is well-defined. The AgentRunner constructor accepts a schema reference alongside the existing system prompt, tool list, and worker; the runner invokes the schema's validation hooks before dispatching to the worker and after receiving a step result. The worker's tool invocations route through a schema-aware tool dispatcher that consults the agent's current credential and the schema's admissible-tool predicate. Memory writes route through a schema-bound memory adapter that rejects inadmissible mutations at the boundary. The public API that developers depend on — `agent.chat()`, `agent.query()`, `AgentRunner.from_tools()` — remains stable; the schema is an additional, optional argument that defaults to a permissive schema preserving current behavior for users who do not opt in.

Multi-agent topologies — a planner agent dispatching to specialist agents, a critic agent reviewing a worker's output, a supervisor coordinating a team — gain structurally validated handoffs in place of ad-hoc tool invocations. The handoff is expressed as a schema-declared composition contract: the planner's schema declares what it may dispatch and to which specialist schemas; the specialist's schema declares what it accepts and what it returns. LlamaCloud-hosted deployments gain a substrate on which multi-tenant agent isolation is a property of the schema rather than a property of the deployment wrapper — tenant A's agent and tenant B's agent are instances of schemas whose credentials structurally prevent cross-tenant state access, regardless of how the deployment is sliced.

For LlamaParse and the broader retrieval surface, the composition is straightforward: parsed documents and indexed nodes are inputs to schema-validated mutations rather than free-form writes to agent memory. The chain of provenance — which document, parsed by which LlamaParse run, indexed under which credential, surfaced by which retriever, integrated into which agent's memory under which schema rule — becomes a structural property rather than a logging artifact. Adoption can be incremental: a team can add a schema to one critical agent without touching the rest of its LlamaIndex application, and expand schema coverage as the governance need intensifies.

5. Commercial Position and Licensing Implication

For LlamaIndex Inc., adopting agent-schema as a substrate is commercially aligned with the trajectory the company is already on. Enterprise procurement of agent frameworks is increasingly gated on auditability and multi-tenant isolation; the open-source core's reputation for developer ergonomics does not by itself answer those gates, and the LlamaCloud roadmap already points toward governance, observability, and isolation as commercial differentiators. A schema-bound agent object closes the gap without displacing the AgentRunner / AgentWorker public API that the developer community has standardized on, which preserves the framework's adoption advantage while elevating its enterprise floor.

It also reframes the competitive surface against LangChain, CrewAI, AutoGen, Microsoft's Semantic Kernel, and the steadily growing list of agent frameworks. Differentiation on tool count and integration breadth has commoditized — every framework has connectors to the same vector stores, the same LLM providers, the same enterprise SaaS — and differentiation on developer ergonomics is converging. Differentiation on structural agent governance has not commoditized; no major open-source agent framework today offers credentialed schema and schema-bound mutation as first-class properties of the agent object. The vendor that ships this first establishes a reference point that competitors must respond to, and that response takes architectural work, not feature checklists.

The agent-schema primitive licenses cleanly beneath the LlamaIndex open-source core and the LlamaCloud commercial layer. The primitive supplies the substrate — schema definition, credentialed validation, schema-bound mutation enforcement, composition-contract semantics — while LlamaIndex continues to own the framework ergonomics, the LlamaParse and indexing surface, the developer community, and the hosted commercial layer. The fitting commercial structure is an embedded substrate license under which LlamaIndex Inc. integrates agent-schema beneath the AgentRunner / AgentWorker public API and sub-licenses schema participation to LlamaCloud customers as part of the enterprise tier. Pricing is per-schema-instance or per-credentialed-tenant rather than per-seat, which aligns with how regulated customers consume agent governance. Existing open-source users see the substrate as an optional, drop-in upgrade for governance-sensitive deployments; LlamaCloud customers see it as the architectural foundation that makes regulated multi-tenant agent deployments procurement-viable. The honest framing — agent-schema does not replace LlamaIndex; it gives LlamaIndex the substrate enterprise procurement is increasingly demanding and that no agent framework has yet supplied.