What Is LlamaIndex? A 2026 Guide to Workflows, Deployment, and Evaluation

What Is LlamaIndex?

LlamaIndex is an open-source Python framework for building LLM applications over your own data. It handles the work between a raw document and a grounded answer: loading data from hundreds of sources, chunking and embedding it, retrieving the relevant parts at query time, and orchestrating the multi-step logic around that retrieval. It is provider-agnostic, so the same application can run on OpenAI, Anthropic, Gemini, or a self-hosted open-weight model.

The name is a leftover from what it used to be. LlamaIndex started in 2022 as a library for indexing documents so an LLM could answer questions about them, and for a long time “LlamaIndex equals RAG” was a fair summary. It is not fair any more. The framework is now organised around event-driven workflows and ships a production server and a deployment CLI, which puts it in the same category as LangGraph rather than in a category of its own.

LlamaIndex vs LangChain: the short answer

They have converged, and anyone telling you one is categorically better is selling something.

Both are open-source Python orchestration frameworks, both are provider-agnostic, both have roughly 30-plus vector store integrations, and both now model applications as composable steps rather than monolithic chains. The differences that remain are about emphasis and execution model:

• LlamaIndex leads on document ingestion and retrieval quality. LlamaParse for messy enterprise PDFs and the LlamaHub connector registry are genuinely ahead. Its Workflows model execution as steps that consume and emit typed events, with the runtime routing each event to whichever step subscribes to it.
• LangChain / LangGraph leads on explicit state machines. LangGraph models execution as a graph of nodes and edges you define, which is a better fit when you need checkpointing, replay, and human-in-the-loop pauses at specific points.

Pick LlamaIndex when documents are the hard part. Pick LangGraph when control flow is the hard part. For most teams either will work, and the decision matters less than having an evaluation layer that tells you whether the thing you built is any good.

What you get in the box

• LlamaHub. A registry of 200+ data loaders (PDFs, web pages, Notion, S3, SQL, Slack), vector store integrations, and embedding model wrappers.
• Workflows. Event-driven step composition, the recommended way to build anything non-trivial.
• Query engines and retrievers. Still the right primitive for simple Q&A over a corpus; Workflows underneath.
• Agents. ReAct, function-calling, and tool-use patterns built on Workflows.
• llama-agents. The production server, CLI, and client for running workflows as services.
• LlamaParse. Hosted parser for messy enterprise documents.
• LlamaCloud. Managed retrieval and parsing infrastructure.

Everything except LlamaParse and LlamaCloud can be self-hosted.

TL;DR: LlamaIndex in 2026 at a Glance

LlamaIndex in 2026 is no longer “just a RAG indexing library”. It is an event-driven workflow framework with a production runtime and a built-in observability and evaluation story.

A Minimal LlamaIndex Workflow Example

Here is a single-step Workflow that turns a question into an LLM response with a retrieval step in between. Save as simple_rag.py.

One note on imports before you copy this. There are two paths to the same Workflow primitives: the in-core llama_index.core.workflow, used below, and the standalone workflows package that ships as llama-index-workflows and is what the llama-agents server expects. They are the same model; pick one per project rather than mixing them, and prefer the standalone package if you intend to deploy.

from llama_index.core.workflow import (
    Workflow, StartEvent, StopEvent, Event, step,
)
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.core.schema import NodeWithScore
from llama_index.llms.openai import OpenAI

class RetrievedEvent(Event):
    query: str
    nodes: list[NodeWithScore]

class SimpleRAG(Workflow):
    @step
    async def retrieve(self, ev: StartEvent) -> RetrievedEvent:
        # Build the index once at startup in real code; rebuilding it inside the
        # step re-reads and re-embeds the corpus on every run.
        documents = SimpleDirectoryReader("./data").load_data()
        index = VectorStoreIndex.from_documents(documents)
        retriever = index.as_retriever(similarity_top_k=4)
        nodes = retriever.retrieve(ev.query)
        return RetrievedEvent(query=ev.query, nodes=nodes)

    @step
    async def generate(self, ev: RetrievedEvent) -> StopEvent:
        # Check OpenAI's current model list for the exact identifier;
        # snapshot strings in tutorials go stale fast.
        llm = OpenAI(model="<current-model-id>")
        context = "\n\n".join([n.get_content() for n in ev.nodes])
        prompt = f"Use the context to answer.\n\nContext:\n{context}\n\nQuestion: {ev.query}"
        resp = await llm.acomplete(prompt)
        return StopEvent(result=str(resp))

async def main():
    wf = SimpleRAG(timeout=60)
    result = await wf.run(query="What does the document say about X?")
    print(result)

if __name__ == "__main__":
    import asyncio
    asyncio.run(main())

This is the entire 2026 pattern in one file. Two steps, two events, one workflow runner. Add tool-calling, conditional routing, or parallel branches by adding more steps and event types.

Adding Observability and Evaluation

The production-grade version adds two lines. One for tracing, one for span-attached evaluation.

from fi_instrumentation import register
from fi_instrumentation.fi_types import ProjectType
from traceai_llama_index import LlamaIndexInstrumentor

tracer_provider = register(project_name="rag_demo", project_type=ProjectType.OBSERVE)
LlamaIndexInstrumentor().instrument(tracer_provider=tracer_provider)

After this, every workflow run produces a trace with one root span per run() call and child spans for every retrieval and LLM call inside. Open the FutureAGI Observe UI and the trace tree appears with latency, model, token counts, and tool arguments on every span.

To attach evaluation scores to those spans, call enable_auto_enrichment() once at startup and evaluate() inside the active span:

from fi.evals import evaluate
from fi.evals.otel import enable_auto_enrichment

enable_auto_enrichment()

# Inside the generate step, after resp is computed:
context = "\n\n".join([n.get_content() for n in ev.nodes])
r = evaluate("groundedness", output=str(resp), context=context, model="turing_flash")
# Score, reason, latency_ms are now span attributes on the active generate span

That is the full integration. One pip install traceai-llama-index ai-evaluation, one register(), one instrumentor call, one enable_auto_enrichment(), one evaluate() per scoring step.

Deploying with llama-agents

If you are on llama-deploy, migrate

The llama_deploy repository now carries an unambiguous notice: “This project is deprecated. To serve workflows, use llama-agents instead.” Earlier versions of this article recommended llama-deploy, including its Redis control-plane setup. That advice is no longer current, and if you built on it, this section is your migration target.

The replacement is a set of smaller packages rather than one runtime:

The workflow code itself is unchanged, which is the good news about this migration. The same class you wrote above still runs:

from workflows import Workflow, step
from workflows.events import StartEvent, StopEvent

class HelloWorkflow(Workflow):
    @step
    async def greet(self, ev: StartEvent) -> StopEvent:
        return StopEvent(result=f"Hello, {ev.name}")

Serving it

Install the CLI and scaffold, run, and deploy from the terminal:

uv tool install -U llamactl

llamactl init                  # pick a template and scaffold the app
llamactl serve                 # run the development server locally
llamactl deployments create    # build and deploy from a Git repository
llamactl deployments logs      # tail logs from a running deployment

You can run the result locally, self-host it, or deploy it to LlamaCloud, though note that LlamaIndex currently labels cloud deployments a beta preview rather than generally available. The observability and evaluation setup from the previous section continues to work unchanged, because instrumentation attaches to the workflow steps rather than to whatever is serving them.

One more deprecation worth knowing if you touch the managed services: the llama_cloud_services packages were deprecated and their maintenance window ended on 1 May 2026. If you are still importing them, you are on unmaintained code. Python users should move to llama-cloud>=1.0 and TypeScript users to @llamaindex/llama-cloud.

Where LlamaIndex Fits Versus Alternatives

The honest comparison is that LlamaIndex and LangChain / LangGraph have converged on similar capability surfaces. Pick LlamaIndex when document parsing, retrieval quality, and pub-sub step composition are central; pick LangGraph when explicit state-machine semantics and human-in-the-loop pauses are central; pick a custom stack when you only need one vendor and one workflow shape.

Common Pitfalls in 2026

1. Mixing the old query-engine API with new Workflows in the same app. Both work, but they have different state and event models. Pick one per app and stick with it.
2. Forgetting that Workflows are async-native. All step methods are coroutines. Calling them synchronously will not raise; it will silently return a coroutine object you forgot to await.
3. Still building on llama-deploy. It is deprecated, and any tutorial that has you standing up a Redis control plane to serve a workflow is describing the old world. Move to llama-agents-server and llamactl.
4. Instrumenting after the workflow is built. Call LlamaIndexInstrumentor().instrument() before instantiating the workflow class, or the span tree will be incomplete.
5. Scoring outputs in a different process from where they ran. If you call evaluate() in a downstream service, you lose the span-attachment benefit. Score inside the workflow step that produced the output.

How Future AGI Pairs with LlamaIndex

LlamaIndex is an orchestration framework. Future AGI is the evaluation and observability layer that sits on top, the same way you would pair Datadog with a Flask app. Once you have a Workflow running:

• traceAI auto-instruments every step, retriever, and LLM call into OpenTelemetry spans (vendor-portable, OTel GenAI semantic conventions compatible).

• ai-evaluation scores any output through fi.evals.evaluate("groundedness", ...), evaluate("hallucinations_v1", ...), or any of 100+ Turing-cloud templates, with auto-enrichment attaching scores to the active span.

• agent-simulate drives multi-turn persona scenarios against the workflow for pre-production scenario testing.

• agent-opt searches the prompt space for variants that lift evaluator scores on a failing-trace dataset.

You keep LlamaIndex for retrieval, parsing, orchestration, and deployment. You add Future AGI for span-attached evaluation, persona scenarios, and prompt optimization. No vendor lock-in; ai-evaluation is Apache 2.0 and traceAI is Apache 2.0.

Conclusion

LlamaIndex in 2026 is a more opinionated, more production-shaped framework than the 2024 version. Workflows are the composition primitive, llama-agents is the production runtime, and OpenTelemetry-compatible observability is available through traceAI instrumentation. If you are building any document-heavy or retrieval-heavy LLM application this year, LlamaIndex is one of the two defaults to evaluate (the other is LangGraph). Once your workflow runs, the natural next step is span-attached evaluation through traceAI plus fi.evals.evaluate(), which turns “does the RAG pipeline work” from an offline notebook question into a continuous production signal. For the metrics that matter most, see our guide to RAG evaluation metrics.

Get started with LlamaIndex | Future AGI evaluate platform | traceAI on GitHub

Sources

• LlamaIndex documentation: https://developers.llamaindex.ai/python/framework/
• LlamaAgents overview: https://developers.llamaindex.ai/python/llamaagents/overview/
• llamactl getting started: https://developers.llamaindex.ai/python/llamaagents/llamactl/getting-started/
• llama-agents (workflows-py) repository: https://github.com/run-llama/workflows-py
• llama-deploy deprecation notice: https://github.com/run-llama/llama_deploy
• llama_cloud_services deprecation notice: https://github.com/run-llama/llama_cloud_services
• traceai-llama-index (OTel auto-instrumentation): https://github.com/future-agi/traceAI
• Future AGI ai-evaluation (Apache 2.0): https://github.com/future-agi/ai-evaluation/blob/main/LICENSE
• Future AGI evaluate docs: https://docs.futureagi.com/docs/sdk/evals/evaluate/
• OpenInference spec: https://github.com/Arize-ai/openinference
• OpenTelemetry GenAI semantic conventions: https://opentelemetry.io/docs/specs/semconv/gen-ai/
• LlamaHub: https://llamahub.ai/
• LlamaParse changelog and docs: https://developers.llamaindex.ai/python/framework/changelog/