Getting Started

Install

pip install bandito

The base package includes tracing, grading, and the TUI. Optional extras for additional backends:

pip install bandito[s3]           # S3 storage
pip install bandito[postgres]     # PostgreSQL storage
pip install bandito[langfuse]    # Read traces from Langfuse

Or install all extras: pip install bandito[s3,postgres,langfuse]

Create an account

bandito signup

Free account. Required for grading, judge, and analysis features. Tracing works without one.

Instrument your app

Wrap your LLM calls with bandito.trace():

import bandito

with bandito.trace("my-chatbot") as t:
    response = my_llm_call(query)
    t.done(input=query, output=response)

That's it. Traces are written to ~/.bandito/traces/ by default.

Tag your traces

Use tag= to label traces by environment, version, or anything else:

with bandito.trace("my-chatbot", tag="prod") as t:
    response = my_llm_call(query)
    t.done(input=query, output=response)

Or set BANDITO_TAG=prod in your environment — all traces get tagged automatically.

Filter by tag in any command:

bandito observe basics --project my-chatbot --tag prod

Framework auto-extraction

If you use Pydantic AI, Anthropic, OpenAI, or LangChain, pass the result directly to t.done() — Bandito extracts input, output, LLM spans, tool calls, and token usage automatically:

from pydantic_ai import Agent

agent = Agent("anthropic:claude-sonnet-4-20250514")

with bandito.trace("my-agent") as t:
    result = agent.run_sync("What's the weather?")
    t.done(result)  # auto-extracts everything

Add detail with spans

For visibility into multi-step pipelines, add spans:

with bandito.trace("my-chatbot") as t:
    with t.span("retrieve-docs", "retrieval") as s:
        docs = search(query)
        s.done(output=docs)

    with t.span("generate", "llm", model="gpt-4o") as s:
        response = llm.call(query, docs)
        s.done(output=response, cost=0.003, input_tokens=312)

    t.done(input=query, output=response)

Span kinds: llm, tool, retrieval, agent, chain, embedding, guardrail, custom.

Local-only mode

Force traces to local storage regardless of org storage config:

with bandito.trace("my-chatbot", local=True) as t:
    response = my_llm_call(query)
    t.done(input=query, output=response)

Or set BANDITO_LOCAL=true in your environment.

Configure storage

By default, traces are stored locally at ~/.bandito/traces/. To use S3 or PostgreSQL for your org:

bandito setup config

Interactive flow — choose backend, enter settings, validates connectivity. All projects in your org use the same storage.

For deployed apps, set environment variables:

BANDITO_API_KEY=bnd_live_...
AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...

What gets captured

Each trace includes:

• Input/output — what went in, what came out
• Spans — every step with timing, cost, tokens
• Metadata — model, provider, session_id, user_id, tags

What's next

bandito observe projects          # see your projects
bandito observe basics --project my-chatbot  # stats + patterns
bandito tui                       # grade traces in the terminal

See Observing Traces to understand what's happening in your app.