Grading

Grade your LLM outputs in the terminal. Human grades power judge calibration, quality analysis, and improvement testing.

Setup

Grading requires a Bandito account (evaluations are stored on the backend):

bandito signup

Launch the TUI

bandito tui
bandito tui --project my-chatbot
bandito tui --project my-chatbot --tag prod

Two screens:

Dashboard — project selector, trace counts, grading progress, cost/latency averages, span breakdown.

Grading — trace list on the left, input/output on the right. Grade with keyboard shortcuts.

Keybindings

Key	Action
y	Grade good (1)
n	Grade bad (0) — enters span selection mode
s	Skip trace
c	Add/edit comment
d	Toggle span detail view
q	Back / quit

Span selection (after pressing n)

When you grade bad, you can mark which spans caused the problem:

Key	Action
Up/Down	Navigate spans
Space	Toggle span as problematic
Enter	Submit grade with selected spans
Esc	Cancel span selection

What gets recorded

Each grade creates an evaluation on the backend with:

• human_score — 0 (bad) or 1 (good)
• human_comment — optional explanation
• human_span_ids — which spans were problematic (for bad grades)
• human_graded_by — your identity

Multiple team members can grade the same project. Last grade wins, with full history in the audit trail.

How many to grade

15-20 grades is enough to calibrate a judge. Focus on variety — grade some good outputs, some bad ones, some edge cases. The patterns you notice while grading will inform your rubric.

What's next

Once you've graded 15+ traces:

bandito judge calibrate --project my-chatbot

See LLM-as-Judge to scale your quality evaluation beyond manual grading.