Skip to content
Back to Advanced agents

Advanced agents · Chapter 8 of 10

Spending compute

Evaluator-optimizer loops, reflection, best-of-n and tree search spend more model calls to get a better answer, and each needs a budget.

20 minutes0 of 4 goals done

Goals

  • Run a generator and judge loop with a rubric and a round cap.To do
  • Generate several answers and pick one by vote.To do
  • Search a small tree of plans under an expansion budget.To do
  • Pass the check.To do

Step 1 of 4: The read

More calls for a better answer

One model call gives one answer. More calls can give a better one: a judge grades drafts, several samples are compared, or a search looks ahead. Each pattern multiplies the calls, the cost and the wait, so each needs a budget that stops it.

A generator and a judge

An evaluator-optimizer loop has two roles. The generator writes a draft. The judge is a separate model call with its own prompt: it scores the draft against a rubric, a short list of criteria each scored on a scale, and writes notes for the next draft. A threshold says when a draft passes, such as every criterion 4 or more out of 5. A round cap bounds the loop: when it is reached, the loop returns the best draft so far, marked as not passed.

Reflection, or Self-Refine, is the same loop with the model critiquing its own draft. A separate judge is easier to keep strict and to test. As in Agent systems chapter 8, compare its scores with people's before you trust it.

Best-of-n and voting

Best-of-n asks for n answers to the same prompt, sampled at a temperature above 0 so they differ, and keeps one: a judge picks the best, or a vote keeps the most common final answer. Voting on final answers is called self-consistency; it suits answers you can compare exactly, such as an amount. In LangGraph the samples are n Sends in one superstep: n calls that run at once, so you wait only for the slowest.

Tree search

A tree search plans steps ahead. It expands a step into possible next steps, gives each a value estimate, how promising it looks, and keeps expanding the most promising open step. Language Agent Tree Search (LATS, 2023) does this with Monte Carlo tree search, using the model to propose steps, estimate values and reflect on failures. An expansion budget caps how many steps it expands. The lab runs a smaller best-first search over two levels, with simulated values.

When it is not worth it

Extra calls pay off when a better answer matters and something can tell better from worse: a rubric, an exact answer to vote on, a value to estimate. If one call already passes, or nothing can check quality, they only add cost and time. Measure it: run your evaluation cases with and without the pattern.

Step 2 of 4: Try it: Three ways to spend more calls

Three ways to spend more calls

Halden's ticket T-48213, three ways: a judge loop polishes the reply, a vote picks the refund amount, and a tree search plans the steps. Each stops at its budget.

Mode:RealSimulatedReal: LangGraph.js 1.5.1 runs in this page: the three graphs, their Sends, reducers and loops, and the caps that stop them. Simulated: the model's decisions are scripted: the drafts, the judge's scores, the samples and the tree search's values. The ticket is fictional.

Step 3 of 4: Check

Check

Code
const CAP = 3;
const PASS = 4; // per criterion, of 5
const scores = (r) =>
  Object.values(r.scores);
const passes = (r) =>
  scores(r).every((s) => s >= PASS);
const total = (r) =>
  scores(r).reduce((a, b) => a + b);

function review(state) {
  const last = state.rounds.at(-1);
  if (passes(last)) return 'finish';
  if (state.rounds.length >= CAP)
    return 'finish';
  return 'generate';
}

function finish(state) {
  const last = state.rounds.at(-1);
  const best = passes(last) ? last
    : state.rounds.reduce((b, r) =>
        (total(r) > total(b) ? r : b));
  return {
    answer: best.draft,
    passed: passes(best),
  };
}

// The judge's scores, round by round:
// 1: accurate 3, complete 2, tone 4
// 2: accurate 4, complete 4, tone 3
// 3: accurate 4, complete 3, tone 3
What does this run return, and after how many model calls (one per draft, one per judgement)?

Step 4 of 4: What you can do now

What you can do now

  • Build an evaluator-optimizer loop with a separate judge, a rubric, a pass threshold and a round cap.
  • Run best-of-n samples as Sends in one superstep and keep the answer most of them agree on.
  • Explain how a tree search expands, scores and backs up values under an expansion budget.
  • Decide when extra calls are worth their cost, and measure it with your evaluation cases.