Education

Stanford CME295 Transformers & LLMs | Autumn 2026 | Lecture 7 - LLM Evaluation

by Stanford Online

Share:

📚 Main topics

  • Defining LLM performanceEvaluation can focus on quality, reliability, or product outcomes; this lecture concentrates on quality. The challenge is that LLMs perform many kinds of tasks, quality has multiple dimensions, and open-ended tasks can have many valid answers. 1:09
  • Human ratings and agreementHuman raters can judge nuanced outputs, but their judgments may differ, especially on subjective criteria. Cohen’s kappa compares observed agreement with agreement expected by chance. 6:25
  • Rule-based evaluation and Pass@kCode and math can often be checked against tests or known answers. Pass@k estimates the chance that at least one of k attempts succeeds, while consensus@k uses the most common answer when correctness cannot be checked directly at inference time. 14:43
  • Natural-language metricsBLEU, METEOR, and ROUGE compare generated text with references, but can miss valid paraphrases and stylistic variations. 28:55
  • LLM-as-a-judgeA model can score outputs against task-specific criteria and provide a rationale, without requiring a single reference answer. Small human-rated examples can guide the prompt and help measure judge performance. 33:40
  • Prompt iteration and structured outputsImprove judge prompts by reviewing errors, adjusting instructions, and re-evaluating against a target metric. Constrained decoding or structured outputs can ensure responses follow a parseable format such as JSON. 37:20
  • Judge biases and practicesLLM judges may favor a particular answer position, verbosity, or their own outputs. Use clear criteria, often binary labels, ask for rationale before the score, check for biases, calibrate against human ratings, and account for nondeterminism. 44:38
  • Evaluating agentsAgent performance depends on tool selection, arguments, tool execution, tool results, and final response generation. Diagnosing failures at each step can reveal fixes to routing, prompts, tool descriptions, implementations, or outputs. 53:24
  • Benchmarks and model comparisonsArtificial Analysis aggregates benchmarks across general knowledge, knowledge work, coding, and science. These areas differ in how reliably outputs can be checked, with coding and science often allowing deterministic tests. 12:39
  • Benchmark gaming and qualityPublic benchmarks can be learned or gamed, so scores may not reflect real-world performance. Private evaluations, varied tasks, and careful benchmark design can help, but any metric may become unreliable when it becomes a target. 1:24:05
  • Cost, speed, and user preferencesThe Pareto frontier identifies models that offer the best trade-offs between quality and cost or other constraints. Pairwise user comparisons can capture preferences, but are costly, slow, and subjective. 1:33:54

✨ Key takeaways

  • Match evaluation to the taskUse deterministic checks when possible, especially for coding and math; use human ratings or LLM judges when outputs are open-ended. 15:45
  • Treat benchmark scores as evidence, not proofBenchmark results can diverge from user experience, particularly when tasks are public or narrowly specified. 1:24:05
  • Evaluate the whole agent systemA poor final answer may originate in tool routing, arguments, execution, tool output, or synthesis, so inspect each stage rather than blaming the model alone. 1:09:02
  • Build evaluations around real needsDefine what matters for the use case, then use evaluation results to improve prompts, tools, and agent behavior. 1:42:47

🧠 Lessons learned

  • Make subjective criteria explicitClear rater guidelines and agreement measurements help reveal when a quality dimension is ambiguous or inconsistently judged. 7:58
  • Calibrate and scrutinize LLM judgesCompare judge decisions with human judgments, inspect errors, and test for position, verbosity, and self-enhancement bias. 49:14
  • Design tools for models to useClear tool names, descriptions, arguments, and interpretable outputs can prevent common agent failures. 59:40
  • Prefer robust, task-relevant checksHeuristic or deterministic evaluation is valuable when available; judge-based evaluation needs careful tuning and validation. 1:43:48

📚 Conclusion/next steps

  • Adopt an evaluation-driven workflowDefine the desired outcomes first, build an evaluation around them, and iteratively improve the agent and its tools based on measured failures. 1:42:47
  • Choose trade-offs deliberatelyPrioritize what is crucial for the application, and compare models on the dimensions that matter, such as quality, cost, and response time. 1:34:25

Transcript excerpt

0:06 Hello everyone, and welcome to lecture seven of CME 295. So today is an exciting day because we'll be talking about LLM evaluation, and in particular, how to quantify the performance of our model on a specific task. So this is an important topic because if you think about it, nowadays there are just many models out there, and you need to figure out which one you want to use for your use case.

0:38 And not only that, you also need to know what you need to improve. So you need to know how well you're performing in order to know what to improve. So for all these reasons, today's lecture is actually quite important. And with that, we're going to start by just motivating our setup. So from the start, what we're doing is we're interested in LLMs, and we've said that LLMs have set

1:09 up the problem as a text to text problem. And the key question we want to answer is whether our LLM is performing well. So this question is actually a pretty vague question because you can define performing well in a bunch of different ways. So the first way to define performing well is from a quality perspective. So as an example, to say whether your

🔒 The full, searchable transcript is available with Pro.

🔒 Unlock Premium Features

This is a premium feature. Upgrade to unlock unlimited Q&A, transcripts, mindmaps, and translations.

Questions & Answers

Common questions about this video

What makes evaluating LLM output quality challenging?

LLMs handle many kinds of tasks, and quality can involve different dimensions such as correctness, relevance, safety, and tone. Many tasks also have multiple valid answers, while increasingly complex systems like agents make it harder to diagnose where a failure occurred. 2:44

Why can raw agreement rates be misleading when measuring agreement between human raters?

Raters can agree by chance, especially when they choose between labels at random. Cohen’s kappa addresses this by comparing observed agreement with expected chance agreement: (observed agreement − expected agreement) / (1 − expected agreement). 9:02

What does Pass@k measure, and how is it estimated?

Pass@k is the probability that at least one of k attempts succeeds. If n attempts are sampled and c succeed, it can be estimated as 1 − (n − c choose k) / (n choose k). 17:19

What is an LLM-as-a-judge, and what are some advantages?

An LLM-as-a-judge uses a language model to evaluate an output against criteria such as relevance. It can assess responses without a reference answer, handle different ways of expressing the same idea, and provide a rationale that can help with debugging. 33:40

What biases should you watch for when using an LLM as a judge?

Common biases include position bias, where the judge favors one response position; verbosity bias, where it favors longer answers; and self-enhancement bias, where it favors its own responses. Comparing responses in reversed order and explicitly telling the judge not to reward verbosity can help mitigate these biases. 44:38

How can developers debug failures in an AI agent?

Inspect each stage of the agent loop: tool selection and argument generation, tool execution, and final response synthesis. Check for issues such as missing tools, unclear tool descriptions, invalid arguments, backend errors, uninformative tool outputs, or excessive irrelevant context. 53:24

What is the Pareto frontier in model evaluation?

The Pareto frontier is the boundary of models that offer the best quality for a given cost, or the lowest cost for a given quality. It can also be used to consider trade-offs involving other factors, such as response time. 1:33:54

What is Goodhart’s law, and why does it matter for benchmarks?

Goodhart’s law says that when a measure becomes a target, it ceases to be a good measure. If models or providers optimize directly for benchmark scores, they may improve the proxy without improving the real-world capability it was intended to represent. 1:39:35

🔒 Unlock Premium Features

Access to Chat is a premium feature. Upgrade now to unlock unlimited studying tools.

🔒 Unlock Premium Features

Access to Mindmap is a premium feature. Upgrade now to unlock unlimited studying tools.

🔒 Unlock Premium Features

Access to Translation is a premium feature. Upgrade now to unlock unlimited studying tools.

Get unlimited summaries, Q&A, transcripts and more with Pro

Upgrade to Pro

Suggestions

🔒 Unlock Premium Features

Access to AI Suggestions is a premium feature. Upgrade now to unlock unlimited studying tools.