0:06 Hello everyone, and welcome to lecture seven of CME 295. So today is an exciting day because we'll be talking about LLM evaluation, and in particular, how to quantify the performance of our model on a specific task. So this is an important topic because if you think about it, nowadays there are just many models out there, and you need to figure out which one you want to use for your use case.
0:38 And not only that, you also need to know what you need to improve. So you need to know how well you're performing in order to know what to improve. So for all these reasons, today's lecture is actually quite important. And with that, we're going to start by just motivating our setup. So from the start, what we're doing is we're interested in LLMs, and we've said that LLMs have set
1:09 up the problem as a text to text problem. And the key question we want to answer is whether our LLM is performing well. So this question is actually a pretty vague question because you can define performing well in a bunch of different ways. So the first way to define performing well is from a quality perspective. So as an example, to say whether your
🔒 The full, searchable transcript is available with Pro.
Common questions about this video