Machine Learning FAQ

Evaluating LLM outputs is difficult because, unlike many classic machine-learning tasks, there is often no single uniquely correct answer.

For text classification, evaluation can be straightforward: compare predicted labels with ground-truth labels and compute accuracy or F1. But for open-ended generation, the model may produce a response that is:

• factually correct but phrased differently from the reference
• partially correct
• correct but too verbose
• stylistically poor but still useful
• fluent but wrong

That means exact-match evaluation is often too brittle, while purely subjective judgment can be inconsistent and expensive.

The repo makes this contrast explicit in chapter 7: after instruction finetuning, it becomes clear that evaluating generated responses is much less straightforward than measuring spam-classification accuracy in chapter 6.

One common approach is to use task benchmarks with constrained answers, such as multiple-choice or short-answer tests. These are easier to score automatically, but they only cover part of what makes a model useful.

A second approach is human evaluation. Humans can judge whether a response is helpful, correct, safe, concise, and aligned with the instruction. This is often high quality, but it is slow, expensive, and can vary across raters.

A third approach, which the repo demonstrates in chapter 7, is LLM-as-a-judge evaluation. Here, another stronger model evaluates the candidate response against the prompt and a reference answer and returns a score or preference.

This can be useful because it is faster and cheaper than large-scale human review, and it can scale to many examples. The repo includes both a local Llama 3 based workflow via Ollama and an OpenAI API based workflow for this style of evaluation.

Still, LLM-based evaluation is not perfect. Judge models can have their own biases, formatting preferences, or failure modes. So it is best understood as a practical tool, not an infallible oracle.

In practice, common evaluation methods include:

• automatic benchmark scores for tasks with clear answers
• human pairwise or scalar judgments
• LLM-as-a-judge scoring
• head-to-head preference comparisons between models
• task-specific metrics, when a downstream application has a clear objective

The main reason evaluation is hard is that LLM quality is multi-dimensional. A response may need to satisfy correctness, relevance, style, completeness, safety, and instruction adherence all at once. Different evaluation methods capture different parts of that picture.

So the best evaluation strategy usually depends on the application. If the task has a single correct answer, automatic scoring may work well. If the task is open-ended, human or model-based judgment often becomes necessary.

In short, LLM evaluation is difficult because open-ended language tasks rarely have one obvious gold answer, so practitioners usually combine benchmarks, human judgment, and model-based judging rather than relying on one simple metric.