Evaluator-Optimizer Agent
An iterative refinement loop where an 'Evaluator' provides granular feedback on an 'Optimizer’s' output until quality thresholds are met.
errorWhat Problem Does the Evaluator-Optimizer Agent Pattern Solve?
LLMs often struggle with complex tasks requiring high precision in a single pass. A single prompt may produce output that is partially correct but contains subtle errors in logic, formatting, or completeness that are difficult to catch without structured review.
check_circleHow the Evaluator-Optimizer Agent Pattern Works
Decouple the roles. One agent (Optimizer) focuses on creative generation, while a second agent (Evaluator) applies structured rubrics to critique the output. The loop continues—refine, evaluate, refine—until the Evaluator’s criteria are fully satisfied or a maximum iteration count is reached.
Evaluator-Optimizer Agent Pattern Architecture
Rendering diagram...
Implementation by Language
Evaluator-Optimizer Agent Pattern in the Real World
“Think of a student writing an essay (Optimizer) and a teacher grading it with detailed feedback (Evaluator). The student revises based on the red-ink comments and resubmits. This cycle repeats until the essay meets the teacher’s standards—or the deadline (max iterations) is reached.”
Frequently Asked Questions
helpWhat is the difference between the Evaluator-Optimizer pattern and simple retry logic?
Simple retry logic repeats the same prompt hoping for a better result. The Evaluator-Optimizer pattern provides structured, rubric-based feedback from the Evaluator that guides the Optimizer's next attempt, making each iteration targeted rather than random.
helpHow many iterations should an Evaluator-Optimizer loop run?
Most implementations cap at 3-5 iterations. Beyond that, quality gains diminish while cost and latency increase linearly. Set a max iteration limit and accept the best output if the quality threshold isn't met.
helpCan I use the same LLM for both the Evaluator and Optimizer roles?
Yes, but using different system prompts or even different models for each role often improves results. The key is role separation — the Evaluator needs clear rubrics and should not 'remember' its own generation attempts.