Agentic AIPHPverifiedVerified

Evaluator-Optimizer Agent Pattern in PHP

An iterative refinement loop where an 'Evaluator' provides granular feedback on an 'Optimizer’s' output until quality thresholds are met.

How to Implement the Evaluator-Optimizer Agent Pattern in PHP

1Step 1: Define the Feedback value object and interfaces

class Feedback
{
    public function __construct(
        public readonly bool $isPass,
        public readonly string $critique,
        public readonly float $score,
    ) {}
}

interface Optimizer
{
    public function generate(string $task): string;
    public function refine(string $current, string $feedback): string;
}

interface Evaluator
{
    public function check(string $output): Feedback;
}

2Step 2: Implement the iterative refinement loop

const MAX_ITERATIONS = 5;

function refinementLoop(
    string $task,
    Optimizer $optimizer,
    Evaluator $evaluator,
): string {
    $currentOutput = $optimizer->generate($task);

    for ($i = 0; $i < MAX_ITERATIONS; $i++) {
        $feedback = $evaluator->check($currentOutput);

        if ($feedback->isPass) {
            return $currentOutput;
        }

        $currentOutput = $optimizer->refine(
            $currentOutput,
            $feedback->critique,
        );
    }

    return $currentOutput;
}

<?php
declare(strict_types=1);

// [step] Define value objects and interfaces with strict typing
final readonly class Feedback
{
    /**
     * @param array<string, float> $categories
     */
    public function __construct(
        public bool $isPass,
        public string $critique,
        public float $score,
        public array $categories = [],
    ) {
        if ($score < 0 || $score > 100) {
            throw new \InvalidArgumentException("Score must be between 0 and 100, got {$score}");
        }
    }
}

final readonly class OptimizerConfig
{
    public function __construct(
        public string $model,
        public float $temperature = 0.7,
        public int $maxTokens = 2048,
    ) {}
}

final readonly class EvaluatorConfig
{
    public function __construct(
        public string $model,
        public string $rubric,
        public float $passThreshold = 80.0,
    ) {}
}

final readonly class RefinementResult
{
    public function __construct(
        public string $output,
        public int $iterations,
        public float $finalScore,
    ) {}
}

interface LoggerInterface
{
    public function info(string $message, array $context = []): void;
    public function warning(string $message, array $context = []): void;
}

// [step] Implement the Optimizer agent with LLM integration
final class OptimizerAgent
{
    public function __construct(
        private readonly OptimizerConfig $config,
        private readonly LoggerInterface $logger,
    ) {}

    public function generate(string $task): string
    {
        $this->logger->info('Generating initial output', ['task' => $task]);
        return $this->callLLM("Generate: {$task}");
    }

    public function refine(string $current, string $critique): string
    {
        $prompt = implode("\n", [
            'Improve the following output based on feedback.',
            "\nCurrent output:\n{$current}",
            "\nFeedback:\n{$critique}",
            '\nProvide the improved version:',
        ]);

        $this->logger->info('Refining output', ['promptLength' => strlen($prompt)]);
        return $this->callLLM($prompt);
    }

    private function callLLM(string $prompt): string
    {
        // Replace with actual LLM API call (e.g., cURL to OpenAI/Anthropic)
        return sprintf('[LLM Response for: %s...]', substr($prompt, 0, 50));
    }
}

// [step] Implement the Evaluator agent with rubric-based scoring
final class EvaluatorAgent
{
    public function __construct(
        private readonly EvaluatorConfig $config,
        private readonly LoggerInterface $logger,
    ) {}

    public function check(string $output): Feedback
    {
        $prompt = implode("\n", [
            "Evaluate against rubric: {$this->config->rubric}",
            "\nOutput:\n{$output}",
            '\nRespond with JSON: { isPass, critique, score, categories }',
        ]);

        $raw = $this->callLLM($prompt);

        /** @var array{isPass: bool, critique: string, score: float, categories: array<string, float>} $data */
        $data = json_decode($raw, true, flags: JSON_THROW_ON_ERROR);

        return new Feedback(
            isPass: $data['isPass'],
            critique: $data['critique'],
            score: (float) $data['score'],
            categories: $data['categories'] ?? [],
        );
    }

    private function callLLM(string $prompt): string
    {
        return json_encode([
            'isPass' => true,
            'critique' => 'Meets all criteria',
            'score' => 95,
            'categories' => ['accuracy' => 96, 'completeness' => 94],
        ], JSON_THROW_ON_ERROR);
    }
}

// [step] Implement the refinement loop with cancellation and iteration tracking
function refinementLoop(
    string $task,
    OptimizerAgent $optimizer,
    EvaluatorAgent $evaluator,
    int $maxIterations = 5,
    ?callable $onIteration = null,
): RefinementResult {
    $currentOutput = $optimizer->generate($task);

    for ($i = 0; $i < $maxIterations; $i++) {
        $feedback = $evaluator->check($currentOutput);

        if ($onIteration !== null) {
            $onIteration($i + 1, $feedback);
        }

        if ($feedback->isPass) {
            return new RefinementResult(
                output: $currentOutput,
                iterations: $i + 1,
                finalScore: $feedback->score,
            );
        }

        $currentOutput = $optimizer->refine($currentOutput, $feedback->critique);
    }

    $finalFeedback = $evaluator->check($currentOutput);

    return new RefinementResult(
        output: $currentOutput,
        iterations: $maxIterations,
        finalScore: $finalFeedback->score,
    );
}

Evaluator-Optimizer Agent Pattern Architecture

hourglass_empty

Rendering diagram...

lightbulb

Evaluator-Optimizer Agent Pattern in the Real World

“Think of a student writing an essay (Optimizer) and a teacher grading it with detailed feedback (Evaluator). The student revises based on the red-ink comments and resubmits. This cycle repeats until the essay meets the teacher’s standards—or the deadline (max iterations) is reached.”