The LLM ranker acts as an objective referee by evaluating candidate snippets or full pages across N samples using a chosen judge model, recording both the median rank and structured rationales for why items placed higher or lower than competitors.
sequenceDiagram
autonumber
participant Cycle as Optimization Loop
participant Ranker as LLM Ranker
participant Judge as Judge Model (LLM)
participant Agg as Rank Aggregator
Cycle->>Ranker: Send variant + competitor set
par N Parallel Samples
Ranker->>Judge: Prompt pairwise/listwise evaluation
Judge-->>Ranker: Return rank order + rationales
end
Ranker->>Agg: Aggregate N evaluation samples
Agg-->>Ranker: Compute median rank + structured rationales
Ranker-->>Cycle: Return ranking trace + posterior update