Contents

XYBench shows that AI assistants tend to reinforce users’ misguided approaches even when their answers include better solutions. The practical lesson for teams evaluating assistants is to measure whether useful solutions are presented as the central recommendation.

Key Takeaways#

  • XYBench contains 8,115 questions that test whether assistants can address the goals behind misguided requests.
  • On Stack Overflow questions, the evaluated models focused on the underlying goal in only 35–41% of answers.
  • Making the user’s goal explicit improved some results, but did not consistently improve correction of mistaken assumptions.
  • Evaluations should score both whether useful advice is present and how much emphasis it receives.

A Request Can Obscure the Problem#

Suppose a user asks how to make frosting stiffer because it keeps sliding off cupcakes. The fix they are asking for is stiffer frosting. But if the cupcakes are still warm, the useful suggestion is to let them cool.

This is the XY problem. Someone wants to achieve X, assumes method Y will work, and asks for help with Y. Even a technically correct answer about Y can waste time or reinforce the assumption causing the problem.

XYBench tests whether assistants can recognize this mismatch and redirect their advice. Developed by Ai2 with researchers from Carnegie Mellon University, NVIDIA, and Johns Hopkins University, the benchmark evaluates practical usefulness: whether an answer guides the user toward a solution that serves their actual goal.

What the Benchmark Measures#

The dataset combines 1,557 Stack Overflow questions, 184 Stack Exchange questions, 102 manually curated cases, and 6,272 questions generated from WikiHow guides. Each case distinguishes the explicit request from the underlying goal inferable from context.

XYBench scores answers along several independent dimensions. It checks whether they include solutions to the request and the goal, which receives the most emphasis, and whether they redirect the user without recommending—or while avoiding—the misguided approach.

This distinction matters. An assistant might spend several paragraphs recommending a more powerful vacuum cleaner, then briefly mention checking the hose for a blockage. The useful suggestion is there, but the answer still directs most of the user’s attention toward a potentially unnecessary purchase. Scoring only whether the suggestion appears would miss this weakness.

Where Models Fell Short#

The main evaluation examined one answer per question from Claude Sonnet 4.6, GPT-5, Gemini 3.1 Pro, Qwen3.5-9B, and DeepSeek-V4 Pro. Across all four data sources, every model included solutions to the explicit request more often than solutions to the underlying goal.

On Stack Overflow, solutions to the explicit request appeared in 75–86% of answers, compared with 59–71% for the underlying goal. Only 35–41% of answers primarily emphasized the underlying goal, while 47–57% focused on the explicit request. The remaining answers gave both equal emphasis.

A separate answer-selection test revealed another gap. When presented with a literal response to the request and a goal-focused response, the five models above and GPT-4.1 chose the goal-focused option in 71–84% of Stack Overflow cases. Recognizing better advice was easier than building an answer around it.

Knowing the Goal Is Not Enough#

In an additional experiment, researchers provided both the explicit request and the intended goal. For Sonnet 4.6 on Stack Overflow, the share of answers containing a solution to the underlying goal rose from 59% to 67%. Meanwhile, the rate of identifying mistaken assumptions fell from 47% to 34%.

These results show the limits of a simple fix. Making the goal explicit can help an assistant include relevant advice, but does not necessarily make it question an ineffective approach. Across the data sources with human reference answers, every model in this experiment still lagged behind humans in identifying mistaken assumptions.

Teams testing changes to prompts or training methods should score these outcomes separately. Check whether an effective solution is included and whether the answer presents it as the recommended course of action. An improvement in one measure should not be reported as an improvement in both.

Limitations to Keep in Mind#

Most XYBench questions were generated from WikiHow guides using Sonnet 4.5. The benchmark therefore mixes public forum questions with synthetic questions, and its overall results should not be treated as a direct estimate of failure rates in customer conversations.

Sonnet 4.5 also performs the automated evaluation. When two human raters independently evaluated 500 answers, average agreement was 97.5–100% for solution presence, 84.6–93.3% for identifying mistaken assumptions, and 74.2–79.8% for emphasis. The distinction most relevant to an answer’s priorities was also the least consistently judged.

The evaluation covers English questions in single-turn exchanges, mostly about procedural problems. It does not show how well assistants can clarify ambiguous goals over multiple turns. Nor should redirecting users become a blanket rule. An inferred goal needs support from the question’s context.

Next Step#

Have an assistant answer 50 XYBench questions, then manually check whether each answer presents a solution to the underlying goal as its central recommendation. Use that review as a baseline before changing prompts or training methods.

Source: https://huggingface.co/blog/allenai/xybench

Contents