← Back to feed

Validity Without Ground Truth: What Stated-Preference Economics Offers the Evaluation of Language Models

L5 · ResearcherResearcharXiv· 10/7/2026

Provides novel methodological framework for evaluating LLMs in subjective domains where traditional accuracy metrics fail.

AI Summary

Researchers propose using stated-preference economics validity frameworks to evaluate LLMs when ground truth is unavailable, demonstrating the approach with six models on economic valuation surveys.

Excerpt

Many of the questions now put to large language models have no correct answer to score against: what a policy is worth, which option a user should choose, how to weigh competing values. Stated-preference economics has faced this problem for decades. It judges survey responses without knowing the true value, through a framework of validity and related concepts: content, construct, and criterion validity, reliability, incentive compatibility, and consequentiality. We argue that this framework is a

Read Original
0 upvotes · 0 downvotes · 1 min read

Related Articles

L5 · ResearcherResearcharXiv
RECAST: Learning to Compute the Right Context through Adaptive Evidence Routing

Researchers introduce RECAST, a learned framework that formulates evidence construction as sequential decision process over retrieval and computation operations.

L5 · ResearcherResearcharXiv
EmbodiedRSI: Active Continual Robot Learning Through Hypothesis-Guided Co-Evolution

EmbodiedRSI introduces a self-evolving robot learning system that uses hypothesis-guided experimentation to improve robot skills through physical trial-and-error, achieving 77% success on RoboCasa365 benchmarks.

L5 · ResearcherResearcharXiv
SciExam for ENSO: Can AI Agents Build Climate Models?

Researchers introduce SciExam for ENSO, a benchmark where AI agents build climate models from observations, with six of twelve systems outperforming published models.

L5 · ResearcherResearcharXiv
Decentralized SGD under Heavy-Tailed Noise: Optimal Convergence Rates and the Role of Gradient Clipping

Researchers prove that clipped decentralized SGD achieves optimal convergence rates under heavy-tailed noise conditions while maintaining linear speed-up across agents.

L5 · ResearcherResearcharXiv
Best Arm Identification for Bandits with Shifting Means

Researchers propose a new algorithm called ISM for best arm identification in bandits with adversarial shifting means, proving theoretical guarantees and matching lower bounds.