AI Benchmark from Sunday Puzzles
This is a news story, published by TechCrunch, that relates primarily to Will Shortz news.
Will Shortz news
For more Will Shortz news, you can click here:
more Will Shortz newsNews about Ai research
For more Ai research news, you can click here:
more Ai research newsTechCrunch news
For more news from TechCrunch, you can click here:
more news from TechCrunchAbout the Otherweb
Otherweb, Inc is a public benefit corporation, dedicated to improving the quality of news people consume. We are non-partisan, junk-free, and ad-free. We use artificial intelligence (AI) to remove junk from your news feed, and allow you to select the best tech news, business news, entertainment news, and much more. If you like this article about Ai research, you might also like this article about
AI benchmark. We are dedicated to bringing you the highest-quality news, junk-free and ad-free, about your favorite topics. Please come every day to read the latest reasoning effort news, ai news, news about Ai research, and other high-quality news about any topic that interests you. We are working hard to create the best news aggregator on the web, and to put you in control of your news feed - whether you choose to read the latest news through our website, our news app, or our daily newsletter - all free!
reasoning benchmarksTechCrunch
•Technology
Technology
These researchers used NPR Sunday Puzzle questions to benchmark AI 'reasoning' models | TechCrunch

82% Informative
NPR host Will Shortz hosts a weekly crossword puzzle series called the Sunday Puzzle .
Researchers created an AI benchmark using riddles from the show.
They say AI models sometimes “give up” and provide incorrect answers they know aren’t correct.
The current best-performing model on the benchmark is o1 with a score of 59% , followed by the recently released o3-set to high-reasoning models.
VR Score
86
Informative language
87
Neutral language
51
Article tone
informal
Language
English
Language complexity
51
Offensive language
not offensive
Hate speech
not hateful
Attention-grabbing headline
not detected
Known propaganda techniques
not detected
Time-value
long-living
External references
2
Source diversity
2
Affiliate links
no affiliate links