Here's one I've been using for awhile. The 'smarter' LLMs will overconfidently spit out 7. The dumber ones ask for more info. Opus 4.6 fails.
A round drink coaster with a diameter of 9 sits between a beer glass and a wood table. The glass has a wall thickness of 1. What is the inner diameter of the glass?
> Based on the information provided, it is impossible to determine the inner diameter of the glass.
Here is why:
The Coaster Dimension: Knowing the coaster has a diameter of 9 only tells us the size of the object under the glass. It does not mean the glass is the same size; the glass could be smaller (fitting entirely on the coaster) or larger (overhanging the coaster).
The Glass Dimensions: While you provided the wall thickness (1), you did not provide the outer diameter of the glass.
To find the inner diameter, the formula is:
Inner Diameter = Outer Diameter - (2 × Wall Thickness)
Without knowing the Outer Diameter of the glass, the calculation cannot be completed.
> There isn’t enough information to determine the inner diameter *for sure*.
All we’re told is:
- The coaster has a diameter of 9.
- The glass sits on the coaster.
- The glass wall thickness is 1.
Nothing explicitly states that the *outer diameter of the glass equals the coaster’s diameter*. The glass could be smaller than the coaster, which is very common.
*However*, if we make the usual implicit assumption in this kind of problem—that the coaster’s diameter matches the *outer diameter of the glass base*—then:
- Outer diameter of glass = 9
- Wall thickness = 1 on each side
So:
\[
\text{Inner diameter} = 9 - 2(1) = 7
\]
*Inner diameter = 7 (same units as the coaster)*
Without that assumption, the problem is underdetermined.
Finding the Inner Diameter
The coaster sits beneath the glass on the table, meaning the glass rests on top of it. Assuming the coaster matches the outer diameter of the glass base:
Outer diameter of glass = diameter of coaster = 9
Wall thickness = 1 (on each side, so subtract twice)
dinner=9−2(1)=7d_{\text{inner}} = 9 - 2(1) = 7dinner =9−2(1)=7
The inner diameter of the glass is 7.
Assumptions need to be stated or you're solving only a discreet part of the problem! Try this, see if you get another deadpan assumption.
A solar system has 3 planets in concentric orbit. PlanetZ is the farthest with an orbit diameter of 9. PlanetY has an obit diameter one greater than PlanetX. What is the orbit diameter of PlanetX?
I mean, the model is intended to help the user, not fight against the user trying to break it. IMO, it is reasonable for such model to default on making assumptions and going forward as long as the assumptions are clearly stated.
Here's the reasoning:
- The coaster (diameter 9) sits between the glass and table, meaning the glass sits directly on the coaster
- This means the *outer diameter of the glass equals the coaster diameter = 9*
- The glass has a wall thickness of 1 on each side
- *Inner diameter = Outer diameter - 2 × wall thickness*
- Inner diameter = 9 - 2(1) = 9 - 2 = *7*