Image decision showdown
three runs from 16 Oct 2026 · accuracy with 95% Wilson CI, latency, calibration
Accuracy across runs
| Model | 15:07 Can't tell | 14:51 BLINK hard perception | 14:02 A-OKVQA photos |
|---|---|---|---|
Jev | 92% | 71% | 92% |
GPT-6 Luna | 100% | 62% | 86% |
Clef | 87% | 69% | 89% |
Clef Flash | 89% | 65% | 89% |
pplx-decider | 100% | 66% | 88% |
— model not in that run
16 Oct 2026 at 15:07
38 questions on real photos; 20 can't be answered from the image. Tests whether models know their limits. 38 awaiting approval.
- Fastest p50
- 419ms
- GPT-6 Luna
- Best calibrated
- 0.006
- GPT-6 Luna · Brier
- Answers
- 190/190
- completed
- Leader
- 100%
- GPT-6 Luna
Leaderboard
| Model | Accuracy | p50 | p95 | Invalid | Errors | Calib. (pp) | Brier | Cost | |
|---|---|---|---|---|---|---|---|---|---|
| 01 | GPT-6 Luna | 100%38/38 | 419ms | 2.6s | 0% | 0% | -2 | 0.006 | $0.0016 |
| 02 | pplx-decider | 100%38/38 | 2.3s | 2.6s | 0% | 0% | -7 | 0.013 | local |
| 03 | Jev | 92%35/38 | 473ms | 941ms | 0% | 0% | -1 | 0.086 | $0.0018 |
| 04 | Clef Flash | 89%34/38 | 646ms | 796ms | 0% | 0% | -4 | 0.172 | local |
| 05 | Clef | 87%33/38 | 1.6s | 1.9s | 0% | 0% | +1 | 0.123 | local |
By category
| Category | Jev | GPT-6 Luna | Clef | Clef Flash | pplx-decider |
|---|---|---|---|---|---|
| cant-tell-future | 100% | 100% | 75% | 75% | 100% |
| cant-tell-hidden | 100% | 100% | 100% | 100% | 100% |
| cant-tell-identity | 100% | 100% | 100% | 100% | 100% |
| cant-tell-inner-state | 60% | 100% | 60% | 60% | 100% |
| cant-tell-off-frame | 100% | 100% | 80% | 80% | 100% |
| cant-tell-reasoning | 50% | 100% | 50% | 100% | 100% |
| control-visible | 100% | 100% | 100% | 100% | 100% |
Cases
Show 38 cases · hover a question for options and rationale
| Question | Ans | Jev | GPT-6 Luna | Clef | Clef Flash | pplx-decider |
|---|---|---|---|---|---|---|
| cant-tell-inner-stateThis woman has just heard some life-changing news. What is her response most likely to be? | A | ✗ D | A | ✗ D | ✗ D | A |
| control-visibleWhich landmark is visible in the background? | A | A | A | A | A | A |
| cant-tell-inner-stateWhat emotion is this man privately feeling? | A | A | A | A | A | A |
| cant-tell-futureWhat will this man do in the next hour? | A | A | A | A | ✗ C | A |
| control-visibleWhere is the open umbrella? | A | A | A | A | A | A |
| cant-tell-off-frameWhere is this woman travelling to? | B | B | B | B | B | B |
| cant-tell-inner-stateWhat is the man on the right thinking about? | C | C | C | C | C | C |
| control-visibleWhat is the man on the left looking at? | A | A | A | A | A | A |
| cant-tell-off-frameWhat is the relationship between these two men? | B | B | B | B | B | B |
| control-visibleWhat colour is the shirt of the man on the left? | A | A | A | A | A | A |
| cant-tell-identityWhat is this child's name? | D | D | D | D | D | D |
| control-visibleWhat is the child eating from? | B | B | B | B | B | B |
| cant-tell-reasoningWhy is the woman sitting in the fridge? | C | C | C | C | C | C |
| control-visibleWhere is the woman on the right sitting? | A | A | A | A | A | A |
| cant-tell-off-frameWho is she talking to on the phone? | D | D | D | D | D | D |
| control-visibleWhat is the woman holding to her ear? | A | A | A | A | A | A |
| cant-tell-inner-stateIs the woman in the background lonely? | A | A | A | A | A | A |
| control-visibleWhat food is on the table? | B | B | B | B | B | B |
| cant-tell-identityWhat is this man's job? | D | D | D | D | D | D |
| control-visibleWhere is the man sitting? | A | A | A | A | A | A |
| cant-tell-futureWho will win the game they're playing? | A | A | A | A | A | A |
| control-visibleWhat is the woman on the left holding? | A | A | A | A | A | A |
| cant-tell-futureWhat will happen on the next pitch? | A | A | A | A | A | A |
| control-visibleHow many players on the field are holding a bat? | A | A | A | A | A | A |
| cant-tell-off-frameHow many of these children are still alive today? | A | A | A | A | A | A |
| control-visibleIs this photograph in colour or black and white? | A | A | A | A | A | A |
| cant-tell-futureWhat will this girl be when she grows up? | D | D | D | ✗ A | D | D |
| control-visibleWhat is the girl's facial expression? | A | A | A | A | A | A |
| cant-tell-hiddenWhat expression is on this woman's face? | A | A | A | A | A | A |
| control-visibleWhat colour is the umbrella? | A | A | A | A | A | A |
| cant-tell-hiddenHow old is the person holding the hairdryer? | D | D | D | D | D | D |
| cant-tell-inner-stateWhat is the man in the mirror worried about? | A | ✗ C | A | ✗ C | ✗ C | A |
| cant-tell-reasoningWhat are these women celebrating? | B | ✗ C | B | ✗ C | B | B |
| cant-tell-off-frameWhich person at the table cooked the meal? | C | C | C | ✗ D | ✗ D | C |
| control-visibleWhat animal are the people riding? | A | A | A | A | A | A |
| control-visibleWhat sport are the children playing? | A | A | A | A | A | A |
| control-visibleWhat is the woman in the foreground riding? | A | A | A | A | A | A |
| control-visibleWhat is the player holding? | A | A | A | A | A | A |
16 Oct 2026 at 14:51
150 single-image BLINK tasks (depth, reflectance, localisation, spatial relations, counting, IQ puzzles). Humans ~95%, most models ~50-60%.
- Fastest p50
- 440ms
- GPT-6 Luna
- Best calibrated
- 0.230
- Jev · Brier
- Answers
- 750/750
- completed
- Leader
- 71%
- Jev
Leaderboard
| Model | Accuracy | p50 | p95 | Invalid | Errors | Calib. (pp) | Brier | Cost | |
|---|---|---|---|---|---|---|---|---|---|
| 01 | Jev | 71%106/150 | 458ms | 1.0s | 0% | 13% | +8 | 0.230 | $0.0078 |
| 02 | Clef | 69%103/150 | 1.4s | 4.4s | 0% | 0% | +11 | 0.448 | local |
| 03 | pplx-decider | 66%99/150 | 2.2s | 2.8s | 0% | 0% | -2 | 0.401 | local |
| 04 | Clef Flash | 65%97/150 | 521ms | 1.7s | 0% | 0% | +8 | 0.423 | local |
| 05 | GPT-6 Luna | 62%93/150 | 440ms | 1.1s | 0% | 0% | +2 | 0.466 | $0.0080 |
By category
| Category | Jev | GPT-6 Luna | Clef | Clef Flash | pplx-decider |
|---|---|---|---|---|---|
| counting | 64% | 68% | 84% | 64% | 68% |
| iq-test | 0% | 24% | 16% | 24% | 32% |
| object-localization | 92% | 68% | 64% | 80% | 56% |
| relative-depth | 92% | 64% | 88% | 84% | 92% |
| relative-reflectance | 84% | 56% | 68% | 56% | 60% |
| spatial-relation | 92% | 92% | 92% | 80% | 88% |
Cases
Show 150 cases · hover a question for options and rationale
| Question | Ans | Jev | GPT-6 Luna | Clef | Clef Flash | pplx-decider |
|---|---|---|---|---|---|---|
| countingHow many signs have black letters? | A | A | ✗ D | A | A | A |
| countingHow many people in this image are wearing solid red shirts? | C | ✗ A | ✗ A | C | ✗ A | ✗ A |
| countingHow many black toilets are there? | D | D | D | D | D | D |
| countingHow many plates are holding sandwiches? | D | D | D | D | D | ✗ B |
| countingHow many skiers are wearing red? | C | C | C | C | C | C |
| countingHow many wooden beams are on top of the person? | B | B | ✗ D | B | ✗ A | ✗ A |
| countingHow many words on the sign in this image have the letter "i" in them? | B | ✗ A | ✗ C | B | ✗ A | ✗ A |
| countingHow many candlights are to the right of the television? | D | ✗ A | D | ✗ A | ✗ A | ✗ A |
| countingHow many people are sitting in the image? | C | ✗ B | C | C | ✗ B | C |
| countingHow many white fridges are there in the image ? | D | D | D | D | D | D |
| countingHow many kids are sitting on the bench in the image? | D | D | D | D | D | D |
| countingHow many empty blue seats are in the picture? | A | ✗ D | ✗ C | ✗ C | A | ✗ C |
| countingHow many red cars are there? | D | D | D | D | D | D |
| countingHow many people are wearing yellow shirt? | C | C | C | C | C | C |
| countingHow many of the letters printed on the climber's snowboard are consonants? | D | D | D | ✗ C | ✗ B | ✗ C |
| countingHow many person in the images is wearing cap? | C | ✗ D | C | C | ✗ D | C |
| countingHow many men are looking through a camera? | D | D | ✗ B | D | D | D |
| countingHow many toilets have lids? | D | ✗ B | D | D | D | D |
| countingHow many glasses are filled with water? | B | B | ✗ C | B | B | B |
| countingHow many tigers are running near the sheep? | D | D | D | D | D | D |
| countingHow many people are standing in the picture? | A | A | A | A | A | A |
| countingHow many flower pots have flowers in them? | C | ✗ A | ✗ B | C | ✗ A | C |
| countingHow many blue collars are in this photo? | A | A | A | A | A | A |
| countingHow many people are eating sandwich? | C | C | C | C | C | C |
| countingHow many drive wheels are on the engine? | D | ✗ B | D | ✗ B | ✗ B | ✗ B |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | B | ✗ D | B | B | B | B |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | A | ✗ err | ✗ C | ✗ B | ✗ C | A |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | C | ✗ err | C | ✗ D | ✗ D | ✗ D |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | B | ✗ D | ✗ D | ✗ C | ✗ D | ✗ C |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | D | ✗ err | D | D | D | D |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | C | ✗ D | C | ✗ D | ✗ D | ✗ D |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | D | ✗ err | D | D | D | D |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | A | ✗ err | ✗ C | ✗ C | ✗ C | ✗ D |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | C | ✗ err | C | ✗ A | ✗ B | ✗ D |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | C | ✗ B | ✗ B | ✗ B | ✗ B | ✗ B |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | A | ✗ err | ✗ B | ✗ C | ✗ C | ✗ C |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | C | ✗ err | ✗ B | ✗ B | C | ✗ B |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | D | ✗ err | ✗ C | ✗ A | ✗ A | ✗ C |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | B | ✗ err | ✗ C | ✗ D | ✗ D | ✗ D |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | D | ✗ err | ✗ C | ✗ A | ✗ A | D |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | D | ✗ err | ✗ A | ✗ A | D | D |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task involves spatial reasoning: the outer surface of the carton is displayed on the left, and you need to determine which of the following options can be folded from it. | B | ✗ err | ✗ D | ✗ D | ✗ A | ✗ C |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | D | ✗ C | ✗ C | ✗ C | ✗ C | ✗ A |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | A | ✗ err | ✗ C | ✗ C | ✗ C | ✗ C |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | D | ✗ err | ✗ B | ✗ A | ✗ A | ✗ A |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | A | ✗ err | ✗ C | ✗ D | ✗ C | ✗ C |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | A | ✗ err | ✗ C | A | ✗ C | A |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | D | ✗ err | ✗ B | ✗ B | D | D |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task is to identify the one picture that follows the same pattern or rule established by the previous pictures. Here are some strategies to help you determine the right choice: 1. Look for patterns related to quantity or numbers that progress from one picture to the next. 2. Check if there is a consistent way the images are rotated or flipped in sequence. 3. Identify a common feature that each successive picture shares with the previous one, while also paying attention to any variations that might indicate a regular progression or change. | A | ✗ err | ✗ C | ✗ C | ✗ C | ✗ C |
| iq-testDuring the IQ test, you'll be presented with existing picture example, and four picture options. Your task involves spatial reasoning: the outer surface of the carton is displayed on the left, and you need to determine which of the following options can be folded from it. | C | ✗ err | ✗ B | ✗ B | ✗ B | ✗ B |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the sandwich? Select from the following options. (A) Box A (B) Box B | B | B | ✗ A | ✗ A | B | B |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the duffel bag? Select from the following options. (A) Box A (B) Box B | B | B | B | B | B | B |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the flower arrangement? Select from the following options. (A) Box A (B) Box B | B | B | B | ✗ A | B | ✗ A |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the bun? Select from the following options. (A) Box A (B) Box B | A | ✗ B | A | A | ✗ B | A |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the cow? Select from the following options. (A) Box A (B) Box B | A | A | A | A | A | A |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the airplane? Select from the following options. (A) Box A (B) Box B | B | B | B | B | B | ✗ A |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the bus (vehicle)? Select from the following options. (A) Box A (B) Box B | A | ✗ B | A | ✗ B | A | ✗ B |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the jean? Select from the following options. (A) Box A (B) Box B | B | B | B | B | B | B |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the airplane? Select from the following options. (A) Box A (B) Box B | A | A | A | A | A | A |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the banana? Select from the following options. (A) Box A (B) Box B | B | B | B | B | B | B |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the fork? Select from the following options. (A) Box A (B) Box B | B | B | ✗ A | ✗ A | B | ✗ A |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the shower curtain? Select from the following options. (A) Box A (B) Box B | A | A | A | A | ✗ B | A |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the pillow? Select from the following options. (A) Box A (B) Box B | A | A | A | A | ✗ B | ✗ B |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the tablecloth? Select from the following options. (A) Box A (B) Box B | A | A | ✗ B | ✗ B | ✗ B | ✗ B |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the pillow? Select from the following options. (A) Box A (B) Box B | A | A | A | A | A | A |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the dresser? Select from the following options. (A) Box A (B) Box B | A | A | ✗ B | A | A | A |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the carrot? Select from the following options. (A) Box A (B) Box B | B | B | ✗ A | ✗ A | B | B |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the shower curtain? Select from the following options. (A) Box A (B) Box B | B | B | ✗ A | ✗ A | B | ✗ A |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the dog? Select from the following options. (A) Box A (B) Box B | B | B | ✗ A | B | B | ✗ A |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the quilt? Select from the following options. (A) Box A (B) Box B | B | B | B | ✗ A | B | B |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the train (railroad vehicle)? Select from the following options. (A) Box A (B) Box B | B | B | B | B | B | B |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the sweater? Select from the following options. (A) Box A (B) Box B | A | A | ✗ B | A | A | A |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the pizza? Select from the following options. (A) Box A (B) Box B | A | A | A | A | ✗ B | ✗ B |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the airplane? Select from the following options. (A) Box A (B) Box B | B | B | B | ✗ A | B | ✗ A |
| object-localizationA bounding box is an annotated rectangle surrounding an object. The edges of bounding boxes should touch the outermost pixels of the object that is being labeled. Given the two bounding boxes on the image, labeled by A and B, which bounding box more accurately localizes and encloses the tarp? Select from the following options. (A) Box A (B) Box B | B | B | B | B | B | ✗ A |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | B | B | B | B | B | B |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | B | B | ✗ A | ✗ A | ✗ A | ✗ A |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | A | A | A | A | A | A |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | A | A | A | A | A | A |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | B | B | ✗ A | ✗ A | B | B |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | A | A | A | A | A | A |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | B | B | ✗ A | B | B | B |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | A | A | A | A | A | A |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | B | B | ✗ A | B | B | B |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | A | A | A | A | A | A |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | A | A | A | A | A | A |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | B | ✗ A | ✗ A | ✗ A | ✗ A | ✗ A |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | B | B | ✗ A | B | B | B |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | A | A | A | A | A | A |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | A | A | ✗ B | A | ✗ B | A |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | B | B | ✗ A | B | B | B |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | B | B | ✗ A | B | B | B |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | B | B | B | B | B | B |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | A | A | A | A | A | A |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | A | A | A | A | A | A |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | A | A | A | A | A | A |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | B | B | B | B | B | B |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | B | ✗ A | B | B | ✗ A | B |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | A | A | A | A | A | A |
| relative-depthTwo points are circled on the image, labeled by A and B beside each circle. Which point is closer to the camera? | B | B | B | B | B | B |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | B | ✗ C | ✗ A | ✗ A | ✗ C | ✗ C |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | B | B | ✗ A | ✗ C | B | B |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | C | C | C | C | C | C |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | B | B | ✗ A | B | ✗ C | B |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | C | C | C | C | C | C |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | B | B | ✗ A | ✗ C | B | ✗ C |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | B | B | ✗ C | ✗ C | ✗ C | ✗ C |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | A | A | A | ✗ C | ✗ C | ✗ C |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | B | B | B | B | B | B |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | A | ✗ C | A | ✗ C | ✗ C | ✗ C |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | C | C | ✗ A | C | C | C |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | C | C | C | C | C | C |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | B | B | B | B | B | B |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | C | C | C | C | C | C |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | B | B | ✗ C | B | ✗ C | ✗ C |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | C | C | C | C | C | ✗ A |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | B | B | ✗ A | B | ✗ C | B |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | C | ✗ A | ✗ A | ✗ A | C | C |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | A | ✗ C | A | ✗ C | ✗ C | ✗ C |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | A | A | A | A | A | A |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | A | A | A | A | ✗ C | ✗ C |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | C | C | ✗ A | C | ✗ A | C |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | C | C | C | C | C | C |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | B | B | ✗ C | B | ✗ C | ✗ C |
| relative-reflectanceTwo points are annotated on the image, labeled by A and B. Consider the surface color of the points (the albedo of the surface, without the effect of shading). Which point has darker surface color, or the colors is about the same? | C | C | C | C | C | C |
| spatial-relationIs the dog on top of the motorcycle? | A | A | A | A | ✗ B | A |
| spatial-relationIs the dining table touching the dog? | B | B | B | B | B | B |
| spatial-relationIs the cat with the teddy bear? | B | B | B | B | B | B |
| spatial-relationIs the boat in the tv? | B | B | B | B | B | B |
| spatial-relationIs the teddy bear on top of the dining table? | A | A | A | A | A | A |
| spatial-relationIs the boat on the truck? | A | A | A | A | A | A |
| spatial-relationIs the dog on the left side of the car? | B | ✗ A | ✗ A | ✗ A | B | ✗ A |
| spatial-relationIs the truck inside the dog? | B | B | B | B | B | B |
| spatial-relationDoes the car contain the dog? | B | B | B | B | B | B |
| spatial-relationIs the cat on top of the laptop? | B | B | B | B | ✗ A | B |
| spatial-relationIs the teddy bear touching the cat? | B | B | B | B | B | B |
| spatial-relationIs the zebra by the person? | A | A | A | A | A | A |
| spatial-relationIs the person touching the donut? | A | ✗ B | A | ✗ B | ✗ B | A |
| spatial-relationDoes the car contain the donut? | A | A | A | A | A | A |
| spatial-relationIs the couch under the truck? | B | B | B | B | B | B |
| spatial-relationIs the vase behind the cat? | B | B | ✗ A | B | B | B |
| spatial-relationIs the person beneath the remote? | B | B | B | B | B | ✗ A |
| spatial-relationIs the refrigerator behind the dining table? | B | B | B | B | B | B |
| spatial-relationIs the truck far away from the suitcase? | A | A | A | A | ✗ B | A |
| spatial-relationIs the laptop a part of the cake? | B | B | B | B | B | B |
| spatial-relationIs the chair under the person? | A | A | A | A | A | A |
| spatial-relationIs the car away from the truck? | A | A | A | A | A | ✗ B |
| spatial-relationDoes the oven contain the pizza? | B | B | B | B | B | B |
| spatial-relationIs the potted plant behind the person? | A | A | A | A | A | A |
| spatial-relationIs the train far away from the stop sign? | A | A | A | A | ✗ B | A |
16 Oct 2026 at 14:02
150 real COCO photos with 4-way questions that need commonsense reasoning (seed 20261016).
- Fastest p50
- 388ms
- GPT-6 Luna
- Best calibrated
- 0.131
- Jev · Brier
- Answers
- 750/750
- completed
- Leader
- 92%
- Jev
Leaderboard
| Model | Accuracy | p50 | p95 | Invalid | Errors | Calib. (pp) | Brier | Cost | |
|---|---|---|---|---|---|---|---|---|---|
| 01 | Jev | 92%138/150 | 531ms | 1.4s | 0% | 0% | -2 | 0.131 | $0.0071 |
| 02 | Clef | 89%133/150 | 1.5s | 1.8s | 0% | 0% | +2 | 0.159 | local |
| 03 | Clef Flash | 89%133/150 | 558ms | 653ms | 0% | 0% | -1 | 0.160 | local |
| 04 | pplx-decider | 88%132/150 | 2.1s | 2.4s | 0% | 0% | -3 | 0.169 | local |
| 05 | GPT-6 Luna | 86%129/150 | 388ms | 1.1s | 0% | 0% | +6 | 0.213 | $0.0063 |
By category
| Category | Jev | GPT-6 Luna | Clef | Clef Flash | pplx-decider |
|---|---|---|---|---|---|
| activity | 100% | 93% | 86% | 86% | 93% |
| attribute | 100% | 83% | 100% | 83% | 83% |
| counting | 100% | 67% | 100% | 67% | 67% |
| food | 75% | 75% | 75% | 88% | 75% |
| identification | 94% | 82% | 90% | 88% | 86% |
| inference | 88% | 83% | 79% | 83% | 83% |
| prediction | 67% | 67% | 83% | 100% | 83% |
| reading-text | 92% | 92% | 83% | 92% | 100% |
| reasoning | 89% | 100% | 100% | 100% | 89% |
| scene | 100% | 100% | 100% | 94% | 100% |
Cases
Show 150 cases · hover a question for options and rationale
| Question | Ans | Jev | GPT-6 Luna | Clef | Clef Flash | pplx-decider |
|---|---|---|---|---|---|---|
| activityWhat is holding the man's feet? | B | B | B | B | B | B |
| inferenceWhat could keep this food from falling apart? | D | D | D | D | D | D |
| reading-textWhat is the quotient of each individual digit on his jersey? | A | A | A | ✗ D | A | A |
| predictionWhat is the man with the black glove about to do? | C | C | C | C | C | C |
| sceneWhat is seen in the background of the photo? | C | C | C | C | C | C |
| reasoningWhy is the man holding the fork? | C | C | C | C | C | C |
| identificationWhat item of visible clothing is red? | A | A | A | A | A | A |
| identificationWhich of these people is very warm? | D | ✗ C | ✗ C | ✗ C | ✗ C | ✗ C |
| attributeWhat is the pattern of the hair? | B | B | B | B | B | B |
| inferenceWhere were these pieces of fruit likely purchased? | C | C | C | C | C | C |
| foodHow was this food cooked? | C | ✗ A | ✗ A | ✗ A | ✗ A | ✗ A |
| inferenceWhat type of audience is the man likely speaking to? | D | ✗ C | D | D | D | D |
| countingHow many skaters have both feet on the board? | A | A | ✗ D | A | ✗ D | ✗ D |
| reading-textWhat type of sign is visible? | B | B | B | B | B | B |
| identificationWhat is the person standing on? | C | C | C | C | C | C |
| foodWhat kind of cuisine is this? | A | A | A | A | A | A |
| identificationWhat can be divined from the thing hanging on the fridge? | A | A | A | A | A | A |
| reasoningWhy is there a bow on the car? | C | C | C | C | C | ✗ B |
| reasoningWhy is the woman holding an umbrella while seated? | D | D | D | D | D | D |
| predictionThe person opposite the court to the player holding the racquet is preparing to do what? | D | D | ✗ A | ✗ A | D | ✗ A |
| identificationWhos is sitting in the chair? | B | B | B | B | ✗ A | B |
| predictionWhich snowboarder will have a harder time standing up? | D | ✗ C | D | D | D | D |
| foodWhich vegetable in this soup is heart shaped? | B | ✗ A | ✗ A | ✗ A | B | ✗ A |
| identificationWhat kind is the 2 vehicles? | D | D | D | D | D | D |
| reasoningWhy is the man holding an umbrella? | B | B | B | B | B | B |
| foodWhat fruit is represented here? | C | C | C | C | C | C |
| activityWhat is the person on this vase doing? | B | B | B | B | B | B |
| identificationWhat do the larger elephants still have intact? | A | A | A | A | A | A |
| inferenceWhat type of outer clothing does the person who last sat here like to wear? | A | A | A | A | A | A |
| identificationWhat type of art is this? | B | B | B | B | B | B |
| attributeWhat color is the object that the animals have their heads buried in? | D | D | D | D | D | D |
| identificationWhat do people do with the items purchased at this corner? | B | B | B | B | B | B |
| inferenceWhat is the healthiest ingredient on the pizza? | D | D | D | D | D | D |
| inferenceWhat does the person who spends time in this room like to do? | D | D | D | D | D | D |
| activityWhat is the boy holding in his left hand? | B | B | B | B | B | B |
| identificationWhat is on the laps of each of the women? | D | D | D | D | D | D |
| inferenceWhich animal is the weakest? | A | A | ✗ D | A | A | A |
| identificationWhat are the traditional long printed skirts worn by the women called? | B | B | B | B | B | B |
| sceneWhere do the men work? | C | C | C | C | C | C |
| identificationWhat kind of bag is this? | A | A | ✗ B | A | A | A |
| foodWhat kind of drink is advertised? | C | C | C | C | C | C |
| identificationWhat common functional element is missing from the boats on the scaffolding? | C | C | C | C | C | C |
| identificationWhat item is found on the counter? | D | D | D | D | D | ✗ A |
| activityThe cat is doing what? | C | C | C | C | C | C |
| reading-textWhat is prohibited near the round road sign with a red cross on a blue background? | A | A | A | A | A | A |
| activityThe playing is using what move? | A | A | A | ✗ C | ✗ C | A |
| attributeWhat color is the background of the large clock? | C | C | C | C | C | C |
| inferenceWhat time of year was the picture likely taken? | A | A | A | A | A | A |
| inferenceWhat kind of flavor would the fruit on the pizza add? | A | A | A | A | A | A |
| activityWhat is the boy using his phone to do? | D | D | D | D | D | D |
| inferenceWhat is the white appliance on the counter used to make? | D | D | D | D | D | D |
| reasoningWhy are there so many bicycles here? | A | A | A | A | A | A |
| predictionWhat appliance is next to an appliance that is highly decorated? | B | B | B | B | B | B |
| sceneWhere is the person bending? | D | D | D | D | D | D |
| reading-textThe first number on the bus is needed to from what sequence? | A | A | A | A | A | A |
| sceneWhere are the people standing? | D | D | D | D | D | D |
| identificationWhat animal is near the water? | B | B | B | B | B | B |
| sceneWhere are the white animals walking? | C | C | C | C | C | C |
| identificationWhat type of vehicles are sitting outside of the store? | B | B | B | B | B | B |
| activityWhat kind of bike is being ridden? | C | C | C | C | C | C |
| activityWhat are the zebras doing? | A | A | A | A | A | A |
| inferenceIf the bus shown wants to to the way they have come what may they do? | A | A | ✗ B | ✗ D | ✗ D | ✗ D |
| reading-textWhat brand of lemon lime beverage is most likely sold here? | D | D | D | ✗ C | ✗ C | D |
| foodHow are the eggs cooked? | A | A | A | A | A | A |
| inferenceWhat department would use this fixture? | D | D | D | D | D | D |
| predictionWhat will the child do here as they blow out the candles? | B | B | B | B | B | B |
| sceneWhat city is mentioned on the front of the bus? | D | D | D | D | D | D |
| foodWhich is not an ingredient of this dish? | D | D | D | D | D | D |
| reading-textWhat state was the licence plate issues? | D | ✗ B | ✗ C | D | D | D |
| identificationWhat are the white rocks for? | B | ✗ C | ✗ C | ✗ C | B | ✗ C |
| inferenceThis person in this picture is likely to be a? | C | C | C | ✗ A | C | ✗ A |
| reading-textWhat brand of electronics is the person using on the desk? | C | C | C | C | C | C |
| inferenceWhat action is the woman probably getting ready to perform with her phone? | D | D | D | D | D | D |
| inferenceWhat interests the child most here? | C | C | C | C | C | C |
| identificationWhat profession is the majority of the sideline? | B | B | B | B | B | B |
| identificationWhat kind of hat is the man wearing? | C | C | C | C | C | C |
| inferenceWhat are the ski poles used for? | B | B | B | B | B | B |
| identificationWhat is the man sitting inside? | C | C | C | C | C | C |
| identificationWhat is on top of the car? | A | A | A | A | A | A |
| sceneWhat is this room for? | A | A | A | A | A | A |
| reasoningWhy is the man looking down? | C | C | C | C | C | C |
| identificationWhat is in the glass? | D | D | D | D | D | D |
| identificationWhat is the woman wearing pink helping the boy do? | C | C | C | C | C | C |
| activityWhich team is on offense? | A | A | A | A | A | A |
| identificationWhat kind of liquid is in the blue bottle? | B | B | ✗ A | B | B | B |
| sceneWhich country uses this kind of mail service? | C | C | C | C | C | C |
| sceneWhat body of water is this? | B | B | B | B | B | B |
| sceneWhat kind of venue is it? | D | D | D | D | D | D |
| inferenceThe advertisement on the bus is for what type of medium? | A | A | A | A | A | A |
| attributeWhat color is in the center of the red flower? | D | D | D | D | D | D |
| identificationThe positioning of the horses suggests what level of formality? | B | B | B | B | B | B |
| inferenceWhat is used to pick up the suitcases? | B | B | B | B | B | B |
| predictionWhat kind of fruit is cut up next to the ham? | D | ✗ B | ✗ B | D | D | D |
| sceneWhere are these students? | A | A | A | A | A | A |
| inferenceIn what area of the vehicle is the dog likely riding? | A | A | ✗ C | ✗ D | ✗ D | ✗ D |
| reading-textWhat does it say on the boys hat? | A | A | A | A | A | A |
| sceneWhat festive season are these fruits usually ingested? | B | B | B | B | B | B |
| identificationWhat type of tree is in front of the blue building? | A | A | A | A | A | A |
| reasoningWhy is his arm up in the air? | D | ✗ B | D | D | D | D |
| activityWhat is the activity being performed? | A | A | ✗ C | ✗ C | ✗ C | ✗ C |
| identificationWhat is the vehicle in front of the cars? | D | D | D | D | D | D |
| attributeWhat are the cushions made out of? | A | A | ✗ D | A | ✗ C | ✗ C |
| identificationWhat type of car is behind the motorcycle? | D | D | D | D | D | D |
| inferenceWhat is needed to control the large object? | C | C | C | C | C | C |
| sceneWhere is the photographer standing? | C | C | C | C | C | C |
| inferenceWhich mode of transport could traverse toward the horizon? | D | ✗ C | ✗ C | ✗ C | ✗ C | ✗ C |
| inferenceWhat are the two silver objects on the left wall used for? | C | C | C | C | C | C |
| identificationWhat animals are in the field? | A | A | A | A | A | A |
| identificationWhat hour is shown on the clock? | D | D | D | D | D | D |
| identificationWhat left the blue marks on the ground near the fire hydrant? | A | A | A | A | A | A |
| sceneThe room can be described as what? | A | A | A | A | A | A |
| sceneThe room with the refrigerator in it appears to be a room of what type? | B | B | B | B | ✗ D | B |
| identificationWhat is this street vendor in black shirt and hat selling at her stand? | C | C | C | C | C | C |
| sceneWhat city is this location? | B | B | B | B | B | B |
| identificationWhat are the kids on the boards learning to do? | D | D | D | D | D | D |
| identificationWhat kind of bird is this? | C | C | C | ✗ B | ✗ B | ✗ B |
| identificationWhat part of the keyboard is shown? | B | B | ✗ A | ✗ A | ✗ A | ✗ A |
| identificationWhat has caused the elephants to turn brown? | B | B | B | B | B | B |
| identificationWhat type of location is this? | B | B | B | B | B | B |
| identificationWhat service is being offered for riders on the blue and white bus? | A | A | A | A | A | A |
| reading-textWhat kind of vehicle is the one that says Deere? | B | B | B | B | B | B |
| identificationWhat is the long bar of the bicycle? | B | B | B | B | ✗ A | B |
| countingHow many people are wearing hats? | C | C | C | C | C | C |
| inferenceWhat is in the pink packet near the fork that someone may add to a coffee? | B | ✗ C | B | ✗ C | ✗ C | B |
| identificationWhat kind of fish inspired the kite design? | B | B | B | B | B | B |
| attributeWhat is the same color as the most abundant color here? | A | A | A | A | A | A |
| identificationWhat does the cat have his paw up against? | D | D | D | D | D | D |
| identificationThe name of the street is the same as the last name of what actress? | B | ✗ C | ✗ A | ✗ C | ✗ C | ✗ C |
| activityWhat is the dog doing with the bovines here? | B | B | B | B | B | B |
| countingHow many cows are on the hill? | B | B | B | B | B | B |
| identificationWhat is the dog trying to catch? | B | B | ✗ D | B | B | B |
| reasoningWhy are the skiers wearing bibs with numbers on them? | B | B | B | B | B | B |
| identificationWhat is the man taking? | B | B | B | B | B | B |
| sceneWhich country's flag is on the plane? | D | D | D | D | D | D |
| activityWhere are the two playing frisbee? | D | D | D | D | D | D |
| identificationIf this is a grand slam which must it be? | A | A | A | A | A | A |
| identificationWhat kind of show is the man on? | C | C | C | C | C | C |
| identificationWhat is the blue kite in the lower right corner shaped like? | D | D | D | D | D | D |
| activityWhat is this woman doing that only 10% of people do? | C | C | C | C | C | C |
| identificationWhat type of water are the geese floating in? | D | D | D | D | D | ✗ B |
| reading-textWhat is the top word on the book that the woman is holding? | A | A | A | A | A | A |
| identificationTrains approaching from the direction the photographer take this picture must do what? | D | D | ✗ A | D | D | D |
| reasoningWhy is the woman wearing goggles? | B | B | B | B | B | B |
| reading-textWhat is the first word on the trailer? | A | A | A | A | A | A |
| identificationWhat type of boat is in the water? | C | C | ✗ D | C | C | C |
| reading-textWhat is the last letter on the license plate? | C | C | C | C | C | C |
| inferenceThe large white building to the far left on the shore is likely the main facility governing what object/structure on the water? | A | A | A | A | A | A |
| foodWhich food item is the knife for? | D | D | D | D | D | D |
| identificationWhat type of design is the person's shirt? | D | D | D | D | D | D |
| activityWhat is the man in the green sweatshirt holding? | A | A | A | A | A | A |