Synth AI
WorkshopBlogDocsStackSign in
WorkshopBlogDocsStackSign in
← Synth Research/Evals/Craftax
Visible models · 12/12Organization

Open-ended agents in Craftax

Craftax

A long-horizon, open-ended world combining survival, crafting, exploration, combat, and dungeon progression. We evaluate whether language-model agents can turn its achievement ladder into durable capability.

Read the Craftax paper ↗Original environment ↗
Original Craftax dungeon-crawling gameplay
Original Craftax gameplay · dungeon crawling
ParetoAchievementsTrajectories

Cost vs. Performance

Average distinct achievements against average API cost per terminal Craftax rollout. Lines connect each model's measured reasoning efforts in low → medium → high order; larger markers mean more effort.

ModelsGPT-5.4-nanoDeepSeek V4 Flash 0731GPT-5.6-terraGPT-5.6-lunaDeepSeek V4 FlashGemini 3.5 Flash LiteGLM 5.2Gemini 2.5 Flash LiteKimi K2.6Kimi K3DeepSeek V4 ProGemini 3.1 Flash LiteEffortDefaultlowmediumhigh
03.46.710.113.4$0.00$0.09$0.18$0.27$0.37ACHIEVEMENTS ↑INFERENCE COST / ROLLOUT →GPT-5.4-nano · medium: n=20, μ13.4DeepSeek V4 Flash 0731 · xhigh · Jul 31: n=7, μ13.0GPT-5.6-terra · low · Jul 9 launch price: n=19, μ12.7GPT-5.6-terra · low · Jul 30 price-cut snapshot: n=19, μ12.7DeepSeek V4 Flash 0731 · max · Jul 31: n=11, μ12.2GPT-5.6-luna · high · Jul 9 launch price: n=20, μ11.3GPT-5.6-luna · high · Jul 30 price-cut snapshot: n=20, μ11.3GPT-5.6-luna · low · Jul 9 launch price: n=34, μ10.5GPT-5.6-luna · low · Jul 30 price-cut snapshot: n=34, μ10.5GPT-5.4-nano · high: n=20, μ10.4GPT-5.6-luna · medium · Jul 9 launch price: n=20, μ10.4GPT-5.6-luna · medium · Jul 30 price-cut snapshot: n=20, μ10.4DeepSeek V4 Flash · max · Apr 24: n=6, μ10.2DeepSeek V4 Flash · xhigh · Apr 24: n=7, μ9.4GPT-5.4-nano · low: n=40, μ8.7DeepSeek V4 Flash 0731 · low · Jul 31: n=20, μ7.8DeepSeek V4 Flash 0731 · high · Jul 31: n=20, μ7.6DeepSeek V4 Flash 0731 · medium · Jul 31: n=20, μ7.3Gemini 3.5 Flash Lite · high: n=4, μ7.3GLM 5.2 · Default: n=19, μ6.7Gemini 2.5 Flash Lite · low: n=15, μ6.1DeepSeek V4 Flash · low · Apr 24: n=20, μ5.3Kimi K2.6 · Default: n=9, μ5.2DeepSeek V4 Flash · medium · Apr 24: n=20, μ5.2DeepSeek V4 Flash · high · Apr 24: n=20, μ5.0Kimi K3 · Default: n=9, μ4.9DeepSeek V4 Pro · Default: n=18, μ4.4Gemini 3.5 Flash Lite · medium: n=3, μ4.3Gemini 3.5 Flash Lite · low: n=4, μ3.5Gemini 2.5 Flash Lite · Default: n=16, μ3.4DeepSeek V4 Flash · Default · Apr 24: n=15, μ3.3Gemini 3.1 Flash Lite · Default: n=1, μ3.0Gemini 3.5 Flash Lite · minimal: n=4, μ3.0DeepSeek V4 Flash 0731 · Default · Jul 31: n=20, μ2.9

Hover a point for its cohort detail. Terra and Luna retain their Jul 9 launch-price points and add Jul 30 price-cut snapshots using the same measured token usage.

Per-Achievement Breakdown

All 66 possible achievements, ordered by Craftax progression and annotated by capability family and world region. Each cell is based on 20 terminal rollouts.

SurvivalGatheringMiningCraftingCombatExplorationEmpowerment·D = dungeonO = overworld
Model
Odrink
Ofood
Osapling
Owood
Oeat cow
Ozombie
Owood pickaxe
Owood sword
Oplace plant
Oplace table
Ocoal
Ostone
Oskeleton
Oeat plant
Ostone pickaxe
Ostone sword
Oplace stone
Owake up
Oiron
Ddungeon
Oarrow
Otorch
Oplace furnace
Oplace torch
Dgnome archer
Dgnome warrior
Dgnomish mines
Oiron armour
Oiron pickaxe
Oiron sword
Odiamond
Dorc mage
Dorc solider
Ddrink potion
Dsewers
Dfind bow
Dfire bow
Dopen chest
Darcher
Dknight
Dkobold
Dlizard
Dvault
Odiamond armour
Odiamond pickaxe
Odiamond sword
Dcast spell
Druby
Dsapphire
Ddeep thing
Dtroll
Deat bat
Deat snail
Denchant armour
Denchant sword
Dtroll mines
Dlearn spell
Dfire elemental
Dfrost troll
Dice elemental
Dpigman
Dfire realm
Dice realm
Ddamage necromancer
Dnecromancer
Dgraveyard
Score
GPT-5.4-nanon=20 · medium
65
90
95
100
80
65
100
100
5
100
65
100
50
0
65
55
25
5
45
15
5
30
5
30
0
0
0
0
0
5
0
0
0
5
0
10
0
10
0
0
0
0
0
0
0
5
0
0
0
0
0
0
10
0
0
0
0
0
0
0
0
0
0
0
0
0
13.4
DeepSeek V4 Flash 0731n=7 · xhigh
71
29
86
100
29
29
100
100
43
100
57
100
0
0
86
100
43
0
29
0
43
29
57
29
0
0
0
0
29
14
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
13.0
GPT-5.6-terran=19 · low
84
74
63
100
74
53
95
95
37
100
68
89
16
0
79
68
47
0
37
11
0
0
68
0
0
0
0
0
0
0
0
0
0
0
0
5
0
5
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
12.7
DeepSeek V4 Flash 0731n=11 · max
55
18
64
100
18
9
100
91
64
100
64
100
18
0
73
73
36
0
55
9
27
18
82
18
0
0
0
0
27
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
12.2
GPT-5.6-lunan=20 · high
70
80
80
100
80
45
100
95
30
100
35
80
20
0
70
45
35
0
30
5
0
5
10
0
0
0
0
0
0
0
0
0
0
5
0
5
0
5
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
11.3
GPT-5.6-lunan=34 · low
79
85
97
100
82
47
94
91
26
100
24
88
0
0
38
6
26
0
21
6
0
6
9
0
0
0
0
0
3
0
0
3
0
3
0
6
0
6
0
0
0
0
0
0
0
0
0
0
0
0
0
0
3
0
0
0
0
0
0
0
0
0
0
0
0
0
10.5
GPT-5.4-nanon=20 · high
75
55
60
100
50
50
95
100
5
100
35
75
30
0
50
45
0
0
25
15
0
25
0
20
0
0
0
0
0
0
0
0
10
0
0
10
0
10
0
0
0
0
0
0
0
0
0
0
0
0
0
0
5
0
0
0
0
0
0
0
0
0
0
0
0
0
10.4
GPT-5.6-lunan=20 · medium
75
70
85
100
70
45
90
80
35
100
40
80
10
0
45
30
45
0
10
0
0
0
25
0
0
0
0
0
5
0
5
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
10.4
DeepSeek V4 Flashn=6 · max
83
17
67
100
17
17
83
83
50
83
50
83
17
0
83
67
17
0
0
17
0
17
17
0
0
0
0
0
0
0
17
0
0
0
0
17
0
17
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
10.2
DeepSeek V4 Flashn=7 · xhigh
71
43
86
100
43
0
100
100
43
100
0
86
0
0
57
57
29
0
0
0
0
0
29
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
9.4
GPT-5.4-nanon=40 · low
65
55
75
100
53
53
73
90
13
100
43
68
10
0
15
8
8
3
10
8
3
3
3
0
0
0
0
0
0
0
3
0
0
3
0
3
0
3
0
0
0
0
0
0
0
0
0
0
0
0
0
0
3
0
0
0
0
0
0
0
0
0
0
0
0
0
8.7
DeepSeek V4 Flash 0731n=20 · low
30
0
35
100
0
20
95
95
25
95
25
80
0
0
75
55
5
0
20
5
5
0
15
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
7.8
DeepSeek V4 Flash 0731n=20 · high
25
15
35
100
15
15
90
85
15
100
30
70
5
0
60
50
0
0
10
5
0
5
25
0
0
0
0
0
5
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
7.6
DeepSeek V4 Flash 0731n=20 · medium
30
15
55
100
15
10
85
85
25
100
15
65
0
0
55
50
5
0
10
0
0
0
10
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
7.3
Gemini 3.5 Flash Liten=4 · high
50
25
50
100
25
25
100
100
25
100
0
50
0
0
25
25
25
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
7.3
GLM 5.2n=19
58
26
95
100
26
37
58
47
42
95
21
42
5
0
0
0
11
0
0
0
0
0
5
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
6.7
Gemini 2.5 Flash Liten=15 · low
53
27
73
93
27
7
53
47
13
93
0
53
0
0
13
0
20
40
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
6.1
DeepSeek V4 Flashn=20 · low
5
10
50
100
10
5
80
55
5
85
5
55
5
0
35
15
0
0
10
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
5.3
Kimi K2.6n=9
56
44
78
100
44
11
33
11
11
89
0
22
0
0
11
0
11
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
5.2
DeepSeek V4 Flashn=20 · medium
5
10
45
100
10
5
85
75
10
90
10
45
0
0
15
15
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
5.2
DeepSeek V4 Flashn=20 · high
10
5
25
90
5
10
80
65
5
80
5
50
5
0
30
20
0
0
5
0
0
0
5
0
0
0
0
0
5
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
5.0
Kimi K3n=9
33
44
89
100
44
33
33
11
11
89
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
4.9
DeepSeek V4 Pron=18
50
17
94
100
17
28
28
0
11
78
0
11
0
0
0
0
6
6
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
4.4
Gemini 3.5 Flash Liten=3 · medium
0
0
33
100
0
0
67
67
0
100
0
67
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
4.3
Gemini 3.5 Flash Liten=4 · low
0
25
50
100
25
0
25
25
0
75
0
25
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
3.5
Gemini 2.5 Flash Liten=16
25
13
100
88
13
13
0
0
31
25
0
0
0
0
0
0
0
38
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
3.4
DeepSeek V4 Flashn=15
47
13
100
100
13
7
7
0
0
40
0
7
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
3.3
Gemini 3.1 Flash Liten=1
100
0
100
100
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
3.0
Gemini 3.5 Flash Liten=4 · minimal
25
0
50
100
0
0
0
50
0
75
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
3.0
DeepSeek V4 Flash 0731n=20
45
20
90
85
20
5
5
5
0
10
0
5
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
2.9

Trajectories

Select a model and rollout, then inspect the complete Craftax world replay, policy reasoning, actions, survival state, and achievement timeline.

Rollouts

Craftax · sealed V5

deepseek/deepseek-v4-flash-0731 / xhigh / seed 7

21
achievements
20.40
reward
210
steps
$0.0461
cost
Browse all traces →
Loading replay…

trajectory evidence

Achievements over policy turns

The Craftax analogue of Voyager's capability curve: each stepped line is the mean number of distinct achievements across the selected real rollouts. A policy turn equals one in-game action. Crafting icons mark where each cohort first hits a ≥50% unlock rate; one callout per achievement lands on the earliest such unlock.

Diagram models · 12/12TrajectoryOrganization
05.310.515.82113977IN-GAME ACTION (POLICY TURN)DISTINCT ACHIEVEMENTS0.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.015.00.00.00.00.00.00.00.0collect wood · T2 · DeepSeek V4 Flash 0731 · high · 100%place table · T16 · DeepSeek V4 Flash 0731 · high · 100%make wood pickaxe · T19 · DeepSeek V4 Flash 0731 · high · 90%make wood sword · T19 · DeepSeek V4 Flash 0731 · high · 85%collect stone · T40 · DeepSeek V4 Flash 0731 · high · 70%make stone pickaxe · T58 · DeepSeek V4 Flash 0731 · high · 60%collect wood · T2 · DeepSeek V4 Flash 0731 · low · 100%place table · T13 · DeepSeek V4 Flash 0731 · low · 95%make wood pickaxe · T14 · DeepSeek V4 Flash 0731 · low · 95%make wood sword · T18 · DeepSeek V4 Flash 0731 · low · 95%collect stone · T54 · DeepSeek V4 Flash 0731 · low · 80%make stone pickaxe · T63 · DeepSeek V4 Flash 0731 · low · 75%collect wood · T2 · DeepSeek V4 Flash 0731 · max · 100%place table · T12 · DeepSeek V4 Flash 0731 · max · 100%make wood pickaxe · T14 · DeepSeek V4 Flash 0731 · max · 100%make wood sword · T15 · DeepSeek V4 Flash 0731 · max · 91%collect stone · T30 · DeepSeek V4 Flash 0731 · max · 100%make stone pickaxe · T49 · DeepSeek V4 Flash 0731 · max · 73%make stone sword · T67 · DeepSeek V4 Flash 0731 · max · 73%collect sapling · T68 · DeepSeek V4 Flash 0731 · max · 64%collect wood · T2 · DeepSeek V4 Flash 0731 · medium · 100%place table · T13 · DeepSeek V4 Flash 0731 · medium · 100%make wood pickaxe · T14 · DeepSeek V4 Flash 0731 · medium · 85%make wood sword · T23 · DeepSeek V4 Flash 0731 · medium · 85%collect stone · T53 · DeepSeek V4 Flash 0731 · medium · 65%make stone pickaxe · T77 · DeepSeek V4 Flash 0731 · medium · 55%collect wood · T2 · DeepSeek V4 Flash 0731 · none · 85%collect sapling · T8 · DeepSeek V4 Flash 0731 · none · 90%collect wood · T2 · DeepSeek V4 Flash 0731 · xhigh · 100%place table · T15 · DeepSeek V4 Flash 0731 · xhigh · 100%make wood pickaxe · T16 · DeepSeek V4 Flash 0731 · xhigh · 100%make wood sword · T20 · DeepSeek V4 Flash 0731 · xhigh · 100%collect sapling · T47 · DeepSeek V4 Flash 0731 · xhigh · 86%collect stone · T56 · DeepSeek V4 Flash 0731 · xhigh · 100%make stone sword · T66 · DeepSeek V4 Flash 0731 · xhigh · 100%make stone pickaxe · T67 · DeepSeek V4 Flash 0731 · xhigh · 86%collect wood · T2 · DeepSeek V4 Flash · high · 90%place table · T19 · DeepSeek V4 Flash · high · 80%make wood pickaxe · T27 · DeepSeek V4 Flash · high · 80%make wood sword · T33 · DeepSeek V4 Flash · high · 65%collect stone · T69 · DeepSeek V4 Flash · high · 50%collect wood · T2 · DeepSeek V4 Flash · low · 100%place table · T19 · DeepSeek V4 Flash · low · 85%make wood pickaxe · T36 · DeepSeek V4 Flash · low · 80%collect sapling · T44 · DeepSeek V4 Flash · low · 50%make wood sword · T47 · DeepSeek V4 Flash · low · 55%collect stone · T75 · DeepSeek V4 Flash · low · 55%collect wood · T2 · DeepSeek V4 Flash · max · 100%place table · T15 · DeepSeek V4 Flash · max · 83%make wood pickaxe · T27 · DeepSeek V4 Flash · max · 83%make wood sword · T43 · DeepSeek V4 Flash · max · 83%collect stone · T47 · DeepSeek V4 Flash · max · 83%make stone pickaxe · T67 · DeepSeek V4 Flash · max · 83%collect wood · T2 · DeepSeek V4 Flash · medium · 100%place table · T17 · DeepSeek V4 Flash · medium · 90%make wood pickaxe · T31 · DeepSeek V4 Flash · medium · 85%make wood sword · T50 · DeepSeek V4 Flash · medium · 75%collect wood · T2 · DeepSeek V4 Flash · none · 100%collect sapling · T8 · DeepSeek V4 Flash · none · 100%collect wood · T2 · DeepSeek V4 Flash · xhigh · 100%place table · T14 · DeepSeek V4 Flash · xhigh · 100%make wood sword · T43 · DeepSeek V4 Flash · xhigh · 100%make wood pickaxe · T44 · DeepSeek V4 Flash · xhigh · 100%collect sapling · T47 · DeepSeek V4 Flash · xhigh · 86%collect stone · T58 · DeepSeek V4 Flash · xhigh · 86%collect wood · T2 · DeepSeek V4 Pro · none · 100%collect sapling · T18 · DeepSeek V4 Pro · none · 94%place table · T21 · DeepSeek V4 Pro · none · 78%collect wood · T3 · Gemini 2.5 Flash Lite · low · 93%place table · T25 · Gemini 2.5 Flash Lite · low · 93%collect sapling · T66 · Gemini 2.5 Flash Lite · low · 73%collect wood · T3 · Gemini 2.5 Flash Lite · none · 88%collect sapling · T10 · Gemini 2.5 Flash Lite · none · 100%collect wood · T2 · Gemini 3.1 Flash Lite · none · 100%collect sapling · T6 · Gemini 3.1 Flash Lite · none · 100%collect wood · T1 · Gemini 3.5 Flash Lite · high · 100%place table · T13 · Gemini 3.5 Flash Lite · high · 100%make wood pickaxe · T22 · Gemini 3.5 Flash Lite · high · 100%make wood sword · T31 · Gemini 3.5 Flash Lite · high · 100%collect wood · T2 · Gemini 3.5 Flash Lite · low · 100%collect sapling · T5 · Gemini 3.5 Flash Lite · low · 50%place table · T12 · Gemini 3.5 Flash Lite · low · 75%collect wood · T2 · Gemini 3.5 Flash Lite · medium · 100%place table · T12 · Gemini 3.5 Flash Lite · medium · 100%collect wood · T2 · Gemini 3.5 Flash Lite · minimal · 100%place table · T27 · Gemini 3.5 Flash Lite · minimal · 75%collect sapling · T50 · Gemini 3.5 Flash Lite · minimal · 50%make wood sword · T70 · Gemini 3.5 Flash Lite · minimal · 50%collect wood · T2 · GPT-5.4 Nano · high · 100%place table · T15 · GPT-5.4 Nano · high · 100%make wood sword · T23 · GPT-5.4 Nano · high · 100%make wood pickaxe · T25 · GPT-5.4 Nano · high · 95%collect stone · T55 · GPT-5.4 Nano · high · 75%collect wood · T2 · GPT-5.4 Nano · low · 100%place table · T14 · GPT-5.4 Nano · low · 100%make wood sword · T30 · GPT-5.4 Nano · low · 90%make wood pickaxe · T57 · GPT-5.4 Nano · low · 73%collect sapling · T65 · GPT-5.4 Nano · low · 75%collect wood · T2 · GPT-5.4 Nano · medium · 100%place table · T14 · GPT-5.4 Nano · medium · 100%make wood sword · T22 · GPT-5.4 Nano · medium · 100%make wood pickaxe · T29 · GPT-5.4 Nano · medium · 100%collect stone · T54 · GPT-5.4 Nano · medium · 100%collect wood · T2 · GPT-5.6 Luna · high · 100%place table · T16 · GPT-5.6 Luna · high · 100%make wood pickaxe · T28 · GPT-5.6 Luna · high · 100%make wood sword · T37 · GPT-5.6 Luna · high · 95%collect stone · T58 · GPT-5.6 Luna · high · 80%collect wood · T2 · GPT-5.6 Luna · low · 100%place table · T21 · GPT-5.6 Luna · low · 100%collect sapling · T24 · GPT-5.6 Luna · low · 97%make wood pickaxe · T50 · GPT-5.6 Luna · low · 94%make wood sword · T63 · GPT-5.6 Luna · low · 91%collect wood · T2 · GPT-5.6 Luna · medium · 100%place table · T20 · GPT-5.6 Luna · medium · 100%collect sapling · T29 · GPT-5.6 Luna · medium · 85%make wood pickaxe · T38 · GPT-5.6 Luna · medium · 90%make wood sword · T47 · GPT-5.6 Luna · medium · 80%collect wood · T2 · GPT-5.6 Terra · low · 100%place table · T14 · GPT-5.6 Terra · low · 100%make wood pickaxe · T18 · GPT-5.6 Terra · low · 95%make wood sword · T28 · GPT-5.6 Terra · low · 95%collect stone · T53 · GPT-5.6 Terra · low · 89%collect wood · T2 · Kimi K2.6 · none · 100%collect sapling · T11 · Kimi K2.6 · none · 78%place table · T41 · Kimi K2.6 · none · 89%collect wood · T2 · Kimi K3 · none · 100%place table · T23 · Kimi K3 · none · 89%collect sapling · T27 · Kimi K3 · none · 89%collect wood · T2 · GLM 5.2 · none · 100%collect sapling · T12 · GLM 5.2 · none · 95%place table · T24 · GLM 5.2 · none · 95%collect woodT1 · 100% earliestcollect saplingT5 · 50% earliestplace tableT12 · 100% earliestmake wood pickaxeT14 · 100% earliestmake wood swordT15 · 91% earliestcollect stoneT30 · 100% earliestmake stone pickaxeT49 · 73% earliestmake stone swordT66 · 100% earliest
SeriesDeepSeek V4 Flash 0731 · highn=20DeepSeek V4 Flash 0731 · lown=20DeepSeek V4 Flash 0731 · maxn=11DeepSeek V4 Flash 0731 · mediumn=20DeepSeek V4 Flash 0731 · nonen=20DeepSeek V4 Flash 0731 · xhighn=7DeepSeek V4 Flash · highn=20DeepSeek V4 Flash · lown=20DeepSeek V4 Flash · maxn=6DeepSeek V4 Flash · mediumn=20DeepSeek V4 Flash · nonen=15DeepSeek V4 Flash · xhighn=7DeepSeek V4 Pro · nonen=18Gemini 2.5 Flash Lite · lown=15Gemini 2.5 Flash Lite · nonen=16Gemini 3.1 Flash Lite · nonen=1Gemini 3.5 Flash Lite · highn=4Gemini 3.5 Flash Lite · lown=4Gemini 3.5 Flash Lite · mediumn=3Gemini 3.5 Flash Lite · minimaln=4GPT-5.4 Nano · highn=20GPT-5.4 Nano · lown=40GPT-5.4 Nano · mediumn=20GPT-5.6 Luna · highn=20GPT-5.6 Luna · lown=34GPT-5.6 Luna · mediumn=20GPT-5.6 Terra · lown=19Kimi K2.6 · nonen=9Kimi K3 · nonen=9GLM 5.2 · nonen=19

survival envelope

Where each run takes damage—and gives out

One row summarizes each model’s terminal cohort. Lines are median vitals across still-active runs; the health ribbon spans the middle 50%. Red density bars aggregate damage, and bottom ticks show the distribution of death times.

226/461
survived
235
deaths
t233
median death
1537
damage events
healthfooddrinkenergydamage│ death timing● survivors
t0t113t225t338t450deepseek/deepseek-v4-flash-0731 · highn=20 · 1 died● 19deepseek/deepseek-v4-flash-0731 · high: n=20; 1 deaths; 19 survived; median death t114deepseek/deepseek-v4-flash-0731 · lown=20 · 1 died● 19deepseek/deepseek-v4-flash-0731 · low: n=20; 1 deaths; 19 survived; median death t64deepseek/deepseek-v4-flash-0731 · mediumn=20 · 1 died● 19deepseek/deepseek-v4-flash-0731 · medium: n=20; 1 deaths; 19 survived; median death t37deepseek/deepseek-v4-flash-0731 · nonen=20 · 18 died● 2deepseek/deepseek-v4-flash-0731 · none: n=20; 18 deaths; 2 survived; median death t241DeepSeek V4 Flash · highn=20 · 1 died● 19deepseek/deepseek-v4-flash · high: n=20; 1 deaths; 19 survived; median death t25DeepSeek V4 Flash · lown=20 · 2 died● 18deepseek/deepseek-v4-flash · low: n=20; 2 deaths; 18 survived; median death t51DeepSeek V4 Flash · mediumn=20 · 2 died● 18deepseek/deepseek-v4-flash · medium: n=20; 2 deaths; 18 survived; median death t41DeepSeek V4 Flash · nonen=15 · 13 died● 2deepseek/deepseek-v4-flash · none: n=15; 13 deaths; 2 survived; median death t233DeepSeek V4 Pro · nonen=18 · 17 died● 1deepseek/deepseek-v4-pro · none: n=18; 17 deaths; 1 survived; median death t277Gemini 2.5 Flash Lite · lown=15 · 11 died● 4google/gemini-2.5-flash-lite · low: n=15; 11 deaths; 4 survived; median death t200Gemini 2.5 Flash Lite · nonen=16 · 14 died● 2google/gemini-2.5-flash-lite · none: n=16; 14 deaths; 2 survived; median death t282Gemini 3.1 Flash Lite · nonen=1 · 1 diedgoogle/gemini-3.1-flash-lite · none: n=1; 1 deaths; 0 survived; median death t341gpt-5.4-nano · highn=20 · 5 died● 15gpt-5.4-nano · high: n=20; 5 deaths; 15 survived; median death t201gpt-5.4-nano · lown=40 · 20 died● 20gpt-5.4-nano · low: n=40; 20 deaths; 20 survived; median death t218gpt-5.4-nano · mediumn=20 · 10 died● 10gpt-5.4-nano · medium: n=20; 10 deaths; 10 survived; median death t244gpt-5.6-luna · highn=20 · 10 died● 10gpt-5.6-luna · high: n=20; 10 deaths; 10 survived; median death t221gpt-5.6-luna · lown=34 · 24 died● 10gpt-5.6-luna · low: n=34; 24 deaths; 10 survived; median death t242gpt-5.6-luna · mediumn=20 · 12 died● 8gpt-5.6-luna · medium: n=20; 12 deaths; 8 survived; median death t232gpt-5.6-terra · lown=19 · 13 died● 6gpt-5.6-terra · low: n=19; 13 deaths; 6 survived; median death t262GLM 5.2 · nonen=19 · 17 died● 2z-ai/glm-5.2 · none: n=19; 17 deaths; 2 survived; median death t262deepseek/deepseek-v4-flash-0731 · xhighn=7 · 3 died● 4deepseek/deepseek-v4-flash-0731 · xhigh: n=7; 3 deaths; 4 survived; median death t117DeepSeek V4 Flash · maxn=6 · 1 died● 5deepseek/deepseek-v4-flash · max: n=6; 1 deaths; 5 survived; median death t221Gemini 3.5 Flash Lite · highn=4 · 4 diedgoogle/gemini-3.5-flash-lite · high: n=4; 4 deaths; 0 survived; median death t192Gemini 3.5 Flash Lite · lown=4 · 4 diedgoogle/gemini-3.5-flash-lite · low: n=4; 4 deaths; 0 survived; median death t116Kimi K2.6 · nonen=9 · 9 diedmoonshotai/kimi-k2.6 · none: n=9; 9 deaths; 0 survived; median death t248Kimi K3 · nonen=9 · 9 diedmoonshotai/kimi-k3 · none: n=9; 9 deaths; 0 survived; median death t215deepseek/deepseek-v4-flash-0731 · maxn=11 · 2 died● 9deepseek/deepseek-v4-flash-0731 · max: n=11; 2 deaths; 9 survived; median death t75DeepSeek V4 Flash · xhighn=7 · 3 died● 4deepseek/deepseek-v4-flash · xhigh: n=7; 3 deaths; 4 survived; median death t141Gemini 3.5 Flash Lite · minimaln=4 · 4 diedgoogle/gemini-3.5-flash-lite · minimal: n=4; 4 deaths; 0 survived; median death t147Gemini 3.5 Flash Lite · mediumn=3 · 3 diedgoogle/gemini-3.5-flash-lite · medium: n=3; 3 deaths; 0 survived; median death t196

“Damage event” means a negative health delta between sealed engine snapshots. The trace does not yet attribute every loss to a specific attacker, so this view does not invent one.

Achievement capability tree

Direct engine gates and tier prerequisites across all 66 Craftax achievements. Choose a model to color each node by the share of real terminal runs that unlocked it.

SurvivalGatheringMiningCraftingCombatExplorationEmpowerment
SURVIVEWOODSTONEIRONMINESSEWERSVAULTMASTERYELEMENTALBOSScollect wood · Gathering · overworld · 97% of 98 runscollectwoodO97%collect food · Survival · overworld · 14% of 98 runscollectfoodO14%collect drink · Survival · overworld · 38% of 98 runscollectdrinkO38%collect sapling · Gathering · overworld · 57% of 98 runscollectsaplingO57%eat cow · Survival · overworld · 14% of 98 runseatcowO14%place table · Crafting · overworld · 81% of 98 runsplacetableO81%make wood pickaxe · Crafting · overworld · 74% of 98 runsmake woodpickaxeO74%make wood sword · Crafting · overworld · 72% of 98 runsmake woodswordO72%place plant · Survival · overworld · 23% of 98 runsplaceplantO23%defeat zombie · Combat · overworld · 13% of 98 runsdefeatzombieO13%collect stone · Mining · overworld · 63% of 98 runscollectstoneO63%collect coal · Mining · overworld · 26% of 98 runscollectcoalO26%place stone · Crafting · overworld · 9% of 98 runsplacestoneO9%make stone pickaxe · Crafting · overworld · 53% of 98 runsmake stonepickaxeO53%make stone sword · Crafting · overworld · 47% of 98 runsmake stoneswordO47%defeat skeleton · Combat · overworld · 3% of 98 runsdefeatskeletonO3%eat plant · Survival · overworld · 0% of 98 runseatplantO0%wake up · Survival · overworld · 0% of 98 runswakeupO0%place furnace · Crafting · overworld · 23% of 98 runsplacefurnaceO23%collect iron · Mining · overworld · 16% of 98 runscollectironO16%make arrow · Crafting · overworld · 7% of 98 runsmakearrowO7%make torch · Crafting · overworld · 5% of 98 runsmaketorchO5%place torch · Crafting · overworld · 4% of 98 runsplacetorchO4%enter dungeon · Exploration · dungeon · 3% of 98 runsenterdungeonD3%make iron pickaxe · Crafting · overworld · 6% of 98 runsmake ironpickaxeO6%make iron sword · Crafting · overworld · 1% of 98 runsmake ironswordO1%make iron armour · Crafting · overworld · 0% of 98 runsmake ironarmourO0%enter gnomish mines · Exploration · dungeon · 0% of 98 runsenter gnomishminesD0%defeat gnome warrior · Combat · dungeon · 0% of 98 runsdefeat gnomewarriorD0%defeat gnome archer · Combat · dungeon · 0% of 98 runsdefeat gnomearcherD0%collect diamond · Mining · overworld · 0% of 98 runscollectdiamondO0%enter sewers · Exploration · dungeon · 0% of 98 runsentersewersD0%defeat orc solider · Combat · dungeon · 0% of 98 runsdefeat orcsoliderD0%defeat orc mage · Combat · dungeon · 0% of 98 runsdefeat orcmageD0%find bow · Exploration · dungeon · 0% of 98 runsfindbowD0%fire bow · Combat · dungeon · 0% of 98 runsfirebowD0%open chest · Exploration · dungeon · 0% of 98 runsopenchestD0%drink potion · Empowerment · dungeon · 0% of 98 runsdrinkpotionD0%make diamond pickaxe · Crafting · overworld · 0% of 98 runsmake diamondpickaxeO0%make diamond sword · Crafting · overworld · 0% of 98 runsmake diamondswordO0%make diamond armour · Crafting · overworld · 0% of 98 runsmake diamondarmourO0%enter vault · Exploration · dungeon · 0% of 98 runsentervaultD0%defeat lizard · Combat · dungeon · 0% of 98 runsdefeatlizardD0%defeat kobold · Combat · dungeon · 0% of 98 runsdefeatkoboldD0%defeat knight · Combat · dungeon · 0% of 98 runsdefeatknightD0%defeat archer · Combat · dungeon · 0% of 98 runsdefeatarcherD0%collect sapphire · Mining · dungeon · 0% of 98 runscollectsapphireD0%collect ruby · Mining · dungeon · 0% of 98 runscollectrubyD0%enter troll mines · Exploration · dungeon · 0% of 98 runsenter trollminesD0%defeat troll · Combat · dungeon · 0% of 98 runsdefeattrollD0%defeat deep thing · Combat · dungeon · 0% of 98 runsdefeat deepthingD0%eat bat · Survival · dungeon · 0% of 98 runseatbatD0%eat snail · Survival · dungeon · 0% of 98 runseatsnailD0%learn spell · Empowerment · dungeon · 0% of 98 runslearnspellD0%cast spell · Empowerment · dungeon · 0% of 98 runscastspellD0%enchant sword · Empowerment · dungeon · 0% of 98 runsenchantswordD0%enchant armour · Empowerment · dungeon · 0% of 98 runsenchantarmourD0%enter fire realm · Exploration · dungeon · 0% of 98 runsenter firerealmD0%enter ice realm · Exploration · dungeon · 0% of 98 runsenter icerealmD0%defeat pigman · Combat · dungeon · 0% of 98 runsdefeatpigmanD0%defeat fire elemental · Combat · dungeon · 0% of 98 runsdefeat fireelementalD0%defeat frost troll · Combat · dungeon · 0% of 98 runsdefeat frosttrollD0%defeat ice elemental · Combat · dungeon · 0% of 98 runsdefeat iceelementalD0%enter graveyard · Exploration · dungeon · 0% of 98 runsentergraveyardD0%damage necromancer · Combat · dungeon · 0% of 98 runsdamagenecromancerD0%defeat necromancer · Combat · dungeon · 0% of 98 runsdefeatnecromancerD0%

Seed-adjusted comparison

Paired achievement effects

Pick two policies. For every achievement, we compare whether each policy unlocked it on the exact same starting seed. This shows where changing the policy alters the capability ladder—not merely which model has the higher aggregate score.

versusn=20 shared seeds

Achievement score

+2.10 per run

95% paired interval [-0.60, 4.80]

Survival

0 percentage points

candidate minus reference survival rate

Inference cost

+$0.117

mean cost difference per rollout

Achievement outcomePaired seedsEffectCandidateRef.

collect coal

+7 candidate-only · −1 reference-only · 6 both

+30 pp

65%

35%

defeat skeleton

+7 candidate-only · −1 reference-only · 3 both

+30 pp

50%

20%

place torch

+6 candidate-only · −0 reference-only · 0 both

+30 pp

30%

0%

place plant

+1 candidate-only · −6 reference-only · 0 both

-25 pp

5%

30%

make torch

+6 candidate-only · −1 reference-only · 0 both

+25 pp

30%

5%

defeat zombie

+8 candidate-only · −4 reference-only · 5 both

+20 pp

65%

45%

collect stone

+4 candidate-only · −0 reference-only · 16 both

+20 pp

100%

80%

collect iron

+6 candidate-only · −3 reference-only · 3 both

+15 pp

45%

30%

collect sapling

+4 candidate-only · −1 reference-only · 15 both

+15 pp

95%

80%

make stone sword

+6 candidate-only · −4 reference-only · 5 both

+10 pp

55%

45%

enter dungeon

+3 candidate-only · −1 reference-only · 0 both

+10 pp

15%

5%

eat snail

+2 candidate-only · −0 reference-only · 0 both

+10 pp

10%

0%

place stone

+3 candidate-only · −5 reference-only · 2 both

-10 pp

25%

35%

collect food

+4 candidate-only · −2 reference-only · 14 both

+10 pp

90%

80%

make wood sword

+1 candidate-only · −0 reference-only · 19 both

+5 pp

100%

95%

place furnace

+1 candidate-only · −2 reference-only · 0 both

-5 pp

5%

10%

find bow

+2 candidate-only · −1 reference-only · 0 both

+5 pp

10%

5%

open chest

+2 candidate-only · −1 reference-only · 0 both

+5 pp

10%

5%

wake up

+1 candidate-only · −0 reference-only · 0 both

+5 pp

5%

0%

make arrow

+1 candidate-only · −0 reference-only · 0 both

+5 pp

5%

0%

make iron sword

+1 candidate-only · −0 reference-only · 0 both

+5 pp

5%

0%

make diamond sword

+1 candidate-only · −0 reference-only · 0 both

+5 pp

5%

0%

collect drink

+3 candidate-only · −4 reference-only · 10 both

-5 pp

65%

70%

make stone pickaxe

+2 candidate-only · −3 reference-only · 11 both

-5 pp

65%

70%

eat cow

+2 candidate-only · −2 reference-only · 14 both

0 pp

80%

80%

drink potion

+1 candidate-only · −1 reference-only · 0 both

0 pp

5%

5%

candidate onlyreference onlybothneither

These are paired treatment effects of policy choice on achievement outcomes over the observed seeds. They are not CATEs: credible conditional effects require more shared seeds and pre-rollout world features such as biome, nearby resources, and threat density.

What sets the best rollouts apart

The top 115 rollouts by achievement score (score ≥ 11.0) against the remaining 346, under the current filters. Both groups played the same engine, so a gap here is a difference in how the policy spent its turns.

What accompanies a high-scoring run

A direct comparison of the top 115 runs with the other 346. Because the groups are defined by achievement score, score itself is intentionally omitted. These are associated outcomes, not independent causes. Orange is the high-scoring group; gray is everyone else. Values are group averages, except survival, which is the share that finished without dying.

Survived to a cap

70%vs42%

+29 percentage points

Tool types crafted

3.77vs1.51

+2.26 per run

Resources gathered

10.5vs3.77

+6.73 per run

MeasureTop runs / other runsTopOthers

Engine steps taken

how long the run stayed active

302.6174.8

Resources gathered

resource events reached

10.53.77

Tool types crafted

distinct crafting unlocks

3.771.51

Enemy types beaten

distinct combat unlocks

0.760.23

Survived to a cap

finished without dying

70%42%

What stronger runs spent

Output tokens: 88,103 vs 34,674

Inference cost: $0.12 vs $0.04

These are costs of longer, more capable runs—not capabilities by themselves.

“Tool” and “enemy” counts are distinct achievement types reached, not repeated crafts or kills; the trace records unlocks, not lifetime action counters.

The best runs

The ten highest-scoring rollouts under the current filters.

deepseek/deepseek-v4-flash-0731 [xhigh]s7
21210 steps · time_budget_exhausted↗gpt-5.4-nano [medium]s12
20405 steps · time_budget_exhausted↗gpt-5.4-nano [medium]s17
19338 steps · death↗deepseek/deepseek-v4-flash-0731 [xhigh]s6
19213 steps · time_budget_exhausted↗gpt-5.6-luna [high]s8
18393 steps · time_budget_exhausted↗gpt-5.4-nano [high]s11
17264 steps · time_budget_exhausted↗gpt-5.6-luna [low]s4
17380 steps · death↗deepseek/deepseek-v4-flash-0731 [max]s14
17216 steps · time_budget_exhausted↗deepseek/deepseek-v4-flash-0731 [max]s10
16270 steps · time_budget_exhausted↗deepseek/deepseek-v4-flash-0731 [max]s4
16196 steps · time_budget_exhausted↗

Dungeon

Craftax level 0 is the overworld; every level above it is dungeon. This section counts only what happens below ground — descents, floor gates, and the achievements that cannot be earned on the surface.

18 / 461
terminal rollouts that got below the overworld
enter dungeon
deepest floor gate reached by any rollout
1
deepest engine level observed

Descent rate by cohort

Share of each cohort’s terminal rollouts that entered the dungeon at least once. Cohorts are listed with their call budget because a policy with more calls has more turns in which to find a way down.

deepseek/deepseek-v4-flash [max]
1/6committed · 60 calls · plan 5-20
gpt-5.4-nano [high]
3/20committed · 60 calls · plan 5-20
gpt-5.4-nano [medium]
3/20committed · 60 calls · plan 5-20
gpt-5.6-terra [low]
2/19committed · 60 calls · plan 5-20
deepseek/deepseek-v4-flash-0731 [max]
1/11committed · 60 calls · plan 5-20
gpt-5.4-nano [low]
3/40committed · 60 calls · plan 5-20
gpt-5.6-luna [low]
2/34committed · 60 calls · plan 5-20
deepseek/deepseek-v4-flash-0731 [high]
1/20committed · 60 calls · plan 5-20
deepseek/deepseek-v4-flash-0731 [low]
1/20committed · 60 calls · plan 5-20
gpt-5.6-luna [high]
1/20committed · 60 calls · plan 5-20
deepseek/deepseek-v4-flash-0731 [medium]
0/20committed · 60 calls · plan 5-20
deepseek/deepseek-v4-flash-0731 [none]
0/20committed · 60 calls · plan 5-20
deepseek/deepseek-v4-flash [high]
0/20committed · 60 calls · plan 5-20
deepseek/deepseek-v4-flash [low]
0/20committed · 60 calls · plan 5-20
deepseek/deepseek-v4-flash [medium]
0/20committed · 60 calls · plan 5-20
gpt-5.6-luna [medium]
0/20committed · 60 calls · plan 5-20
z-ai/glm-5.2 [none]
0/19committed · 60 calls · plan 5-20
deepseek/deepseek-v4-pro [none]
0/18committed · 60 calls · plan 5-20
google/gemini-2.5-flash-lite [none]
0/16committed · 60 calls · plan 5-20
deepseek/deepseek-v4-flash [none]
0/15committed · 60 calls · plan 5-20
google/gemini-2.5-flash-lite [low]
0/15committed · 60 calls · plan 5-20
moonshotai/kimi-k2.6 [none]
0/9committed · 60 calls · plan 5-20
moonshotai/kimi-k3 [none]
0/9committed · 60 calls · plan 5-20
deepseek/deepseek-v4-flash-0731 [xhigh]
0/7committed · 60 calls · plan 5-20
deepseek/deepseek-v4-flash [xhigh]
0/7committed · 60 calls · plan 5-20
google/gemini-3.5-flash-lite [high]
0/4committed · 60 calls · plan 5-20
google/gemini-3.5-flash-lite [low]
0/4committed · 60 calls · plan 5-20
google/gemini-3.5-flash-lite [minimal]
0/4committed · 60 calls · plan 5-20
google/gemini-3.5-flash-lite [medium]
0/3committed · 60 calls · plan 5-20
google/gemini-3.1-flash-lite [none]
0/1committed · 60 calls · plan 5-20

Floor gates

The dungeon spine in descent order. Each gate must be passed before the next is reachable, so the first empty bar is where every policy stops.

18
enter dungeon
0
enter gnomish mines
0
enter sewers
0
enter vault
0
enter troll mines
0
enter fire realm
0
enter ice realm
0
enter graveyard

Below-ground achievements

All 30 achievements that cannot be earned on the surface. Unreached entries are kept visible: the shape of what is missing is the result.

enter dungeon18enter gnomish minesdefeat gnome warriordefeat gnome archereat batenter sewersdefeat orc solider2defeat orc mage1eat snail5find bow10fire bowopen chest10drink potion4enter vaultdefeat lizarddefeat kobolddefeat knightdefeat archerenter troll minesdefeat trolldefeat deep thingenter fire realmdefeat pigmandefeat fire elementalenter ice realmdefeat frost trolldefeat ice elementalenter graveyarddamage necromancerdefeat necromancer

Dungeon entries

Every terminal rollout that got below the overworld, with the turn it descended.

gpt-5.4-nano [low]s12
descended turn 80level 1deathenter dungeon
deepseek/deepseek-v4-flash-0731 [low]s12
descended turn 90level 1time_budget_exhaustedenter dungeon
deepseek/deepseek-v4-flash-0731 [max]s12
descended turn 90level 1time_budget_exhaustedenter dungeon
deepseek/deepseek-v4-flash-0731 [high]s12
descended turn 91level 1deathenter dungeon
gpt-5.6-terra [low]s12
descended turn 97level 1deathenter dungeon
gpt-5.4-nano [low]s4
descended turn 99level 1call_budget_exhaustedenter dungeon · eat snail
gpt-5.6-luna [low]s8
descended turn 110level 1deathenter dungeon · defeat orc mage · eat snail · find bow · open chest · drink potion
gpt-5.6-luna [high]s8
descended turn 121level 1time_budget_exhaustedenter dungeon · find bow · open chest · drink potion
gpt-5.4-nano [medium]s15
descended turn 126level 1deathenter dungeon · eat snail
gpt-5.6-terra [low]s8
descended turn 134level 1deathenter dungeon · find bow · open chest
gpt-5.4-nano [high]s11
descended turn 138level 1time_budget_exhaustedenter dungeon · defeat orc solider · eat snail · find bow · open chest
gpt-5.4-nano [medium]s17
descended turn 148level 1deathenter dungeon · eat snail · find bow · open chest · drink potion
deepseek/deepseek-v4-flash [max]s8
descended turn 169level 1time_budget_exhaustedenter dungeon · find bow · open chest
gpt-5.4-nano [high]s0
descended turn 174level 1deathenter dungeon
gpt-5.4-nano [low]s8
descended turn 176level 1deathenter dungeon · find bow · open chest · drink potion
gpt-5.4-nano [high]s12
descended turn 221level 1time_budget_exhaustedenter dungeon · defeat orc solider · find bow · open chest
gpt-5.6-luna [low]s4
descended turn 264level 1deathenter dungeon · find bow · open chest
gpt-5.4-nano [medium]s12
descended turn 365level 1time_budget_exhaustedenter dungeon · find bow · open chest
turn 0grey = overworld · orange = below ground410

Score design

The headline is achievement-ladder score: each distinct capability milestone counts once. The Cost vs. Performance chart can isolate one family or region at a time without changing the canonical score used elsewhere.

Policy-visible observations and evaluator-only evidence are kept separate. Engine-rendered frames are derived from the Rust gold lane and every rollout links to its SHA-256-backed artifact.

Related evaluations

DungeonGrid →Shrimp Farm Eval ↗
© 2026 SynthWorkshopChangelogDocsBook a Demo