GAMEPLAY SYSTEMS
28Movement6
Combat6
Physics5
AI & Navigation5
Progression & Economy6
Terminal-Bench for interactive software
GamePhanes
Give agents a game. Grade the engineering.
A Godot-first Terminal-Bench-style runner for coding agents that design, build, debug, and repair interactive software inside task workspaces.
LATEST PUBLIC ENVIRONMENT RUN
golden_starfall_protocol_001
BUILD PASS
RUNTIME PASS
1.00 SCORE
Terminal task, runtime proof
A terminal command exiting 0 does not mean the game works.
GamePhanes starts from a task workspace, records the Agent's terminal work, launches the modified project, probes runtime behavior, and returns deterministic feedback.
28/28Assertions passed
6Playable slices
0Protocol errors
100%Functional score
Benchmark taxonomy
100 tasks across six engineering domains.
TARGET MIX · NOT RELEASED COUNT
Domains describe where the work happens. Task types separately describe whether the Agent fixes, builds, debugs, optimizes, or delivers.
ENGINE & RUNTIME
22Scene Lifecycle6
Engine APIs5
State & Signals4
Performance4
Platform Compatibility3
UI & INTERACTION
16Input & Controls5
HUD & Menus4
Camera & Feedback4
Accessibility3
CONTENT & DESIGN
14Level Design4
Procedural Content3
Asset Integration3
Narrative & Dialogue2
Audio Integration2
ARCHITECTURE & DATA
12Scenes & Resources4
Save & Load3
Data-driven Systems3
Modularity & Dependencies2
DELIVERY & QUALITY
8Regression Safety3
Tests & Instrumentation3
Build & Packaging2
Public task registry
Concrete tasks, runnable contracts.
gamephanes-public==0.1
Open each task to inspect its exact instruction, run command, success criteria, runtime evidence, and evaluator boundary.
Gameplay systemsProgression & economyReference environmentUnrated
Survive the drone encounter, choose a protocol, and defeat the Oracle in an original 8–12 minute Godot slice.
Gameplay systemsMovementReference environmentUnrated
Sprint through an electric skyline, phase-jump, and chain three energy shards into the relay.
Gameplay systemsMovementCoding challengePublic developmentmedium
Restore the phase jump without breaking shard collection or relay completion.
Gameplay systemsCombatReference environmentUnrated
Reposition a field transmitter and burn threats from a living radio map.
Gameplay systemsPhysicsReference environmentUnrated
Reverse polarity, stabilize an energy core, and unlock the sealed chamber.
Gameplay systemsProgression & economyReference environmentUnrated
Spend carefully, place two towers, and hold the miniature keep until dawn.
Gameplay systemsCombatReference environmentUnrated
Enter a low-poly breach, strike its warden, and stabilize the dimensional ring.
Gameplay systemsMovementReference environmentUnrated
Verify movement, jumping, coin collection, and score updates in a compact Godot project.
Public task environments
Runnable projects for interactive software tasks.
Each slice is a reference workspace with a runtime harness, observable state, and deterministic score for coding-agent evaluation.
Golden demo · action vertical slice
Starfall Protocol / 星坠协议
Survive the drone encounter, choose a protocol, and defeat the Oracle in an original 8–12 minute Godot slice.
Momentum platformer
Neon Relay
Sprint through an electric skyline, phase-jump, and chain three energy shards into the relay.
Tactical survival
Last Signal
Reposition a field transmitter and burn threats from a living radio map.
Physics puzzle
Gravity Lab
Reverse polarity, stabilize an energy core, and unlock the sealed chamber.
Micro tower defense
Tiny Bastion
Spend carefully, place two towers, and hold the miniature keep until dawn.
3D arena combat
Rift Arena
Enter a low-poly breach, strike its warden, and stabilize the dimensional ring.
6 PROJECTS6 EXTERNAL HARNESSES28 ASSERTIONSALL VERIFIED
Terminal task lifecycle
From task workspace to hidden score.
- 01ProvisionTask workspace
- 02TerminalAgent commands
- 03EditCode + scene + assets
- 04Build / runEngine feedback
- 05EvaluateHidden assertions
- 06SubmitScore + trajectory
Designed for terminal agents
Give the Agent a workspace. Keep the evaluator hidden.
01
Task workspaces
Every task starts from a declared project state and can be reset without changing the benchmark source.
provision(task)
02
Terminal command audit
Record every command, stdout, stderr, exit code, file change, and cost in the coding trajectory.
{ command, stdout, exit_code }
03
Runtime evaluator
Launch the modified project, drive controlled probes, and grade state, events, logs, and interaction results.
runtime -> assertions -> score
04
Protected evaluator
Hidden harnesses, seeds, and assertions measure the delivered project without exposing benchmark answers.
task@version -> hidden score
05
Coding trajectories
Terminal commands, patches, runtime feedback, failures, retries, and costs form repair trajectories.
{ terminal, patches, feedback, score }
Asset engineering
Assets need contracts, not just prompts.
Every asset can carry its source, license, files, and runtime metadata before an Agent places it in a scene.
AssetSpec→
Manifest→
Normalize→
Validate
Machine-readable proof
Every run ends with a report.
Store it, compare it, or use failures as the next repair prompt.
{
"task_id": "platformer_basic_001",
"build_success": true,
"runtime_success": true,
"functional_score": 1,
"total_score": 1
}Roadmap
From public contract to private evaluation.
M0Runtime evaluatorAvailable now
M1Terminal SessionNext layer
M2GamePhanes-BenchAgent baselines
M3Hidden evaluationPrivate task variants
Open contract. Sealed workspace. Real runtime score.