Training environment
Also called: RL environment, agent environment
A world an AI agent can observe, act in, reset and be scored in, so it can learn by trial and error.
A training environment is not a dataset. A dataset is finished: the model reads what already happened. An environment is live: the lab’s own agent takes an action, the world answers, and the agent tries again. That loop is what agentic training needs and what a recording cannot give.
A commercial game is most of an environment already — it has rules, goals, consequences and feedback. What it lacks is headless running, time control, resets, a defined interface and automatic scoring. Adding those is the work.
Observation space
Everything the agent is allowed to see on each step: pixels, game state, or both.
Pixels mean the agent sees the screen, like a player. It works on any game, needs no engine access, and transfers to other games — but it is expensive to run and slow to learn from.
Game state means structured values: positions, inventories, resources, timers. Cheaper and faster to learn from, and well suited to planning research and language-model agents, but it needs engine access and a decision about what to expose.
Action space
Everything the agent is allowed to do on each step.
For a black-box environment this is usually keyboard and mouse: the same inputs a player has. For an engine integration it can be a discrete set of game actions, which is far easier to learn from and far easier to score.
A wide action space with many valid routes to a goal is what makes a world hard to exhaust, and is one of the three things labs pay most for.
Black-box environment
The game’s normal build runs on a server and the agent drives it through pixels, keyboard and mouse, without engine access.
A black box needs only a licence and a build, which is why it is the fastest route from a first conversation to a running pilot — weeks rather than months.
The cost is precision: resets go through save games or machine snapshots, the clock runs at real time, and determinism is approximate rather than exact.
Engine integration
A plugin inside the game project that exposes state, actions, resets and time control directly.
An engine integration gives the richest and fastest environment: native instant resets, exact determinism under a fixed seed, and play faster than real time where the engine allows it.
It needs project access, so it is a bigger commitment from the studio. A practical path is to start a pilot as a black box and move to engine integration once both sides commit.
Determinism and seeds
The ability to replay an episode exactly, by fixing the random seed and the starting state.
Without determinism a lab cannot tell whether a change in results came from the change it made or from the world behaving differently. It is the difference between an experiment and an anecdote.
Seeds do double duty: fixed for repeatability, varied for generalisation. The same mechanism that lets a lab repeat a run lets it generate a world the agent has never seen.
Episode
One run from reset to finish, success or failure, from which the agent learns.
Throughput is usually counted in episodes per second per core, and it is the number that decides how much training a given budget buys. It is why faster-than-real-time play is worth paying for.
Reward hacking
The agent finds a way to score well that does not do the thing the score was meant to measure.
An agent asked to deliver loads quickly may learn to clip through scenery; an agent scored on profit may learn to exploit a pricing bug. The behaviour is correct by the rules it was given and useless by the intent behind them.
This is the known weakness of training environments, and the reason task design is a craft rather than a configuration step. Scoring has to reward the behaviour you want and be hard to game.
Distillation risk
The risk that a provider serving several labs learns how one lab’s models behave and passes that knowledge on.
For a frontier lab, how its agents behave is the research. A shared environment provider sits in exactly the position to observe it: it sees the trajectories, the failure modes and the rate of improvement.
The answer is architectural rather than contractual. Physically separate hardware per lab, no stored agent data, access keys held by the lab, and licensing directly between lab and studio so no catalogue aggregates several labs’ activity in one place.
Environment licence
The agreement under which a studio lets one lab use a private copy of its game for training, evaluation, or both.
It is narrower than a publishing licence and wider than an NDA. It names the titles and versions, where the build may run, whether the lab may modify it, who owns the models and trajectories that come out, and what happens to every copy at the end of the term.
Training and evaluation are different uses and should be named separately: a lab that may train on your game has not automatically bought the right to benchmark competitors on it.
Gameplay data licensing
Selling recorded play as training data — a different product from an environment.
Recorded gameplay is passive: the model learns from what human players did. An environment is active: the lab’s agent plays itself and learns from its own mistakes.
Both markets exist and a studio can be in both, but they are priced and contracted differently. Recordings are a one-off asset sale; an environment is a service with recurring revenue attached to it.
Concurrent agent
One instance of the environment running for one agent at one moment — the usual unit of usage pricing.
Labs do not train one agent at a time; they run hundreds or thousands of instances in parallel. Pricing per concurrent agent is what makes an environment a recurring revenue line rather than a fee, because it scales with how hard the lab is training.
Headless build
A build that runs with no screen, so many copies fit on one server.
Games render for a human watching. Training does not need the picture drawn to a display, and drawing it wastes the GPU time the lab wants to spend on the agent. Headless running is the first step in making any game agent-ready.
World model
A model that learns to predict how a world behaves, so an agent can plan inside its own prediction.
World-model research is one of the main reasons labs want realistic games rather than toy worlds: the model has to learn physics, consequence and time from something that behaves like reality.
It is also why pixel observation keeps its value despite the cost — a world model that learns from video transfers to worlds that were never instrumented.