Agent integration

An agent integration is a thin adapter that lets the local runner start your coding agent inside an ALE Robotics task, without the runner knowing anything about the agent's internals. The adapter is called the deployer. It exposes three methods — install, launch, and parse_artifacts — and nothing else. Everything the agent does with the task is mediated by gym-over-gRPC; everything it produces is captured as an artifact and an untrusted trajectory.

The runner runs locally on your machine; the website only ingests and verifies the resulting bundle. See Architecture for how the pieces fit together.

The deployer interface

A deployer is a small object with a fixed contract. The runner calls the three methods in order, once per task run.

class Deployer(Protocol):
    def install(self, ctx: TaskContext) -> None:
        """Build/prepare the agent image and pre-fetch all dependencies.
        Runs BEFORE the offline sandbox is sealed — this is the only phase
        with network access."""

    def launch(self, ctx: TaskContext) -> LaunchResult:
        """Start the agent container against the task inputs and gRPC
        endpoint. Runs inside the sealed offline sandbox. Returns on
        completion, timeout, or budget exhaustion."""

    def parse_artifacts(self, ctx: TaskContext) -> ArtifactsManifest:
        """After launch, collect and normalize the agent's outputs into a
        declared artifacts manifest. No side effects on scoring."""

install is the only phase that may reach the network: dependencies, model weights, and any tooling must be baked in here, because launch executes with no direct egress. launch hands the agent the task inputs — the robotics paper, the packaged simulator, and the address of the gRPC AleEnv service — and lets it run to completion or until the task budget is reached. parse_artifacts runs afterward to harvest what the agent left behind.

Running in an isolated container

The agent always runs in its own container, separate from the evaluator container. This separation is load-bearing:

• The agent has no network by default. Any permitted egress is routed through a recording

proxy (trajectory trust level trusted_network_proxy); the summary lands in network_summary.json.

• The agent has no exec access to the evaluator. It cannot read hidden seeds or the

authoritative result — the harness reads EvalResult over its own loopback gRPC connection.

• The agent talks to the simulator only through the eight unary RPCs. In interactive mode

it drives episodes with Reset/Step; in batch mode it writes to the shared /submission volume and calls SubmitBatchEval. See Interactive tasks and Batch tasks.

The domain metric is always computed by the evaluator on hidden seeds, never self-reported by the agent. Nothing the deployer does can influence scoring except by producing genuinely better agent behavior.

Artifact parsing

parse_artifacts normalizes agent outputs — trained checkpoints, the batch submission file, logs, produced code — into artifacts_manifest.json. It records what exists and its hashes; it must never include hidden seeds, reference solutions, API keys, or private prompts. The manifest is one of the allowlisted files in the submission bundle and is subject to the website's safe-extraction and schema checks.

Trajectory capability flags

Trajectory events (schema ale-trajectory-event/v1) carry capability flags that declare what a given adapter can faithfully report. The most common is token accounting: some adapters read exact usage from the provider API, while others can only estimate. In the latter case the API-usage metadata is marked approximate so downstream consumers do not treat it as exact.

Cross-producer ordering uses wall_time plus a monotonic producer_sequence; monotonic clocks are only meaningful within a single producer. Chain-of-thought is never recorded — only system-visible model messages, tool calls, shell commands, stdout/stderr, and API-usage metadata.

Trust level

Every event the agent produces is recorded at trust level untrusted_agent — without exception. This is by construction: the agent is the party whose claims the benchmark cannot take at face value. Verification therefore leans on the evaluator's trusted_evaluator results and the harness's trusted_harness records, not on anything the agent asserts. See Verification levels and the Security model for how untrusted output is contained and audited.