Security model

Scope

This page describes the security model of the self-hostable verification website, not the offline agent sandbox. The website receives encrypted submission bundles, decrypts them server-side, validates them, and publishes a verification level. Everything here applies to any deployment, including your own via Deployment.

The guiding principle: the site is not a benchmark runner. It never executes agent code. It only inspects a fixed set of allowlisted, non-executable artifacts and recomputes scores from the evaluator's own results.

Key custody

Submission bundles are age(zstd(tar)) using age (X25519) via the audited pyrage library — never hand-rolled crypto. Key custody is split:

• The local runner holds ONLY the website public key (served by GET /api/v1/submission-key) and encrypts to it.
• The website holds the private key. It is loaded ONLY from a secret mount at runtime. It is NEVER committed to the repository, baked into a container image, written to logs, or exposed to the browser.

There is no API that returns the private key or decrypted plaintext to a client.

Server-side processing order

Decryption and validation follow a fixed, fail-closed pipeline (see Submission bundle for the bundle contract):

receive ciphertext -> sha256 -> decrypt with private key ->
enforce decompressed size / file-count limits ->
safe tar inspect -> extract allowlisted files only ->
validate JSON schemas -> verify challenge token + nonce ->
verify hash chains -> verify integrity root ->
verify benchmark/task versions + coverage ->
recompute scores -> compare recomputed vs claimed ->
audit summary -> persist metadata ->
delete plaintext + ciphertext temp files

Successful decryption does NOT grant verification. See Verification levels.

Safe archive extraction

Before any file is written to disk, the tar is inspected and rejected if it contains: path traversal (..), absolute paths, symlinks, hardlinks, device nodes, or FIFOs. Decompression-bomb defenses enforce a maximum decompressed size and file count. Only the allowlisted members are extracted; anything else is discarded.

No code execution

Extracted artifacts are pure data: JSON and length-delimited JSONL. The pipeline performs no code execution, no Python pickle loading, and no unsafe YAML parsing. Schemas are validated with strict parsers so unexpected fields or types are rejected rather than interpreted.

Anti-replay, rate limits, and upload caps

Each submission is bound to a server-issued challenge from POST /api/v1/challenges, which returns a one-time nonce and a server-signed challenge_token with issued_at/expires_at. POST /api/v1/submissions is accepted only if the token verifies, is unexpired, and the nonce has not been used before. Uploads are multipart with a 20 MB default cap. Rate limits apply to challenge issuance and submission endpoints.

Web hardening

The web layer enforces CSRF protection on state-changing requests and sends standard security headers (HSTS via Caddy auto-HTTPS, Content-Security-Policy, X-Content-Type-Options, X-Frame-Options, referrer policy). Login is via GitHub OAuth; owner/admin scoping controls who can see a submission's private status.

Logging and retention

Logging is structured, with secret redaction: private keys, tokens, nonces, and OAuth credentials are never emitted, and log fields are filtered rather than logged verbatim. By default the site deletes both plaintext and ciphertext temp files after processing (KEEP_SUBMISSION_BUNDLES=false); only validated metadata and score breakdowns persist. Bundles themselves never contain hidden seeds, reference solutions, API keys, private prompts, hidden reasoning, or unredacted full trajectories.

What self-hosted trajectories cannot prove

Honesty is by construction, not by trust. A validated result proves the bundle decrypts, matches its schemas, has intact hash chains and integrity root, covers the required tasks at declared versions, and that recomputed scores match the claimed ones. It does NOT prove that the local runner was unmodified. A self-hosted trajectory cannot establish provenance of the execution environment, cannot rule out a tampered runner, and the backend refuses any attempt to self-elevate to audited or officially_verified via upload fields. Only officially_verified — from a benchmark-team-controlled runner, an approved third-party or hardware-attested runner, or an official rerun — carries that guarantee.