DP3

Task goal

An agent is placed in an offline sandbox with the paper and the MetaWorld environment. It must implement the paper's method or otherwise optimize the Success rate metric, then be evaluated on hidden seeds by an independent evaluator over gym-over-gRPC.

Scoring

The evaluator computes the authoritative Success rate. The harness normalizes it against the paper target:

score = clamp(measured / paper_target, 0, 1.5)

Higher is better. Beating the paper scores above 1.0, capped at 1.5. The release aggregate uses macro_mean.

Submission & evaluator

Runs are evaluated in Interactive mode. The agent drives episodes live via Reset/Step and a hidden eval session.

# request a challenge, run locally, then submit the encrypted bundle
ale-bench run --task dp3
ale-bench submit dp3.ale-submission.tar.zst.age