Robomimic BC-RNN

Task goal

An agent is placed in an offline sandbox with the paper and the robosuite + robomimic environment. It must implement the paper's method or otherwise optimize the Success rate metric, then be evaluated on hidden seeds by an independent evaluator over gym-over-gRPC.

Scoring

The evaluator computes the authoritative Success rate. The harness normalizes it against the paper target:

score = clamp(measured / paper_target, 0, 1.5)

Higher is better. Beating the paper scores above 1.0, capped at 1.5. The release aggregate uses macro_mean.

Submission & evaluator

Runs are evaluated in Interactive mode. The agent drives episodes live via Reset/Step and a hidden eval session.

# request a challenge, run locally, then submit the encrypted bundle
ale-bench run --task robomimic_bcrnn
ale-bench submit robomimic_bcrnn.ale-submission.tar.zst.age