BARN DWA

Task goal

An agent is placed in an offline sandbox with the paper and the BARN / Gazebo environment. It must implement the paper's method or otherwise optimize the Success rate metric, then be evaluated on hidden seeds by an independent evaluator over gym-over-gRPC.

Scoring

The evaluator computes the authoritative Success rate. The harness normalizes it against the paper target:

score = clamp(measured / paper_target, 0, 1.5)

Higher is better. Beating the paper scores above 1.0, capped at 1.5. The release aggregate uses macro_mean.

Submission & evaluator

Runs are evaluated in Batch mode. The agent writes a submission to a shared volume; the evaluator runs its native pipeline.

# request a challenge, run locally, then submit the encrypted bundle
ale-bench run --task barn_dwa
ale-bench submit barn_dwa.ale-submission.tar.zst.age