CrowdNav SARL

Task goal

An agent is placed in an offline sandbox with the paper and the CrowdSim environment. It must implement the paper's method or otherwise optimize the Success rate metric, then be evaluated on hidden seeds by an independent evaluator over gym-over-gRPC.

Scoring

The evaluator computes the authoritative Success rate. The harness normalizes it against the paper target:

score = clamp(measured / paper_target, 0, 1.5)

Higher is better. Beating the paper scores above 1.0, capped at 1.5. The release aggregate uses macro_mean.

Submission & evaluator

Runs are evaluated in Interactive mode. The agent drives episodes live via Reset/Step and a hidden eval session.

# request a challenge, run locally, then submit the encrypted bundle
ale-bench run --task crowdnav_sarl
ale-bench submit crowdnav_sarl.ale-submission.tar.zst.age