feat: push eval runs to the platform with uv run eval --push

[mikasenghaas]

Summary

• Add a --push flag to uv run eval (off by default) that uploads a finished run to the Prime Intellect platform's private Evaluations tab at the end of the eval.
• New verifiers/v1/samples.py: trace_to_sample, the canonical per-trace → platform (v0) eval-sample converter. Extracted so it is the single source of truth for the "old format" mapping and can be shared with the prime-rl monitor (see companion PR).
• New verifiers/v1/push.py: a tiny sync client (push_traces) that does the /evaluations/ flow — resolve env by name (get-or-create) → create evaluation → push samples → finalize — then logs the evaluation id. Auth + base URL reuse the existing CLI plumbing ($PRIME_API_KEY / ~/.prime/config.json, team-scoped). No new dependencies; no dependency on prime/prime-evals (would be circular).
• Hooked in at the single convergence point in verifiers/v1/cli/eval/main.py where all three execution paths (v1 in-process, --server, and v0 legacy --id) have produced list[Trace], so one hook covers every run type.

Platform record shape

Matched to what the platform's Evaluations UI reads, verified by round-tripping real pushes against api.primeintellect.ai:

• Model + counts: model, num_examples (distinct examples uploaded — config.num_tasks is None on a full run), rollouts_per_example, and the avg_reward/avg_metrics/avg_error triple go in the evaluation metadata, which the overview reads.
• Sub-rewards are flattened onto each sample under their reward-function names; env metrics stay in the nested metrics field / info.
• num_steps and the per-branch reward are not emitted (stray custom state / redundant).

Crash safety

The upload runs after the eval has finished and saved results, so push_traces wraps the HTTP flow in a try/except that logs and returns None — a network blip or platform 4xx/5xx never crashes the run (and on the non-rich path never swallows the final per-trace JSON output).

Companion PR

prime-rl's monitor pushes training-run samples through its own hand-rolled copy of this conversion. The companion PR switches it to verifiers.v1.samples.trace_to_sample so both producers stay aligned on one canonical record shape:
PrimeIntellect-ai/prime-rl#2974

Verification

Live smoke runs against api.primeintellect.ai (deepseek/deepseek-v4-flash), all COMPLETED; each with metadata.model and num_examples populated:

• v1 env (uv run eval gsm8k_v1 --push -n 4 -r 2):
  https://app.primeintellect.ai/dashboard/evaluations/mvh563t4vtgkjulr8toze5ni
• v0 legacy env (uv run eval --id gsm8k --push -n 2 -r 1):
  https://app.primeintellect.ai/dashboard/evaluations/y7v4jwd69mm2grlo9uco581e
• multi-reward env (uv run eval code-golf-v1 --push -n 2 -r 2 — 3 reward funcs correct/most_concise/fastest):
  https://app.primeintellect.ai/dashboard/evaluations/l9f7gw6m3ru8u8akz348zjku
• rlm harness (uv run eval gsm8k_v1 --harness.id rlm --push -n 2 -r 1 — harness metrics in avg_metrics + info.metrics):
  https://app.primeintellect.ai/dashboard/evaluations/fm16rw1nw6h2pe0h6vroe1nk
• agentic / container env (uv run eval terminal-bench-2-v1 --taskset.tasks '["fix-git"]' --harness.runtime.type docker --push -r 1):
  https://app.primeintellect.ai/dashboard/evaluations/e4684w1hkiw0lbz3ajmamtsl
• rlm + compaction (… --harness.id rlm --harness.summarize_at_tokens 4000 --harness.runtime.type docker --push -r 1 — compacted, 2 trajectory branches):
  https://app.primeintellect.ai/dashboard/evaluations/oprmzd1plk4srcc42zesqy2m

Without --push, behavior is unchanged. ruff check, ruff format, and tests/v1/test_configs.py pass.

[macroscopeapp]

🟡 Medium v1/push.py:77

env_name is set to config.taskset.id or config.id and sent to the platform as the environment name in the /environmentshub/resolve call. When the id is a hub-style plugin id like org/name@version, the resolve request uses that raw plugin id instead of the canonical package name, so the run gets attached to the wrong environment and won't match what prime eval push and other producers upload. Consider normalizing env_name with the same env_name(...) transformation used elsewhere in the codebase before sending it to the API.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @verifiers/v1/push.py around line 77:

`env_name` is set to `config.taskset.id or config.id` and sent to the platform as the environment name in the `/environmentshub/resolve` call. When the id is a hub-style plugin id like `org/name@version`, the resolve request uses that raw plugin id instead of the canonical package name, so the run gets attached to the wrong environment and won't match what `prime eval push` and other producers upload. Consider normalizing `env_name` with the same `env_name(...)` transformation used elsewhere in the codebase before sending it to the API.