Skip to content

refactor: share trace_to_sample with verifiers in the prime monitor#2974

Merged
mikasenghaas merged 14 commits into
mainfrom
refactor/monitor-shared-sample
Jul 9, 2026
Merged

refactor: share trace_to_sample with verifiers in the prime monitor#2974
mikasenghaas merged 14 commits into
mainfrom
refactor/monitor-shared-sample

Conversation

@mikasenghaas

@mikasenghaas mikasenghaas commented Jul 7, 2026

Copy link
Copy Markdown
Member

Summary

  • Replace the prime monitor's hand-rolled v1-trace → sample conversion in _rollouts_to_parquet_bytes with verifiers.v1.push.trace_to_sample — the single canonical "old format" converter, now shared with verifiers' eval uv run eval --push.
  • Both producers (a training-run sample push here, and an eval --push / prime eval push in verifiers) now emit one identically-shaped sample record, so the platform renders them consistently and the conversion logic can't drift between the two repos.
  • The RFT-only parquet columns (run_id/step/advantage/problem_id/env_name) are layered on here; the shared function owns the conversation/branch/trajectory extraction.
  • Bumps the deps/verifiers submodule to the companion verifiers PR that adds trace_to_sample.

Behavior

Behavior-preserving: the _SAMPLE_SCHEMA and every column value are unchanged. The existing tests/unit/utils/test_prime_monitor.py asserts (problem_id/sample_id/completion/trajectory) still hold; I verified the refactored mapping produces identical num_input_tokens/num_output_tokens (still branches[-1]) and completion/trajectory content against those cases. The only nuance: messages now serialize with exclude_none=True (drops null fields like tool_calls: null) — smaller strings, same content.

Companion PR

Companion to the verifiers PR that introduced verifiers.v1.push.trace_to_sample (merged):
PrimeIntellect-ai/verifiers#1947

The deps/verifiers submodule is pinned to that PR's squash-merge commit on verifiers main (9c6b155e). Validated end-to-end with a 20-step reverse_text RL run on this branch (reward 0.78 at step 20, 0 errors).


Note

Low Risk
Refactor of monitor upload serialization with intended behavior parity; tests cover key columns; only minor JSON null-field omission differs.

Overview
Prime monitor sample uploads now build each parquet row from verifiers.v1.push.trace_to_sample instead of inline branch/message serialization, aligning training-run pushes with verifiers eval --push.

_rollouts_to_parquet_bytes still owns _SAMPLE_SCHEMA and RFT-only fields (run_id, step, advantage, problem_id, env_name, etc.). It maps example_idproblem_id, injects advantage on every trajectory branch, and reads token counts from the last branch. Empty trajectories are still skipped.

completion, task, reward, metrics, and timing come from the shared sample dict (JSON-encoded where before). Message JSON may omit null fields via exclude_none=True in verifiers—slightly smaller payloads, same semantics per PR notes.

Reviewed by Cursor Bugbot for commit 130bd1d. Bugbot is set up for automated code reviews on this repo. Configure here.

Replace the monitor's hand-rolled v1-trace -> sample conversion in
`_rollouts_to_parquet_bytes` with `verifiers.v1.samples.trace_to_sample`, the same
canonical converter verifiers' eval `uv run eval --push` uses. Both producers now emit
one identically-shaped sample record; the RFT-only parquet columns (run/step/advantage/
problem_id/env_name) are layered on here. Behavior-preserving: the parquet schema and all
column values are unchanged (verified against the existing monitor unit tests).

Bumps the deps/verifiers submodule to the verifiers PR that adds `trace_to_sample`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
mikasenghaas and others added 13 commits July 7, 2026 23:21
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ixing)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
samples.py was dropped on the verifiers branch - trace_to_sample now
lives in verifiers.v1.push

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mikasenghaas mikasenghaas requested a review from samsja July 8, 2026 23:37
@mikasenghaas mikasenghaas marked this pull request as ready for review July 8, 2026 23:37
@mikasenghaas mikasenghaas merged commit 2bc136c into main Jul 9, 2026
11 of 22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants