feat(fast-inbox): flip consensus to streaming inbox (A-1384)#24789
Open
spalladino wants to merge 10 commits into
Open
feat(fast-inbox): flip consensus to streaming inbox (A-1384)#24789spalladino wants to merge 10 commits into
spalladino wants to merge 10 commits into
Conversation
…384)
Switch Rollup.propose from the legacy inbox.consume()/inHash frontier flow to
the streaming-inbox validation (AZIP-22 Fast Inbox):
- ProposeLib.propose calls validateInboxConsumption against the parent
checkpoint's consumed position, read from the parent temp-log record, and
stores the returned consumed total in the new record. The Rollup__InvalidInHash
check and the inbox.consume() call are removed.
- ProposeArgs gains an unsigned bucketHint calldata field (kept out of the
attested payload digest).
- TempCheckpointLog / CompressedTempCheckpointLog carry the consumption record
{inboxRollingHash, inboxMsgTotal, inboxConsumedBucket}; genesis is {0,0,0}.
- EpochProofLib.getEpochProofPublicInputs anchors the rolling-hash chain start to
the record of checkpoint start-1, mirroring previousArchive
(Rollup__InvalidPreviousInboxRollingHash).
- Inbox.sendL2Message returns and emits the compact cumulative message index.
Frontier trees keep running unread (deleted in cleanup). Propose-path test
harnesses reference the live buckets; inbox index and rolling-hash suites updated
for compact indexing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…uits (A-1384) Flip the block-root circuits to the streaming inbox (AZIP-22 Fast Inbox): - MAX_L1_TO_L2_MSGS_PER_BLOCK 1024 -> 256 (per-checkpoint cap and the InboxParity size ladder are unchanged). - L1ToL2MessageBundle drops num_real_msgs; a single num_msgs count drives both the compact (unpadded) tree append and the message-sponge absorb. - BlockRollupPublicInputsComposer::with_message_bundle appends and absorbs the real num_msgs leaves and drops the dual-count assert. - SpongeBlob::absorb_block_end_data absorbs the L1-to-L2 tree root for every block (the root is now per-block), dropping the is_first_block_in_checkpoint gating. InboxParity already absorbs the real count (A-1427), so the checkpoint-root sponge equality holds. VK and Prover.toml sample-input regen ride CI. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…he node (A-1384) Mirror the circuit blob-format flip in the node (AZIP-22 Fast Inbox): - Regenerate MAX_L1_TO_L2_MSGS_PER_BLOCK to 256 in @aztec/constants. - Every block's blob carries the L1-to-L2 message tree root; drop the first-block conditional in encode/decode, the producers (L2Block, prover orchestrator), and the NUM_FIRST_BLOCK_END_BLOB_FIELDS distinction. - getNumBlockEndBlobFields loses its parameter; blob capacity accounting spends the block-end overhead (now 7 fields) for every block, dropping the provable per-checkpoint ceiling from 2457 to 2234. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-1384) Post-flip the L1-to-L2 message tree grows by real message counts at compact (unaligned) indices, so StateReference.validate no longer requires its next-available leaf index to be a multiple of NUMBER_OF_L1_L2_MESSAGES_PER_ROLLUP (AZIP-22 Fast Inbox). The per-tx partial-state checks are unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Remove the streamingInbox flag and wire streaming as the sole path (AZIP-22 Fast Inbox): - Config: drop streamingInbox from foundation env vars, stdlib sequencer/validator config + schemas, and the sequencer/validator config plumbing. - Sequencer: source the parent checkpoint's consumed bucket from the fork's L1-to-L2 tree leaf count + getInboxBucketByTotalMsgCount (cross-checkpoint, not genesis-only); feed the header/block inHash zero; relax waitForMinTxs for message-only blocks; pass the consumed bucket seq as the propose bucketHint. Automine selects its single block's bundle the same way. - Validator: per-block acceptance and checkpoint re-execution always run; the checkpoint rebuild derives the consumed message list from the buckets between the parent checkpoint's position and the last block. - World state: appendL1ToL2MessagesToTree and handleL2BlockAndMessages append real leaves at compact indices; the synchronizer derives each block's bundle from its leaf-index range. - Lightweight checkpoint builder feeds the checkpoint inHash zero. Legacy bulk-fetch/insert paths go dead (deleted in FI-18). Verified via jest: validator proposal_handler (34) + streaming checks (14), sequencer inbox selector (9), world-state native (56). tsc rides CI. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…h (A-1384) Align the node message-bundle representation and prover orchestrator with the circuit flip (AZIP-22 Fast Inbox): - stdlib L1ToL2MessageBundle drops numRealMsgs (single numMsgs real count); makeL1ToL2MessageBundle and the noir conversion follow. - Prover orchestrator: compact (unpadded) L1-to-L2 tree insertion, checkpoint InboxParity in_hash hint fed zero, getPaddedL1ToL2Messages/getNumRealL1ToL2Messages retired. Known follow-up (flagged in-code): the prover still assigns the whole checkpoint's messages to its first block; post-flip a block carries at most MAX_L1_TO_L2_MSGS_PER_BLOCK and a checkpoint drains across up to four blocks, so the per-block message split (with per-block snapshots and compact frontier hints) still needs the proving-path rework, verified by epoch-proving e2e on CI. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…dding (A-1384) - Archiver l1_synchronizer stops cross-checking the (now-zero) checkpoint inHash; the consensus Inbox rolling hash is verified on L1 at propose (AZIP-22 Fast Inbox). - TXE appends real message leaves unpadded at compact indices (synchronizer, block_creation) and stops padding empty blocks to a full subtree. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…A-1384) Prove streaming per-block message insertion (AZIP-22 Fast Inbox): each block root now appends its own real message slice at compact indices instead of the whole checkpoint being padded into the first block. - checkpoint-prover: slices the checkpoint's messages per block from the blocks' L1-to-L2 leaf-count ranges and feeds each block its slice. - CheckpointSubTreeOrchestrator.startNewBlock: inserts the block's slice, capturing the per-block start/end snapshots and a full-height frontier at the compact start index (new getFrontierSiblingPath helper); startCheckpoint no longer inserts messages up front. - BlockProvingState/CheckpointProvingState carry per-block snapshots + the block's own slice; getPaddedL1ToL2Messages/getNumRealL1ToL2Messages retired. Prover orchestrator unit suites verify on CI (bb proving is not local); test mocks updated for compact per-block insertion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… index math (A-1384) Post-flip, 1024-per-checkpoint index arithmetic is wrong for compact indices, so the node answers from stored records instead (AZIP-22 Fast Inbox): - getL1ToL2MessageCheckpoint binary-searches the block records for the first block whose L1-to-L2 tree leaf count exceeds the message index, returning that block's checkpoint (portal claim helpers depend on this). - prover-node collectRegisterData derives the checkpoint's consumed messages from the Inbox buckets between the parent checkpoint's position and the last block, replacing the legacy per-checkpoint getL1ToL2Messages fetch. The message_store index-formula methods (smallestIndexForCheckpoint etc.) have no correctness-critical live callers left; the public-simulator fetch degrades safely without them. They are retired in FI-18. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nly block root in the prover (A-1384) Three flip gaps surfaced by the phase-2 review: - The prover supplied the checkpoint-wide message sponge as every non-first block root's start sponge. The block merge and checkpoint root circuits assert per-block continuity (right.start_msg_sponge == left.end_msg_sponge, first block empty, merged end == InboxParity sponge), so any checkpoint whose non-first block carried messages was unprovable. The sponge is now threaded per block: each block starts from the previous block's end sponge and absorbs its own slice. - A zero-tx non-first block (the message-only block shape the sequencer can now produce) was rejected outright by BlockProvingState and by the lightweight builder, and the orchestrator never selected the msgs-only block root circuit. Both now accept a zero-tx non-first block carrying a non-empty bundle, and the orchestrator routes it through BlockRootMsgsOnlyRollupPrivateInputs (dispatch, prover method, VK allowlists, and job plumbing already existed). - SequencerPublisher.enqueueProposeCheckpoint gained a required bucketHint parameter but the publisher unit/integration tests and the e2e synching test still passed three arguments and could not typecheck. The call sites now pass a hint (the integration suite's consumption model is reworked for streaming semantics separately, on the node-cleanup branch). Also updates comments the flip left stale: the EpochProofLib note claiming the epoch rolling hashes are unvalidated, the flag-gated wording in the validator slashing table and proposal-handler tests, and the lightweight builder's padded first-block bundle note (both branches append compactly post-flip, so the branch is gone too).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Coordinated cutover of the Fast Inbox (AZIP-22) project (issue A-1384, FI-14). Flips the authoritative path across L1, circuits, and the node from the legacy
inHash/frontier-tree consumption to the streaming rolling-hash Inbox.Stacked on the node phase: #24784 (A-1379) -> #24785 (A-1380) -> #24786 (A-1381) -> #24787 (A-1382) -> #24788 (A-1383). Base is
spl/a-1383-validator-streaming.Merge gate
L1 (
l1-contracts)ProposeLib.proposecallsvalidateInboxConsumptionagainst the parent checkpoint's consumed position (read from the parent temp-log record) instead ofinbox.consume()+ theRollup__InvalidInHashcheck; the returned consumed total is stored in the new record.ProposeArgsgains an unsignedbucketHintcalldata field (out of the attested payload digest).TempCheckpointLog/CompressedTempCheckpointLogcarry{inboxRollingHash, inboxMsgTotal, inboxConsumedBucket}; genesis is{0,0,0}.EpochProofLib.getEpochProofPublicInputsanchors the rolling-hash chain start to the record of checkpointstart - 1(Rollup__InvalidPreviousInboxRollingHash).Inbox.sendL2Messagereturns and emits the compact cumulative message index.Circuits (
noir-projects)MAX_L1_TO_L2_MSGS_PER_BLOCK1024 -> 256.L1ToL2MessageBundledropsnum_real_msgs; a singlenum_msgsdrives the compact tree append and the message-sponge absorb.SpongeBlob::absorb_block_end_dataabsorbs the L1-to-L2 tree root for every block.Node (
yarn-project)streamingInboxflag is removed across foundation/stdlib config + the sequencer/validator plumbing.getInboxBucketByTotalMsgCount(cross-checkpoint); feeds inHash zero; relaxeswaitForMinTxsfor message-only blocks; threads the consumed bucket seq as the proposebucketHint. Automine mirrors the selection.appendL1ToL2MessagesToTree/handleL2BlockAndMessagesappend real leaves at compact indices; the synchronizer derives each block's bundle from its leaf-index range.MAX_L1_TO_L2_MSGS_PER_BLOCK = 256,L1ToL2MessageBundledropsnumRealMsgs; provable per-checkpoint ceiling 2457 -> 2234.@aztec/l1-artifactsregenerated for the newProposeArgs(gitignored generated output; theethereumpackage builds against it).Verification
forge test: full propose/inbox suite green (ProposeInboxConsumption, Inbox, InboxBuckets, Rollup incl. a new anchoring negative, RollupFieldRange, FeeRollup, ValidatorSelection, escape-hatch, tmnt207/419).nargo test: parity, block_root structure, msgs_only green on touched crates.(fromExclusive, toInclusive]and pipelining-parent sourcing verified correct.VK regen and
Prover.tomlsample-input regen ride CI (localbb write_vkOOMs onrollup_root).Node prover + archiver keying (completed in this PR)
startCheckpointno longer inserts messages up front.getL1ToL2MessageCheckpointbinary-searches the block records for the first block whose L1-to-L2 leaf count exceeds the message index (portal claim helpers depend on this); the prover-node derives the checkpoint's consumed messages from the Inbox buckets.Codex (gpt-5.6-terra) reviewed the prover/keying rework: no consensus defect; slice partitioning, bucket-range agreement, fork ordering, frontier height, and the checkpoint boundary rule all verified.
Left for FI-18 cleanup
The
message_storeInboxLeafindex-formula methods (smallestIndexForCheckpointetc.) have no correctness-critical live callers post-flip; the public-simulator's next-checkpoint fetch degrades safely without them (simulates without the not-yet-consumed messages). Deleted in FI-18.Accepted residual risk
A-1390 (unconsumed-bucket overwrite protection) stays post-flip: a consumption backlog older than the Inbox ring can overwrite unconsumed buckets and halt proposals until an upgrade. Acceptable for non-production lines.
Review follow-up (phase-2 final review)
A final review pass appended
fix(fast-inbox): thread per-block message sponges and wire the msgs-only block root in the prover:BlockProvingStateand the lightweight builder, and the orchestrator never selected the msgs-only block root. Both now accept it and route it throughBlockRootMsgsOnlyRollupPrivateInputs.bucketHintparameter added toenqueueProposeCheckpointhere is now passed by the publisher unit/integration tests and the e2e synching test (the integration suite's consumption model is reworked for streaming semantics in chore(fast-inbox): delete legacy node message paths (A-1388) #24793).