serviceability: Rust instruction-builder library (SPL-style, one builder per instruction) EVALUATION

[juan-malbeclabs]

Context

There is currently no "pure" way to build instructions for the doublezero-serviceability program. The assembly logic (PDA derivation, AccountMeta ordering, borsh packing) is coupled to execution: every smartcontract/sdk/rs/src/commands/<domain>/<verb>.rs has an XxxCommand::execute() that builds and signs+sends the transaction. There is no reusable build_xxx(args) -> Instruction.

The goal is the classic Solana pattern (SPL spl-token::instruction::transfer(...)): one pure function per instruction that takes already-resolved arguments and returns an unsigned Instruction, with no RPC. The caller composes the Transaction, adds the compute-budget prelude, the blockhash, and signs.

Scope decisions:

• Rust only for now (Go is a later phase, out of scope here).
• Return: a pure Instruction, SPL-style (not a Vec, not a Transaction).
• No RPC: offline pure builders; the caller passes chain-derived values (globalstate indices, dz_prefixes count) as explicit parameters.
• Coverage: all buildable DoubleZeroInstruction variants (~94 of the 116; deprecated placeholders are omitted).

Source of truth for the wire format: the DoubleZeroInstruction enum (variants 0–115) and the per-processor *Args structs. Encoding = 1 tag byte + borsh(args); DoubleZeroInstruction::pack() = borsh::to_vec.

Wire conventions (critical)

1. Each builder returns a single Instruction whose account list is in the exact order the processor consumes them (next_account_info), including the trailing [payer(signer,writable), system_program].
2. The 2 ComputeBudget instructions prelude (CU limit 1_400_000, heap 256 KiB) is transaction-level → exposed as a separate helper (compute_budget_prelude()), not inside each builder.
3. The Permission PDA is derived from the payer inside the builder (get_permission_pda(program_id, payer)), never a caller-supplied parameter. It is appended read-only as the last account only once that instruction's authorize() migration is activated; until then the append ships commented out in common.rs and the builder emits no Permission account (exactly what the pre-migration processor expects). This tracks the program's incremental authorize() migration.
4. Length-detected family: CreateUser (36), DeleteUser (42) and CreateSubscribeUser (59) length-detect the optional trailing account (tenant/feed) via accounts.len(). These builders never append a Permission account, permanently, so a trailing Permission cannot corrupt the count — the hazard is unrepresentable.
5. Counts and variable accounts (resource_count/dz_prefixes) are derived by the builder from its own loop, overwriting the count field in the Args, so they can never disagree.

Location: new crate under crates/

Create crates/doublezero-serviceability-instruction/, crate doublezero_serviceability_instruction (matching the existing convention for standalone workspace crates, e.g. doublezero-cli-core). Acyclic graph:

doublezero-serviceability (enum, Args, pda, resource types)
      ^
doublezero_serviceability_instruction   <- PURE builders (only solana-program + doublezero-program-common; NO solana-client/tokio/backon)
      ^
doublezero_sdk (sdk/rs)  -> commands/* delegate to the builders

Purity is enforced by the dependency graph: the crate cannot touch RPC. External consumers (bots, indexers, the fixture generator) can build instructions without pulling the heavy RPC tree. Same split as spl-token. Rejected alternatives: a builders/ module inside doublezero_sdk (loses the isolation) or inside the on-chain program (BPF binary bloat + unnecessary review burden).

Layout (one module per domain, mirroring processors/ and commands/):

crates/doublezero-serviceability-instruction/
  Cargo.toml
  src/lib.rs        # re-exports domains; compute_budget_prelude(); consts; documented list of excluded deprecated variants
  src/common.rs     # build() — the single place that knows the trailing payer/system/permission convention
  src/device.rs  link.rs  user.rs  location.rs  exchange.rs  contributor.rs
  src/multicastgroup.rs  tenant.rs  permission.rs  topology.rs  feed.rs
  src/accesspass.rs  resource.rs  globalstate.rs  globalconfig.rs  allowlist.rs
  src/index.rs  migrate.rs

common.rs (anti-drift keystone):

pub(crate) fn build(
    program_id: &Pubkey,
    instruction: DoubleZeroInstruction,
    mut accounts: Vec<AccountMeta>,   // instruction-specific metas in processor order, WITHOUT payer/system
    payer: &Pubkey,
) -> Instruction {
    accounts.push(AccountMeta::new(*payer, true));
    accounts.push(AccountMeta::new(solana_system_interface::program::id(), false));

    // Permission PDA (authorize()) — derived from the payer, not passed in.
    // Left commented until the instruction's authorize() migration is activated:
    //
    // let (permission, _) = get_permission_pda(program_id, payer);
    // accounts.push(AccountMeta::new_readonly(permission, false)); // must be last

    Instruction::new_with_bytes(*program_id, &instruction.pack(), accounts)
}

pub fn compute_budget_prelude() -> [Instruction; 2] { /* CU 1_400_000 + heap 256KiB */ }

Canonical signature

Rule: offline-derivable PDAs are derived inside (including the payer-derived Permission PDA — never a parameter); non-derivable external accounts (contributor, location, exchange, device, mgroup, accesspass, side_a/z, feed, tenant) are passed as &Pubkey; RPC-derived values (account_index: u128, dz_prefix_count: u8) are passed as explicit scalars. Returns an infallible Instruction. Infallible normalization that affects the wire (e.g. code.make_ascii_lowercase()) may happen in the builder; fallible validation (charset/length) stays in the caller.

// device.rs — variable dz_prefix blocks (builder fixes args.resource_count)
pub fn create_device(program_id: &Pubkey, payer: &Pubkey,
    contributor: &Pubkey, location: &Pubkey, exchange: &Pubkey,
    account_index: u128, mut args: DeviceCreateArgs) -> Instruction;

// link.rs — fixed accounts
pub fn create_link(program_id: &Pubkey, payer: &Pubkey,
    contributor: &Pubkey, side_a: &Pubkey, side_z: &Pubkey,
    link_index: u128, args: LinkCreateArgs) -> Instruction;

// user.rs — length-detected feed → appended before payer/system
pub fn create_subscribe_user(program_id: &Pubkey, payer: &Pubkey,
    device: &Pubkey, mgroup: &Pubkey, accesspass: &Pubkey,
    dz_prefix_count: u8, feed: Option<Pubkey>, args: UserCreateSubscribeArgs) -> Instruction;

Variable-account instructions

• dz_prefix blocks (create_device, create_subscribe_user): loop 0..count deriving ResourceType::DzPrefixBlock(entity, idx); the count is written back into the Args.
• Length-detected optional trailing (tenant on user create/delete, feed on subscribe): appended conditionally before payer/system; these builders never append a Permission account.
• split_trailing_permission family (link/delete, user/update, interface/update, topology/assign, mcast allowlists): variable list, then payer/system, then — once migrated — the payer-derived Permission PDA last (the same commented-until-activated append; safe because the processor peels it by PDA match).
• Batched instructions (clear_topology, assign_topology_node_segments): a single-chunk builder plus a *_batched(...) -> Vec<Instruction> convenience, with the batch-size consts moved into this crate (the 32-account cap math includes the trailing accounts the builder now owns).

Migrating commands/* (final, riskiest phase)

• Add to the DoubleZeroClient trait: send_transaction(Instruction) -> Result<Signature> (+ quiet variant). The permission cache (note_transaction_sent/invalidation) stays; since migrated builders derive the Permission PDA internally, the command layer no longer resolves or passes a permission pubkey.
• client.rs::assemble_instructions stops appending payer/system (that now lives in common::build); it only prepends the compute-budget prelude over a pre-built Instruction. This removes the double-append risk behind previous regressions.
• Each execute() becomes: validate → resolve RPC values → call the builder → client.send_transaction(ix). The mockall command tests change shape (assert on the built Instruction, now including payer/system).

Anti-drift

1. common::build centralizes the trailing convention (a single place).
2. A doc-comment with the account layout copied verbatim from the processor (processors already carry these, e.g. device/create.rs).
3. solana-program-test integration tests in programs/doublezero-serviceability/tests/: for each buildable variant, build via the builder and run it against the real in-process program; if the account order is wrong the processor fails. This is the refactor's safety net. No proc-macro (it would hide the account order reviewers must see).

Testing and golden fixtures

Extend the Rust generator sdk/serviceability/testdata/fixtures/generate-fixtures/src/main.rs (which already emits user_create_args/user_delete_args), adding the builder crate as a dependency and emitting, per instruction, with deterministic inputs and a fixed program_id:

• ix_<name>.bin = instruction.data (tag + borsh) → the wire bytes.
• ix_<name>.json = { variant, data_hex, accounts: [{pubkey, is_signer, is_writable}] } → captures account order + flags.

Consumers:

• Rust: unit tests in the crate porting the existing mockall expectations (which assert the exact Vec<AccountMeta>, see device/create.rs tests) onto Instruction.accounts, plus data[0] == <tag>.
• The fixtures remain available to verify Go/Python/TS parity in the future (out of scope now). CI fails on uncommitted fixture diffs (make generate-fixtures).

Priority: variable-account builders + the create-family (20, 28, 59) + topology (109, 110), then the rest.

Rollout (respecting the ~500 lines of new code per PR norm, tests excluded)

Tracked as sub-issues below. High level:

• R0 — scaffold + pattern: new crate, workspace member, common.rs, lib.rs, and 4–5 exemplar builders (one per family). Unit tests + first ix_* fixtures.
• R1..Rn — one domain group per PR. Deprecated variants omitted and documented in lib.rs (no Err/panic stubs: builders are infallible).
• Final PR — migrate commands/* to delegate to the builders + add trait methods + simplify assemble_instructions. Safety net: the solana-program-test suite.

Risks and mitigations

• Account-order drift (biggest risk): solana-program-test tests + verbatim doc-comments.
• Trailing-convention regression: centralized common::build; after migration a single place appends payer/system.
• Permission on length-detected instructions: those builders never append a Permission account (unrepresentable) + a test pinning the account count.
• Deferred permission append / rollout sequencing: a migrated-but-not-activated builder emits a shorter account list, so authorize() takes its no-permission path (legacy GlobalState allowlist) until the append is re-enabled. The per-instruction append activation must be sequenced against the permission-model rollout so no caller loses authorization mid-transition.
• Count/account mismatch: the builder derives the count from its loop.
• Discriminant coupling: builders construct the DoubleZeroInstruction variant and call .pack() (correct tag/borsh for free, no hand-written tag bytes).

End-to-end verification

1. make rust-build && make rust-test (includes the new crate's solana-program-test suite) and make rust-fmt.
2. make generate-fixtures and confirm there are no uncommitted diffs.
3. Real sanity check: use a builder to assemble an instruction, wrap it with compute_budget_prelude(), sign and send against the local devnet (dev/dzctl), and verify the processor accepts it.

Critical files

• smartcontract/programs/doublezero-serviceability/src/instructions.rs — enum + pack(), list of buildable vs deprecated variants.
• smartcontract/programs/doublezero-serviceability/src/pda.rs — all PDA derivations.
• smartcontract/programs/doublezero-serviceability/src/processors/<dom>/<verb>.rs — source of truth for Args field order and next_account_info account order.
• smartcontract/sdk/rs/src/client.rs — assemble_instructions + permission cache (extract the convention, refactor the send path).
• smartcontract/sdk/rs/src/commands/device/create.rs — migration template.
• sdk/serviceability/testdata/fixtures/generate-fixtures/src/main.rs — fixture generator to extend.

Sub-issues (PRs)

• R0 — Scaffold crate + common + exemplar builders — serviceability: scaffold Rust instruction-builder crate + common helpers + exemplar builders #4015
• R1 — device domain builders — serviceability: device instruction builders (Rust) #4016
• R2 — link domain builders — serviceability: link instruction builders (Rust) #4017
• R3 — user domain builders — serviceability: user instruction builders (Rust) #4018
• R4 — location + exchange + contributor builders — serviceability: location + exchange + contributor instruction builders (Rust) #4019
• R5 — multicastgroup builders (+ pub/sub allowlists) — serviceability: multicastgroup instruction builders (Rust) #4020
• R6 — tenant + permission builders — serviceability: tenant + permission instruction builders (Rust) #4021
• R7 — topology (incl. batched) + feed builders — serviceability: topology + feed instruction builders (Rust) #4022
• R8 — accesspass + resource builders — serviceability: accesspass + resource instruction builders (Rust) #4023
• R9 — globalstate + globalconfig + allowlist + index + migrate builders — serviceability: globalstate/globalconfig/allowlist/index/migrate instruction builders (Rust) #4024
• R10 — Migrate commands/* to delegate + trait methods + solana-program-test suite — serviceability: migrate commands/* to instruction builders + program-test suite (Rust) #4025
• RF — Extend fixture generator (ix_*.bin/.json) + CI diff guard — serviceability: golden fixtures for instruction builders + CI diff guard (Rust) #4026