sec(parser): BoundedSegment + audit trail for untrusted asSlice

[dfa1]

Summary

Replaces every untrusted MemorySegment.asSlice call in the file-open and decode path with a typed BoundedSegment wrapper, so malformed input throws VortexException (the documented SECURITY.md contract) rather than IndexOutOfBoundsException.

The win is not "no MemorySegment ever exists" — it's every trust transfer is grep-discoverable:

$ git grep unwrapForSubParser

35 hits, one per encoder/inspector boundary. Each entry documents "this code consumes a bounded buffer." Future regressions get caught in code review (or by a Checkstyle ban on raw .asSlice( outside MemorySegments.java / BoundedSegment.java — see follow-ups).

Design

BoundedSegment (a record holding MemorySegment + context label) exposes only:

• slice(off, len, childContext) — bounds-checked, throws VortexException on bad input
• getByte/getIntLE/getLongLE(off) — bounds-checked primitive reads
• asByteBufferLE() — for FlatBuffer parsers (which validate offsets independently)
• unwrapForSubParser(reason) — the audited escape hatch. Every call documents the trust transfer and surfaces reason in error messages.

MemorySegments.slice / .checkRange remain as the implementation detail that BoundedSegment delegates to. Prior art for this pattern: Lucene IndexInput.slice (LUCENE-5827), Trail of Bits bounded_buffer, C++ gsl::span, Rust &[u8] / bytes::Bytes. The JDK FFM team has not formally recommended it; this is the first articulation of the pattern for MemorySegment.

Commits (5)

1. b6a9c24 — original MemorySegments static helper (covers the immediate asSlice audit)
2. b7d583a — Phase 1: introduce BoundedSegment at the mmap boundary (VortexReader.parse, Trailer.parse, PostscriptParser.parse)
3. 413edb8 — Phase 2: thread through DecodeContext.buffer() and FlatSegmentDecoder (14 encoder consumers + 71 test sites migrated)
4. 4352be0 — Phase 3: VortexHandle.slice signature → BoundedSegment; ScanIterator/InspectorTree/TUI/integration-test callers updated
5. a9a5189 — Phase 4: VortexReader internals migrated; MemorySegments becomes implementation detail of BoundedSegment

Out of scope (follow-ups)

Eliminating the remaining 35 unwrapForSubParser sites would require:

• Migrating core/array (BoolArray, IntArray, etc.) to hold BoundedSegment — broad blast radius
• Migrating ProtoReader + every generated proto record decode(MemorySegment, ...) factory
• Adding hot-loop primitive readers on BoundedSegment for encoder buffer math
• Wrapping ZstdEncoding.toArray() + similar

Each is mechanically straightforward but expensive (~7-10 hours total). Deferred — the current audit trail is the actionable property; deeper migration is purity for purity's sake at alpha stage.

Test plan

• ./mvnw verify — 938 unit + 243 integration tests pass (incl. Rust round-trip suite)
• MemorySegmentsTest (7 cases) + BoundedSegmentTest (4 cases) cover in-range, out-of-range, negative offset/length, overflow-prone Long.MAX_VALUE-1, primitive reads, slice context propagation, unwrap identity
• Reviewer: confirm the "left as unwrapForSubParser" categories are actually trust-bounded buffers (FlatSegmentDecoder slices them before they reach encoders)

🤖 Generated with Claude Code

[dfa1]

Audit: unwrapForSubParser call sites (33 total)

git grep unwrapForSubParser enumerates every trust transfer. Listed here for review traceability — each entry is a place where a BoundedSegment is converted to a raw MemorySegment because the downstream API does not accept the typed wrapper. Future removal is purely a matter of migrating those downstream APIs.

encoders → MemorySegment-typed array constructors or buffer math (24)

File	Line	Reason	Downstream
BitpackedEncoding.java	273	bitpacked encoding	fastlanes unpack into pre-allocated output segment
BoolEncoding.java	71	bool encoding	BoolArray stores MemorySegment
ByteBoolEncoding.java	47	bytebool encoding	byte-level scan + BoolArray construction
ConstantEncoding.java	136	constant encoding	ProtoReader(MemorySegment, ...) for scalar payload
DecimalEncoding.java	112	decimal encoding	per-row decimal byte parsing
DictEncoding.java	381	dict encoding values	dict-values direct buffer indexing
DictEncoding.java	438	dict encoding	dict-bytes for VarBinArray
DictEncoding.java	439	dict encoding	dict-offsets for VarBinArray
DictEncoding.java	440	dict encoding	codes for child decode
FsstEncoding.java	209	fsst encoding	FSST symbol table (8B per symbol)
FsstEncoding.java	210	fsst encoding	FSST symbol lengths (1B per symbol)
FsstEncoding.java	211	fsst encoding	FSST-compressed heap
PcoEncoding.java	144	pco encoding	chunk metadata ProtoReader
PcoEncoding.java	163, 189, 205, 228	pco encoding	per-page Pco decode
PrimitiveEncoding.java	315	primitive encoding	MemorySegment.copy into output
Registry.java	80	registry	Registry.decodeAsSegment adapter
SparseEncoding.java	213	sparse encoding	fill-value ProtoReader
VarBinEncoding.java	143	varbin encoding	VarBinArray constructor
VarBinViewEncoding.java	123	varbinview encoding	views buffer for VarBinViewArray
VarBinViewEncoding.java	126	varbinview encoding	per-data-buffer
ZstdEncoding.java	317	zstd encoding	.toArray() for native Zstd dict
ZstdEncoding.java	322	zstd encoding	.toArray() for native Zstd frame
ZstdEncoding.java	351	zstd encoding	frame slice for ZstdJavaDecompressor

scan / inspector / TUI → FlatSegmentDecoder.decode(MemorySegment, ...) (7)

File	Line	Reason
ScanIterator.java	483	flat segment decoder
ScanIterator.java	637	stats segment fbLen read
InspectorTree.java	150	inspector flat segment decoder
InspectorTree.java	205	inspector flat segment decoder
VortexInspectorTui.java	574	inspector tui flat segment
VortexInspectorTui.java	763	inspector tui hex peek
Trailer.java	33	trailer parser (constant-offset magic read)

integration test (1)

File	Line	Reason
PcoFixtureInspectionIntegrationTest.java	104	integration test inspector

Removal cost map (follow-up PRs)

• FlatSegmentDecoder.decode takes BoundedSegment → kills 6 of the scan/inspector/TUI unwraps. ~30 min.
• Trailer.parse uses getByte family instead of asSlice → kills 1. ~5 min.
• ProtoReader(BoundedSegment, long, long) + regenerated proto records → kills the 7 ProtoReader-bound unwraps (Constant, Pco×5, Sparse). Requires CodeGen update + 42 file regen. ~1 hour.
• core/array hierarchy holds BoundedSegment → kills the ~12 Array-constructor unwraps. Broad blast radius. ~3-5 hours.
• BoundedSegment.toArray() proxy → kills 2 Zstd unwraps. ~10 min.

Easy wins (FlatSegmentDecoder + Trailer + Zstd proxy) cumulatively eliminate 9 unwraps in under 1 hour. The remaining 24 require the deeper migrations.

[dfa1]

Tradeoffs

What we gain

1. SECURITY.md contract is honored at the read-side entry points. Before this PR, opening a file with a malformed trailer or postscript threw IndexOutOfBoundsException (and similar) — bypassing every catch (VortexException) in caller code. After, malformed input at every wrapped site throws VortexException carrying a context label naming the on-disk structure that was being parsed.
2. Audit trail by construction. git grep unwrapForSubParser lists every trust transfer the codebase performs. Future PRs that add a new sub-parser or decoder must either accept BoundedSegment or insert a labelled unwrapForSubParser("reason") — and that line shows up in code review.
3. Error messages name the structure, not the offset. "malformed trailer: offset+length 0+8 exceeds segment size 4" is more debuggable than the previous IndexOutOfBoundsException: ... for a corrupted 4-byte truncated file.

What we pay

1. API churn. VortexHandle.slice signature changed from returning MemorySegment to BoundedSegment. Even though it's @Deprecated(forRemoval = true), anyone using it has to add .unwrapForSubParser(...) at the call site. Inspector, TUI, and integration-test callers all needed an update in this PR. External users — none yet, but a real cost once there are.
2. 33 ugly unwraps in encoder code. Each ctx.buffer(i).unwrapForSubParser("foo encoding") is more verbose than ctx.buffer(i). The verbosity is the point (audit trail), but a maintainer who doesn't know the design will read it as noise and consider deleting it.
3. Conceptual gap with "pure" parse-don't-validate. The pattern is not affine like Rust &[u8]. BoundedSegment.unwrapForSubParser returns the raw segment; nothing stops a caller from calling seg.asSlice(badOff, badLen) on the unwrapped result. The enforcement is review-time, not compile-time. A genuinely complete version would (a) remove unwrapForSubParser and migrate core/array, ProtoReader, FlatBuffer integration to BoundedSegment (~7–10 hours of work) or (b) layer Checker Framework / ErrorProne on top.
4. Test-code adapter. The 71 test sites that built a DecodeContext directly now have to route through DecodeContext.ofRawBuffers(...). Mechanical mapping today; one more thing for a contributor to learn tomorrow.
5. Two paths in the codebase. MemorySegments (static helper) still exists as the implementation detail behind BoundedSegment. Some surviving direct uses are documented and intentional, but a reader has to know which to reach for. The doc on MemorySegments covers this; still a paper cut.

Perf

Negligible. BoundedSegment.slice is a record allocation + MemorySegments.slice (one bounds check + one MemorySegment.asSlice). JIT inlines through. Hot read paths (decode loops) operate on the unwrapped MemorySegment once unwrapForSubParser runs, so the inner loops are byte-identical to pre-PR. ./mvnw verify integration suite — all 938 unit + 243 integration tests pass, no regression observed in the integration test wallclock.

When this design is the wrong fit

• A contributor proposes adding a 34th unwrapForSubParser site for a brand-new encoder that handles a different layout shape. The audit trail logic stops scaling once the list is long enough that nobody reads it.
• The lib goes past alpha and gains external users. Then VortexHandle.slice returning BoundedSegment is a breaking change for them; promoting BoundedSegment to public API has a documentation cost.
• Someone runs JFR and finds BoundedSegment.slice allocations show up in a hot decode loop. Then escape analysis isn't doing what we assumed and the record needs to become a primitive value type (Valhalla) or a static helper after all.

Net

For an alpha-stage library where the threat model is "untrusted columnar files from S3 / HTTP" and the contract is documented in SECURITY.md, the audit-trail property is worth the verbosity. The remaining 33 unwraps are documented (see previous comment) with concrete removal costs; they can be picked off opportunistically in future PRs rather than holding this one back.