vortex-java

Alpha — not production-ready. APIs will change without notice.

Pure-Java reader/writer for the Vortex columnar file format. 100% Java, no JNI, no sun.misc.Unsafe. Uses the FFM API (MemorySegment/Arena, Java 25+) for zero-copy memory-mapped reads.

Project	Language	Notes
vortex-data/vortex	Rust	Reference implementation + JNI bindings
LaurieRhodes/vortex-go	Go	Pure-language port
dfa1/vortex-java	Java	This library

Who is this for

• JVM analytics engines and OLAP systems
• Anyone who wants mmap-backed, zero-copy columnar reads without native-library management

Quickstart

<dependency>
  <groupId>io.github.dfa1.vortex</groupId>
  <artifactId>vortex-reader</artifactId>
  <version>0.5.0</version>
</dependency>

Minimal read example

try (VortexReader vf = VortexReader.open(Path.of("data/example.vortex"));
     var iter = vf.scan(ScanOptions.all())) {
    while (iter.hasNext()) {
        try (Chunk chunk = iter.next()) {
            LongArray ts = chunk.column("timestamp");
            for (long i = 0; i < ts.length(); i++) {
                System.out.println(ts.getLong(i));
            }
        }
    }
}

Lifecycle. ScanIterator implements Iterator<Chunk> and Chunk implements AutoCloseable. Each chunk owns a confined Arena; closing it releases the decoded buffers. Calling iter.next() while a prior chunk is still open throws IllegalStateException. Use try-with-resources, or iter.forEachRemaining(c -> ...) which closes each chunk for you. See docs/explanation.md#memory-model.

For more examples — writing, projection, filtering, custom encodings, and the CLI — see the documentation below.

Documentation

Docs follow the Diátaxis framework.

Document	Mode	Contents
docs/tutorial.md	Tutorial	Step-by-step: write and read your first Vortex file
docs/how-to.md	How-to	Recipes: count rows, convert Parquet, filter, project, custom encodings
docs/reference.md	Reference	API surface, CLI subcommands, operator tables, file-format trailer
docs/compatibility.md	Reference	Encoding support table, S3 fixture status
docs/explanation.md	Explanation	Design rationale, memory model, testing strategy, benchmarks

Contributing

Requirements: Java 25+. Build: ./mvnw verify.

Forks welcome. See CONTRIBUTING.md for full build reference, coding conventions, and how to add a new encoding.

This project uses Claude Code for implementation work. Architecture, API design, and all decisions are human-driven.