Conversation
|
Important Review skippedToo many files! This PR contains 863 files, which is 713 over the limit of 150. To get a review, narrow the scope: Upgrade to a paid plan to raise the limit. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (196)
📒 Files selected for processing (863)
You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The term->binding-newtype mapping (Hm->Hmac256, Ore->OreBlock256, Bloom->BloomFilter) is wire-contract data, so it lives on Term beside json_key/ctor/extractor (unit-tested). payload_terms returns the distinct field-bearing terms in wire order. PR 4's bindings emitter matches on these instead of a &str round-trip, keeping it exhaustive at compile time.
render_family_bindings renders each DomainFamily to its struct set + three- method DomainType impls (schema -> schemars::Schema, 1.x), with pinned envelope-then-term field order (Term::payload_terms, matching on the enum), the canonical derive/ts/serde attributes, a precise term-newtype import set, and one catalog-derived struct doc line (no field docs).
render_inventory_rs generates all() in CATALOG order (entries via super::), replacing the hand-maintained inventory. mod.rs stays hand-written (module doc + pub mod decls + re-exports).
generate_bindings writes one <family>.rs per catalog family + inventory.rs under crates/eql-bindings/src/v3 (GeneratedKind::Rust ownership). Exposed as `eql-codegen bindings`; the default no-arg run stays SQL-only so build / codegen:parity are unaffected.
One Rust payload struct per eql_v3 SQL domain, generated by eql-codegen from eql-domains::CATALOG, with ts-rs/schemars deriving the committed TypeScript and JSON Schema. Each struct carries a catalog-derived doc — a summary line plus the supported operators and required payload keys (capability label, Term::operators_for_terms, ENVELOPE_KEYS ++ Term::term_json_keys); the keys list surfaces structural distinctions such as text's dual-term ordered domains (hm + ob) versus the integer ordered domains (ob). No per-field docs: per-term semantics live on the shared term newtypes and non-derivable per-family caveats in mod.rs. mise run types:generate regenerates Rust then TS/JSON; types:check is the committed-reference drift gate.
…ndings Byte-parity and determinism gates for the committed Rust bindings, TS property- order pin, catalog/inventory parity, mod.rs pub-mod pin, and a bindings CLI smoke test — so a stale or hand-edited generated file, a dropped domain, or a reordered payload fails CI.
Document in CLAUDE.md and the adding-a-scalar reference that the Rust payload bindings (and downstream TS/JSON via ts-rs/schemars) are catalog-generated, that generated structs carry a catalog-derived struct doc (summary + operators + required keys, no per-field docs), schemars emits JSON Schema 2020-12, non-derivable caveats live in mod.rs, and types:check is the drift gate.
…ial writes Replace the clean-then-write model (which could leave artifacts deleted-but-not-rewritten on a render panic or IO error) with a uniform, crash-safe render-all → preflight → write-all → delete-orphans discipline across both the SQL surface and the committed Rust bindings. - writer.rs: write_generated_file now stages to a sibling temp file and atomically renames it into place (no truncated/half-written files; temp removed on failure). Add remove_generated_orphans + normalized_set for marker-aware deletion that skips just-written files. - generate.rs: add render_type (render to memory first, so a render panic aborts before any fs mutation). generate_type is render→preflight→write→ delete-orphans. generate_all sweeps every src/v3/scalars/* dir for orphans after writing, absorbing build.sh's catalog-removed-type cleanup safely. Add clean_all. - bindings.rs: generate_bindings switched to write-then-delete-orphans so committed source is never deleted before writes succeed (output unchanged). - main.rs: new `clean` subcommand; EQL_CODEGEN_OUT_ROOT output-root override. - tasks/build.sh: drop the marker-blind `find -delete` pre-sweep; rely on codegen's marker-aware orphan removal. - tasks/codegen-parity.sh: generate into a temp tree (EQL_CODEGEN_OUT_ROOT) and diff against it, so a parity run never mutates the live src/v3/scalars. - Reword ~58 misleading "committed" references to generated/gitignored fixtures across tests/ and docs (genuinely-committed reference goldens keep the word). - Add 6 crash-safety tests (atomic write, no destructive-first window, marker-aware orphan pruning, orphan type-dir cleanup). - docs/reference/catalog-driven-architecture.md: document the new model. Closes #328.
…ning Resolve 7 verified review findings; 1 finding (artifact counts) was a false positive and left unchanged. - main.rs: route bindings subcommand through out_root() so EQL_CODEGEN_OUT_ROOT redirection works; restores test isolation - ts_property_order.rs: honor TS_RS_EXPORT_DIR when resolving the TS file, falling back to bindings/v3, so the test validates freshly generated output - generate.rs: use DirEntry::file_type()?.is_dir() (no symlink follow) in the orphan-sweep scan in generate_all and clean_all, preventing cleanup traversal outside out_root - writer.rs: add process id to staged temp file names to avoid cross-process collisions; use symlink_metadata() in the ownership preflight to detect pre-existing dangling symlinks - cli.rs: assert bindings codegen honors the out-root override - docs: add language tag to scalars tree fence; correct struct-doc description to two lines (summary + operators/required-keys) Adds regression tests for the symlink-traversal and dangling-symlink cases (red->green confirmed).
…codegen feat(codegen): catalog-driven Rust payload bindings (PR 4)
The catalog-driven-architecture reference doc described the test:self_contained_v3 gate using the literal 'eql_v2.*' token, which the docs_v3_grep gate (intentionally inverted-scope) flags in any user-facing reference doc. Reword to 'legacy v2-schema symbols' — same meaning, no literal eql_v2 token.
fix(eql-codegen): harden artifact regeneration + review follow-up (PR 4, #328)
Remove the completed-effort reference-sync cluster (documentation-blockers, documentation-questions, reference-sync-notes, reference-sync-rules) — empty stubs and their rules doc, self-referential with no live consumer. Merge sql-documentation-standards + sql-documentation-templates into a single docs/development/sql-documentation.md, cross-linked to CLAUDE.md as the source of truth, and drop the dead Operator Class / Composite Type templates that contradict the v3 jsonb-domain footgun rules. Update the stale reference-sync example in docs_v3_grep.sh's comment.
Verification pass over docs/reference/adding-a-scalar-encrypted-domain-type.md found several stale or wrong claims; fix them all: - Drop the stale 'codegen gates the PostgreSQL test matrix' claim (sections 4 and 6); the gate was removed (test job needs only changes/setup/build-archive). - Fix filename tasks/pin_search_path.sql -> pin_search_path_v3.sql (5 refs). - Fix non-compiling example int_values!(.., INT4) -> INT4_FIXTURES. - Rewrite section 6 stage-4 to match generate_type: ensure_generated_paths_writable -> write_generated_file -> remove_generated_orphans (no clean_generated_files in that path). - Correct text domain count (five -> six) and 'operators' vs 'CREATE OPERATOR statements' wording; intro reference range 5-7 -> 5-8. - Misc precision nits: text pivots (only mid_pivot overridden), SignedScalar also for timestamptz, binding output paths, int_values! location, splinter constructors/comparators label, self-containment gate pointer.
docs: prune obsolete process docs, consolidate SQL doc guidance, fix scalar-domain reference
…ting
Un-ignore src/v3/scalars/<T>/*_{types,functions,operators,aggregates}.sql and
commit them in place, retiring the byte-identical tests/codegen/reference/
goldens. Both generated targets (SQL + Rust bindings) are now handled the same
way: committed in place, drift-gated by regenerate-and-`git diff`.
- .gitignore: drop the four generated-SQL globs (functions.sql / *_extensions.sql
stay committed as before).
- .gitattributes: mark the committed surface linguist-generated; drop the dead
golden rule.
- codegen:parity: rewrite as regenerate-in-place + `git diff --exit-code` +
untracked check (mirrors types:check for the bindings); delete
tests/codegen/reference/.
- parity tests: repoint at the committed src/v3/scalars/ tree
(committed_scalar_dirs_match_catalog_tokens,
every_generated_sql_file_starts_with_marker); drop the redundant
byte-comparison tests now covered by the git-diff gate.
- install-hooks + tasks/githooks/pre-commit: run codegen:parity + types:check
locally when a commit touches the catalog/generator/generated surfaces,
closing the local-staleness gap for both targets.
- docs: CLAUDE.md, DEVELOPMENT.md, docs/reference/* updated to the
committed-in-place model.
No user-facing change: release/cipherstash-encrypt.sql is byte-identical, so no
CHANGELOG entry.
- pre-commit hook: include staged deletions (-diff-filter=ACMRD) so the local drift gate also fires when a generated file is removed - docs: correct "CI reverts drift" -> "CI fails on drift" in CLAUDE.md and DEVELOPMENT.md; the gate fails on diff, it does not revert
The drift gate (and the sibling types:check) regenerates the generated surface in the working tree before diffing, rather than into a temp tree. Make that side effect explicit in the script header comment and both task descriptions so a developer running the gate ad hoc isn't surprised that it overwrites src/v3/scalars (as mise run build already does on every build).
chore: commit generated SQL surface in place; unify drift gating (eql_v3 / 3.0.0)
…binding release surface
…ety, creds) - F1: pass inputs.identity via env instead of interpolating into run bodies (command-injection fix across the version-check, prepare, and tag steps) - F2: make npm publish and tag creation idempotent for retry-safety - F3: checkout with persist-credentials: false; authenticate the tag push explicitly via extraheader to the repo URL - F4: mkdir -p before writing the generated release manifest - add .github/scripts/release-typescript-workflow.test.sh regression guard and wire it into lint-release.yml
Changesets owns @cipherstash/eql's version V (alpha pre-mode); scripts/sync-lockstep-versions.mjs derives the crate + bundled-SQL versions from V via the root 'version' script. prepare-bindings-assets accepts stable X.Y.Z too.
…sh-only release.yml is the changesets-driven entry point (Version PR via version:, npm publish via publish:, then SQL+docs release at V). release-plz.yml drops its release-pr job — changesets owns versioning; it publishes the committed Cargo.toml V on push to main.
Collapse the release surface onto the unified release.yml (production from main via changesets; alpha/prerelease from eql_v3 via a chore(release): commit). Remove the parallel machinery and roll the remaining release artifacts into the one workflow. - Remove the legacy alpha coordinator: release-alpha.yml, release-typescript.yml, the .github/scripts helpers (derive-identity, release-alpha-resolve, release-alpha-pin-bindings, release-alpha-verify-commit-signature, release-typescript-workflow guard) and their tests, and the tasks/release/* wrappers (all/eql/bindings/rust/typescript/resolve-alpha/pin-bindings). - Remove release-eql.yml: the old manual final-release path. release.yml now builds + attaches the SQL + docs release itself; a GITHUB_TOKEN-created release never triggered its on: release fan-out, and changesets manages the changelog (its verify-changelog gate is obsolete). - Restore the Multitudes production-deploy notification into the automated path: re-gate the _build-sql.yml step to fire on release.yml's non-prerelease SQL release (the old event_name == 'release' gate never matched under push). - Roll the Postgres+EQL image build into release.yml: a build-image job dispatches release-postgres-eql-image.yml on production finals (update_floating_tags=true). Drop its dormant on: release trigger and the dead release-event branch; workflow_dispatch stays for manual/alpha images. - Harden .changeset/config.json access to "public" (was "restricted"; inert for @cipherstash/eql today via publishConfig, but a footgun for future packages). - Fix crates/eql-bindings/README.md (v3 envelope is v: 3, not v: 2) and align the release docs (CLAUDE.md, releasing-an-alpha.md, workflows/README.md, README badge, CHANGELOG) plus the lint-release gate to the single workflow. - Mark the superseded 2026-07-04 coordinator design snapshots as historical.
Add docs/development/releasing.md — a single reference covering the whole release architecture (one release.yml, both modes, four artifacts, one lockstep version): production via changesets on main, prerelease/alpha on eql_v3, the lockstep versioning mechanism (changesets -> sync-lockstep-versions.mjs -> Cargo + bundled SQL), tag namespaces, the GITHUB_TOKEN inline-vs-dispatch model, the changesets pre-mode footgun, and verify/smoke-test steps. Delete docs/development/releasing-an-alpha.md: it was alpha-only and had gone stale after the release.yml consolidation (referenced the deleted verify-changelog job, server-side N derivation, and the coordinator). Repoint CLAUDE.md, .changeset/README.md, and the superseded design-snapshot banners at the new doc.
…elease tag - Notify Multitudes (_build-sql.yml): add continue-on-error. It is the last step of build-sql and uses `curl --fail-with-body`; on the production path npm + SQL have already published upstream, so a Multitudes outage / empty token must not fail build-sql and skip build-docs / build-image. - build-image (release.yml): dispatch release-postgres-eql-image.yml against the eql-<version> tag instead of the branch. build-sql (a `needs`) already created that tag at the release commit, so pinning to it builds the image from the exact released source even if the branch advanced since publish.
Way 2 is now canonical: contributors add a changeset (`.changeset/*.md`); Changesets owns both the version bump and CHANGELOG.md generation. Retire the manual `## [Unreleased]` promotion everywhere the docs still described it. - CLAUDE.md "Release & changelog discipline": add a changeset (not a CHANGELOG.md entry); the entry is the changeset body; the bump type is the changeset frontmatter; "Cutting a release" is now the Changesets Version-PR flow. - CHANGELOG.md preamble: state the file is Changesets-generated, not hand-edited. - DEVELOPMENT.md "Releasing": replace the manual GitHub-release flow (and the stale release-eql.yml reference) with the release.yml + Changesets model. - releasing.md: changelog section rewritten to the Changesets model; fix the prerelease changelog note. - docs/upgrading/v3.0.md: drop the stale `[Unreleased]` phrasing.
Migrate the hand-written CHANGELOG.md `## [Unreleased]` block (the pending 3.0.0 notes) into 33 Changesets files, so Changesets — now the canonical owner — generates the release section. De-duplicated in the process: - dropped 6 exact duplicates and 7 stale pre-rename duplicates (the scalar types were renamed int4->integer, int2->smallint, int8->bigint, bool->boolean, float4->real, float8->double; both old- and new-name entries were present). - bump per entry by nature: 4 major (eql_v2 removed, envelope v:3, eql_v3_internal move, sole installer), 24 minor, 5 patch -> aggregate resolves to 3.0.0. - full dense prose + PR links preserved verbatim in each changeset body. Remove the `[Unreleased]` block and its compare-URL link reference from CHANGELOG.md; historical `## [2.x.y]` sections are untouched. `changeset status` confirms a major bump for @cipherstash/eql.
Rebased onto eql_v3, which added the query-operand domain surface (query_<name> in eql_v3, CIP-3432/3442), COMMENT ON DOMAIN, and the conditional SEM opclass install. Reconcile the branch-local work: - Regenerate eql-bindings + @cipherstash/eql bindings from the updated catalog (adds the query_<T> schemas/types; jsonb_query -> query_jsonb). - Convert base's four new CHANGELOG [Unreleased] entries (#377, #373, CIP-3442, #375) into changesets and remove the reconstructed [Unreleased] block (Changesets owns the changelog).
types:check / typescript:generate / typescript:check invoked 'pnpm --filter @cipherstash/eql sync:generated', but the contexts that run them — test-eql.yml's Rust workspace crates job and the pre-commit hook — provision mise tools only, so CI failed with 'pnpm: command not found' (exit 127). sync-generated.mjs uses node builtins exclusively; it needs neither pnpm's workspace resolution nor node_modules. Invoke it with node directly and add node to mise [tools] so every mise-run context has it. pnpm remains a package-workflow concern (lint-release.yml and release.yml set it up explicitly).
…terministic no-op, not a conflict to resolve
…ning, cache lint coverage Addresses the remaining items from the changes-requested review (#374 (review)): - DEV-placeholder publish guards (finding 6): release-plz.yml refuses to publish when the committed crates/eql-bindings/sql bundle wasn't prepared for the crate's version (enforced only when the version isn't already on crates.io, so routine no-op runs still pass); the npm package gains a prepublishOnly gate (scripts/verify-release-assets.mjs) that covers BOTH publish paths — changeset publish (production) and npm-publish.mjs (prerelease) shell out to npm publish, which runs it. - Prerelease marker (finding 7): classify accepts exactly 'chore(release):' (a bare 'release:' doc commit can no longer trigger a publish) and short-circuits to skip when the identity's eql-typescript-v tag already exists. - One-identity-one-commit (finding 8): prerelease-publish-rust dispatches release-plz.yml against the eql-<V> tag (same pattern as build-image), not the branch, so the crate publishes from the exact release commit. - Supply chain (findings 9/10): npm pinned to an exact version in both publish jobs; mise-action cache disabled in release.yml and release-plz.yml and the no-workflow-caching lint extended to cover jdx/mise-action (with release-plz.yml added to its default file set, plus test cases). - Changelog ownership (finding 5): CHANGELOG.md header, CLAUDE.md, and releasing.md now say what is true — Changesets maintains packages/eql/CHANGELOG.md (the lockstep release changelog); the root file is the frozen pre-3.0 archive. - Access-model note (finding 12): releasing.md documents that prerelease publishes are gated by eql_v3 push access + branch protection. - Tests (coverage finding): bumpCargoPackageVersion extracted from sync-lockstep-versions.mjs, now section-anchored (can never rewrite a column-0 version line outside [package]) with vitest coverage; prepare-bindings-assets.test.sh covers the version-validation gate (wired into lint-release.yml). - Nits: no defaults.run.shell 'bash {0}' override in release.yml / release-plz.yml (GitHub's default bash is -eo pipefail); unused repoRoot dropped from copy-assets.mjs.
feat(release): publish EQL v3 language bindings (Rust + TypeScript) with bundled SQL
Lockstep prerelease pin via 'pnpm run version' (changesets pre-mode): package.json, Cargo.toml, and the bundled exact-version SQL assets all carry 3.0.0-alpha.3. Pushing this marker commit to eql_v3 triggers release.yml's prerelease path: SQL + docs GitHub release (eql-3.0.0-alpha.3), npm publish (@cipherstash/eql, dist-tag alpha, tag eql-typescript-v3.0.0-alpha.3), and the dispatched crate publish (eql-bindings-v3.0.0-alpha.3).
First attempt (b5c586f) failed on two jobs: - prerelease-publish-rust: 'gh workflow run' resolves the workflow via a local git repo, and the job has no checkout — fixed with --repo. - prerelease-publish-npm: ENEEDAUTH — npm Trusted Publishing is not yet configured for @cipherstash/eql + release.yml on npmjs.com (alpha.2 was published manually). Needs the npmjs-side publisher config; will fail again on this retry, which is fine: the eql-typescript tag is only created after a successful npm publish, so classify keeps allowing retries for this identity until npm lands. The marker subject re-triggers the prerelease path for 3.0.0-alpha.3: build-sql/docs update the existing eql-3.0.0-alpha.3 release in place (action-gh-release updates by tag), and the fixed dispatch publishes the crate from the eql-3.0.0-alpha.3 tag.
…red) npm and crates.io OIDC trusted publishers are now configured (@cipherstash/eql -> release.yml; eql-bindings -> release-plz.yml) and release-plz.yml is registered on the default branch (PR #383). This marker retry completes the two package legs: the eql-typescript tag doesn't exist yet so classify proceeds; SQL/docs update in place.
…ranch dispatch) Second retry failed on both package legs for new reasons: - npm ENEEDAUTH persisted despite trusted publishing being configured: mise-action runs after the 'upgrade npm' step and prepends mise's node (from [tools]) to PATH — its bundled npm 10.x has no OIDC trusted publishing, so the upgraded npm 11.5.1 never ran. The upgrade step now runs AFTER mise-action in both publish jobs. - release-plz errored 'cannot determine current branch': dispatching against the eql-<V> tag gives it a detached HEAD, which it refuses. prerelease-publish-rust now pins a release/eql-<V> branch at the release commit and dispatches against that — same exact-commit guarantee, real branch for release-plz. classify proceeds for 3.0.0-alpha.3 (the eql-typescript tag is still uncreated); SQL/docs update in place.
npm publish and the crate publish both succeeded on the previous retry (@cipherstash/eql@3.0.0-alpha.3 with provenance; eql-bindings@3.0.0-alpha.3 via crates.io trusted publishing). The only remaining failure was the eql-typescript tag push: with persist-credentials: false the git extraheader hack didn't authenticate ('could not read Username'). Create the tag ref via 'gh api' instead. This retry converges: npm publish skips (already published), the tag is created, release-plz no-ops (version already on crates.io).
Until the 3.0.0 final ships, the alphas are the package's only release line, so a bare 'npm install @cipherstash/eql' should resolve to the newest alpha rather than whichever version last happened to hold latest (alpha.2 had it from a manual publish while alpha.3 sat under the alpha tag). PRE_GA_LATEST in npm-publish.mjs flips the policy back to channel dist-tags after GA. Policy per James, 2026-07-08.
…warning Doxygen auto-links the bare word `text` (it collides with a documented symbol) into a cross-reference — even inside inline code. In the jsonb selector operator's @warning, `` `col -> 'sel'::text` `` became `<computeroutput>col -> 'sel'<ref>text</ref></computeroutput>`: a link nested inside inline code. Rendered to Markdown that surfaces as an unbalanced `<tt>`, which fails the downstream MDX/docs build (cipherstash/docs was broken by eql-3.0.0-alpha.3's API.md at line 2153, `Expected a closing tag for <tt>`). It also drops the `::`, so the cast read as `'sel'text`. Prefix the two `text` occurrences with Doxygen's `%` no-autolink marker (`::%text`, `-> %text`). Verified with doxygen: the nested <ref> is gone and `col -> 'sel'::text` now renders as clean, balanced inline code.
…arse fix(docs): skip name-dropped CREATE FUNCTION mis-parses
…tolink fix(docs): suppress Doxygen auto-linking of `text` in jsonb selector @warning
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merges the
eql_v3release line intomainfor the EQL 3.0.0 GA release. (Originally opened as the long-running v3 integration tracking PR; description updated now that the branch is release-ready.)Linear: CIP-3465
What this brings to
maineql_v3/eql_v3_internalsurface — theeql_v2schema, its database-side config management, and the multi-variant build were removed in 3.0.0;release/cipherstash-encrypt.sqlis now a single self-contained installer built fromsrc/v3aloneinteger,smallint,bigint,date,timestamp,numeric,text,boolean,real,double) driven by theeql-domains::CATALOG+eql-codegenpipeline, with committed drift-gated SQL undersrc/v3/scalars/src/v3/jsonb/eql-bindings) and TypeScript (@cipherstash/eql) binding packages generated from the same catalog, both bundling the SQL installerrelease.yml, publish-onlyrelease-plz.yml), validated end-to-end by the3.0.0-alpha.xprereleases cut from this branchMerge conflict note
.github/workflows/release-plz.ymlis an add/add conflict:maincarries a registration-only dispatch stub (#383) whose own commit message says to replace it with the full workflow wheneql_v3merges. Resolve by taking theeql_v3side (the full publish workflow).After merge
maintriggersrelease.yml(classifies as no-op — not achore(release):marker commit; finals are owned by the changesets Version PR flow),release-plz.yml(no-op — the committed crate version is already published), andbench-eql.ymlmainby merging the changesets "Version Packages" PR perdocs/development/releasing.md