fix(v3): inline the jsonb_entry domain CHECK; plpgsql jsonb_query validator (+19% field_eq regression)

CHANGELOG.md

@@ -72,6 +72,7 @@ Each entry that ships in a published release links to the PR that introduced it.
 - **The `eql_v2` schema and its entire surface are removed — EQL now ships only the self-contained `eql_v3` encrypted-domain surface.** Dropped with no `eql_v3` replacement: the `eql_v2_encrypted` composite column type and its operators (`=`, `<>`, `~~`/`~~*` `LIKE`/`ILIKE`, containment `@>`/`<@`, ORE comparisons), database-side configuration management (`eql_v2_configuration`, `add_search_config`, `add_column`, `migrate_config`, `diff_config`, `create_encrypted_columns`), the `encryptindex` migration machinery, boolean operators on the encrypted column type, operator-class-on-column indexing, and `GROUP BY` / `grouped_value` on the encrypted column type. Searchable-encryption capabilities (equality, ordered range, `MIN`/`MAX`, encrypted-JSONB document containment and path access) are all provided by the `eql_v3` encrypted-domain families and document surface. Why: `eql_v3` is now fully self-contained — it owns its own SEM index-term types (`eql_v3.hmac_256`, `eql_v3.ore_block_256`, `eql_v3.ore_cllw`, `eql_v3.bloom_filter`) and installs into a database with no `eql_v2` present — and the encryption client (CipherStash Proxy / ProtectJS) now owns the configuration model the database-side `eql_v2` functions previously provided. This is a major (3.0.0) public-API break; callers using the `eql_v2` schema migrate to the `eql_v3` encrypted-domain types. The user-facing reference documentation and tutorials have been migrated to teach only the `eql_v3` surface — the `eql_v2`-based examples, the database-side configuration guide, and the operator-class-on-column indexing recipe are removed. No per-capability migration guide is provided for the dropped capabilities (they have no `eql_v3` equivalent).
 ### Fixed
 
+- **`eql_v3.jsonb_entry` CHECK inlined; `jsonb_query` validator converted to plpgsql — removes SQL-function-executor overhead from the per-query needle casts.** Domain constraints cannot inline SQL functions, so `jsonb_entry`'s function-call CHECK paid ~18 µs on every cast — the needle cast in every `field_eq` query was the ENTIRE +19% v2→v3 regression on that scenario (in-DB 0.011 → 0.029 ms/query with identical `eq_term` costs; cipherstash/benches#23). The `jsonb_entry` CHECK now mirrors the validator body inline, with a leading `VALUE IS NULL OR` preserving STRICT NULL-passes semantics (equivalence pinned over a payload corpus by `jsonb_check::jsonb_entry_check_matches_validator`). `jsonb_query`'s CHECK cannot be inlined — validating sv elements needs a subquery, which CHECK constraints forbid — so `is_valid_ste_vec_query_payload` is plpgsql instead (cached plan vs per-call SQL-function executor; the #353 finding), guarded by `jsonb_check::jsonb_query_validator_is_plpgsql`. The `eql_v3.json` document CHECK — part of the documented privilege contract — is unchanged; `docs/reference/permissions.md` now notes the `jsonb_entry` cast requires no internal grant. ([#354](https://github.com/cipherstash/encrypt-query-language/issues/354))
 - **`ore_block_256` opclass-path helpers converted from `LANGUAGE sql` to plpgsql — restores v2-level ordered-scan performance.** `eql_v3_internal.jsonb_array_to_bytea_array` and `eql_v3_internal.jsonb_array_to_ore_block_256` were `LANGUAGE sql` for inlineability, but their only caller chain (`ore_block_256(val)`, plpgsql, feeding the btree operator class) can never inline SQL functions — every compared value paid the per-call SQL-function executor instead, measured at 3.5× the per-call cost of the logic-identical plpgsql form. Release benchmarks put the end-to-end cost at +43% on ORE ordered index scans vs EQL 2.3 (`0.513 → 0.736 ms` at 1M rows) and +36% on the composite bloom+ORE-order shape (`16.6 → 22.7 ms`); with the plpgsql form both scenarios return to (or beat) the v2 numbers — `0.553 ms` and `13.97 ms` respectively, validated A→B→A on a live 1M-row bench database. Semantics are unchanged: NULL/non-array inputs still return NULL, and the empty-`ob` COALESCE (#262) is preserved. The `eql-inline-critical` markers are retained so the pin_search_path pass keeps both functions unpinned — a `SET search_path` clause on plpgsql forces per-call configuration switching in the same hot path. Full attribution and experiment data: cipherstash/benches#23 (`v3-regressions-report.md`). ([#353](https://github.com/cipherstash/encrypt-query-language/issues/353))
 
 - **`=` / `<>` on `eql_v2.ore_block_u64_8_256` now declare a `COMMUTATOR`, so equality joins over the ORE term no longer raise.** Both operators set `COMMUTATOR` to themselves (equality and inequality are symmetric, so each is its own commutator). Why: without it the planner raised `could not find commutator for operator` the first time an `ore_block_u64_8_256` equality was used as a join / mergejoin qualifier (e.g. via the inlined `eql_v3.integer_ord_ore` equality wrappers, since the operators carry `MERGES`). This only enables previously-erroring join plans — it cannot change which rows match or their ordering. ([#239](https://github.com/cipherstash/encrypt-query-language/pull/239))

docs/reference/permissions.md

@@ -73,9 +73,10 @@ Why the internal grant is needed even though you only call public objects:
 - The **ORE comparison** behind ordering and `MIN`/`MAX` calls pgcrypto
   `encrypt()`, which the installer places in the `extensions` schema — hence the
   `USAGE` there.
-- **Casting raw jsonb to `eql_v3.json`** fires its domain `CHECK`,
-  `eql_v3_internal.is_valid_ste_vec_document_payload`. (Scalar domain CHECKs are
-  pure structural jsonb tests, so casting to a scalar domain needs no internal
+- **Casting raw jsonb to `eql_v3.json` or `eql_v3.jsonb_query`** fires a
+  domain `CHECK` that calls an `eql_v3_internal.is_valid_*` validator. (Scalar
+  domain CHECKs — and, since issue #354, the `eql_v3.jsonb_entry` CHECK — are
+  pure structural jsonb tests, so casting to those domains needs no internal
   grant.)
 
 The hand-written jsonb containment **read** path (`eql_v3.ste_vec_contains` and

src/v3/jsonb/types.sql

@@ -38,11 +38,18 @@ $$;
 --! @return boolean True when `val` is `{"sv":[...]}` and every element carries
 --!         string `s`, no ciphertext, and exactly one string term (`hm` XOR
 --!         `oc`).
+--! @note plpgsql, not LANGUAGE sql (issues #353/#354): the only caller is the
+--!   eql_v3.jsonb_query domain CHECK, where a SQL function can never be
+--!   inlined (and the CHECK itself cannot absorb this body — it needs a
+--!   subquery over the sv elements, which CHECK constraints forbid). plpgsql
+--!   caches its plan across calls instead of paying the per-call SQL-function
+--!   executor on every needle cast.
 CREATE FUNCTION eql_v3_internal.is_valid_ste_vec_query_payload(val jsonb)
   RETURNS boolean
-  LANGUAGE sql IMMUTABLE STRICT PARALLEL SAFE
+  LANGUAGE plpgsql IMMUTABLE STRICT PARALLEL SAFE
 AS $$
-  SELECT COALESCE(
+BEGIN
+  RETURN COALESCE(
     jsonb_typeof(val) = 'object'
      AND jsonb_typeof(val -> 'sv') = 'array'
      AND NOT EXISTS (
@@ -62,7 +69,8 @@ AS $$
        ), false)
      ),
     false
-  )
+  );
+END;
 $$;
 
 --! @brief Validate a root SteVec document payload.
@@ -119,9 +127,34 @@ CREATE DOMAIN eql_v3.json AS jsonb
 --! `i`/`v` merged in by `->`) are allowed.
 --!
 --! @see src/v3/jsonb/operators.sql
+--!
+--! @internal
+--! Implementation note (issue #354): the CHECK is an INLINE expression, not a
+--! call to `eql_v3_internal.is_valid_ste_vec_entry_payload` — domain
+--! constraints cannot inline SQL functions, so the function-call form paid
+--! the per-call SQL-function executor (~18 µs) on EVERY cast: the needle
+--! cast in every field_eq query (+19% end-to-end vs v2, the entire measured
+--! regression on that scenario; see cipherstash/benches#23). The expression
+--! mirrors the validator body; the leading `VALUE IS NULL OR` preserves the
+--! validator's STRICT NULL-passes semantics (a bare COALESCE(..., false)
+--! would reject NULL, which `->` returns for a missing selector). Keep the
+--! two in sync — `jsonb_entry_check_matches_validator` in tests/sqlx pins
+--! the equivalence.
+--! @endinternal
 CREATE DOMAIN eql_v3.jsonb_entry AS jsonb
   CHECK (
-    eql_v3_internal.is_valid_ste_vec_entry_payload(VALUE)
+    VALUE IS NULL
+    OR COALESCE(
+      jsonb_typeof(VALUE) = 'object'
+       AND jsonb_typeof(VALUE -> 's') = 'string'
+       AND jsonb_typeof(VALUE -> 'c') = 'string'
+       AND (
+         (jsonb_typeof(VALUE -> 'hm') = 'string' AND NOT (VALUE ? 'oc'))
+         OR
+         (jsonb_typeof(VALUE -> 'oc') = 'string' AND NOT (VALUE ? 'hm'))
+       ),
+      false
+    )
   );
 
 --! @brief Domain type for an STE-vec containment needle.
@@ -135,6 +168,17 @@ CREATE DOMAIN eql_v3.jsonb_entry AS jsonb
 --! @note Construct from inline JSON via the DOMAIN cast:
 --!       `'{"sv":[{"s":"<sel>","hm":"<hm>"}]}'::eql_v3.jsonb_query`.
 --! @see eql_v3.to_ste_vec_query
+--!
+--! @internal
+--! Implementation note (issue #354): this CHECK CANNOT be inlined like
+--! eql_v3.jsonb_entry's — validating the sv elements requires a subquery
+--! (`NOT EXISTS (SELECT ... FROM jsonb_array_elements(...))`), and CHECK
+--! constraints forbid subqueries. The validator is plpgsql instead (cached
+--! plan; substantially cheaper per call than a non-inlined LANGUAGE sql
+--! function — the same finding as issue #353), since this cast sits on the
+--! per-query hot path of every containment scenario
+--! (`$1::jsonb::eql_v3.jsonb_query`).
+--! @endinternal
 CREATE DOMAIN eql_v3.jsonb_query AS jsonb
   CHECK (
     eql_v3_internal.is_valid_ste_vec_query_payload(VALUE)

tests/sqlx/tests/encrypted_domain/family/jsonb_check.rs

@@ -0,0 +1,160 @@
+//! Equivalence guards for the inline SteVec domain CHECK expressions
+//! (issue #354).
+//!
+//! `eql_v3.jsonb_entry` carries an INLINE CHECK expression rather than
+//! calling `eql_v3_internal.is_valid_ste_vec_entry_payload`: domain
+//! constraints cannot inline SQL functions, so the function-call form paid
+//! the per-call SQL-function executor on every cast — the needle cast in
+//! every field_eq query, the ENTIRE measured +19% v2→v3 regression on that
+//! scenario (cipherstash/benches#23). `jsonb_entry_check_matches_validator`
+//! pins the inline expression to the validator (still the source of truth
+//! for direct callers) over a corpus of payload shapes; the one intentional
+//! divergence is SQL NULL, which both forms accept (the validator via
+//! STRICT, the inline expression via a leading `VALUE IS NULL OR`).
+//!
+//! `eql_v3.jsonb_query`'s CHECK CANNOT be inlined — validating sv elements
+//! needs a subquery, which CHECK constraints forbid — so its validator is
+//! plpgsql instead (cached plan vs the per-call SQL-function executor; the
+//! issue #353 finding). `jsonb_query_check_behaviour` characterises the
+//! accept/reject matrix, and `jsonb_query_validator_is_plpgsql` guards the
+//! language so a revert to LANGUAGE sql fails here.
+
+use anyhow::Result;
+use sqlx::PgPool;
+
+/// Try the domain cast for `payload`; Ok(true) = accepted, Ok(false) = CHECK
+/// rejection. Any non-CHECK error propagates.
+async fn cast_accepts(pool: &PgPool, domain: &str, payload: Option<&str>) -> Result<bool> {
+    let sql = format!("SELECT ($1::jsonb)::{domain} IS NOT DISTINCT FROM $1::jsonb");
+    match sqlx::query_scalar::<_, bool>(&sql)
+        .bind(payload)
+        .fetch_one(pool)
+        .await
+    {
+        Ok(_) => Ok(true),
+        Err(e) if e.to_string().contains("check constraint") => Ok(false),
+        Err(e) => Err(e.into()),
+    }
+}
+
+/// The validator's verdict for `payload`, with the STRICT NULL-passes rule
+/// applied (SQL NULL is accepted by the domain even though the validator
+/// returns NULL for it).
+async fn validator_accepts(pool: &PgPool, validator: &str, payload: Option<&str>) -> Result<bool> {
+    if payload.is_none() {
+        return Ok(true);
+    }
+    let sql = format!("SELECT eql_v3_internal.{validator}($1::jsonb)");
+    Ok(sqlx::query_scalar::<_, bool>(&sql)
+        .bind(payload)
+        .fetch_one(pool)
+        .await?)
+}
+
+async fn assert_equivalent(
+    pool: &PgPool,
+    domain: &str,
+    validator: &str,
+    candidates: &[Option<&str>],
+) -> Result<()> {
+    for payload in candidates {
+        let cast = cast_accepts(pool, domain, *payload).await?;
+        let valid = validator_accepts(pool, validator, *payload).await?;
+        anyhow::ensure!(
+            cast == valid,
+            "{domain} inline CHECK diverges from {validator} for payload {payload:?}: \
+             cast accepted = {cast}, validator = {valid}"
+        );
+    }
+    Ok(())
+}
+
+#[sqlx::test]
+async fn jsonb_entry_check_matches_validator(pool: PgPool) -> Result<()> {
+    let candidates: &[Option<&str>] = &[
+        // SQL NULL — accepted by both forms (STRICT / VALUE IS NULL OR).
+        None,
+        // Valid: hm entry, oc entry, extra fields allowed.
+        Some(r#"{"s":"sel","c":"ct","hm":"h"}"#),
+        Some(r#"{"s":"sel","c":"ct","oc":"o"}"#),
+        Some(r#"{"s":"sel","c":"ct","hm":"h","a":true,"i":{},"v":3}"#),
+        // Invalid: missing s / missing c / both terms / neither term.
+        Some(r#"{"c":"ct","hm":"h"}"#),
+        Some(r#"{"s":"sel","hm":"h"}"#),
+        Some(r#"{"s":"sel","c":"ct","hm":"h","oc":"o"}"#),
+        Some(r#"{"s":"sel","c":"ct"}"#),
+        // Invalid: non-string term / non-string s / wrong jsonb types.
+        Some(r#"{"s":"sel","c":"ct","hm":1}"#),
+        Some(r#"{"s":1,"c":"ct","hm":"h"}"#),
+        Some(r#""scalar""#),
+        Some("5"),
+        Some("null"),
+        Some("[]"),
+        Some("{}"),
+    ];
+    assert_equivalent(
+        &pool,
+        "eql_v3.jsonb_entry",
+        "is_valid_ste_vec_entry_payload",
+        candidates,
+    )
+    .await
+}
+
+#[sqlx::test]
+async fn jsonb_query_check_behaviour(pool: PgPool) -> Result<()> {
+    // (payload, expected accept) — hardcoded verdicts: the CHECK calls the
+    // validator, so a validator-equivalence assertion would be tautological.
+    let candidates: &[(Option<&str>, bool)] = &[
+        (None, true),
+        // Valid: single- and multi-entry needles; empty sv is valid.
+        (Some(r#"{"sv":[{"s":"sel","hm":"h"}]}"#), true),
+        (
+            Some(r#"{"sv":[{"s":"a","hm":"h"},{"s":"b","oc":"o"}]}"#),
+            true,
+        ),
+        (Some(r#"{"sv":[]}"#), true),
+        // Invalid: element carries a ciphertext / both terms / neither term /
+        // missing s.
+        (Some(r#"{"sv":[{"s":"sel","hm":"h","c":"ct"}]}"#), false),
+        (Some(r#"{"sv":[{"s":"sel","hm":"h","oc":"o"}]}"#), false),
+        (Some(r#"{"sv":[{"s":"sel"}]}"#), false),
+        (Some(r#"{"sv":[{"hm":"h"}]}"#), false),
+        // Invalid: sv not an array / missing sv / non-object roots.
+        (Some(r#"{"sv":{"s":"sel","hm":"h"}}"#), false),
+        (Some("{}"), false),
+        (Some(r#""scalar""#), false),
+        (Some("null"), false),
+        (Some("[]"), false),
+    ];
+    for (payload, expected) in candidates {
+        let cast = cast_accepts(&pool, "eql_v3.jsonb_query", *payload).await?;
+        anyhow::ensure!(
+            cast == *expected,
+            "eql_v3.jsonb_query cast verdict changed for {payload:?}: \
+             accepted = {cast}, expected = {expected}"
+        );
+    }
+    Ok(())
+}
+
+/// The jsonb_query validator must stay plpgsql: its only caller is the domain
+/// CHECK (a context that can never inline a SQL function), so LANGUAGE sql
+/// pays the per-call SQL-function executor on every containment-needle cast
+/// (issues #353/#354). A revert fails here.
+#[sqlx::test]
+async fn jsonb_query_validator_is_plpgsql(pool: PgPool) -> Result<()> {
+    let lang: String = sqlx::query_scalar(
+        "SELECT l.lanname FROM pg_proc p \
+         JOIN pg_language l ON l.oid = p.prolang \
+         WHERE p.proname = 'is_valid_ste_vec_query_payload' \
+           AND p.pronamespace = 'eql_v3_internal'::regnamespace",
+    )
+    .fetch_one(&pool)
+    .await?;
+    anyhow::ensure!(
+        lang == "plpgsql",
+        "is_valid_ste_vec_query_payload must be plpgsql (got {lang})"
+    );
+    Ok(())
+}

tests/sqlx/tests/encrypted_domain/family/mod.rs

@@ -2,6 +2,7 @@
 //! the encrypted-domain family (not integer-specific).
 
 pub mod inlinability;
+pub mod jsonb_check;
 pub mod jsonb_operator_surface;
 pub mod mutations;
 pub mod sem;