feat(gfql): add GFQL validation framework

CHANGELOG.md

@@ -7,6 +7,20 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
 
 ## Dev
 
+## [0.40.0 - 2025-07-23]
+
+### Added
+* GFQL: Add comprehensive validation framework with detailed error reporting
+  * Built-in validation: `Chain()` constructor validates syntax automatically
+  * Schema validation: `validate_chain_schema()` validates queries against DataFrame schemas
+  * Pre-execution validation: `g.chain(ops, validate_schema=True)` catches errors before execution
+  * Structured error types: `GFQLValidationError`, `GFQLSyntaxError`, `GFQLTypeError`, `GFQLSchemaError`
+  * Error codes (E1xx syntax, E2xx type, E3xx schema) for programmatic error handling
+  * Collect-all mode: `validate(collect_all=True)` returns all errors instead of fail-fast
+  * JSON validation: `Chain.from_json()` validates during parsing for safe LLM integration
+  * Helpful error suggestions for common mistakes
+  * Example notebook: `demos/gfql/gfql_validation_fundamentals.ipynb`
+
 ### Fixed
 * Engine: Fix resolve_engine() to use dynamic import for Plottable isinstance check to avoid Jinja dependency from pandas df.style getter (#701)
 * Engine: Fix resolve_engine() to check both Plotter and Plottable classes for proper type detection
@@ -23,6 +37,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
   * Wire protocol JSON format for client-server communication
   * Fix terminology: clarify g._node (node ID column) vs g._nodes (DataFrame)
   * Emphasize GFQL's declarative nature for graph-to-graph transformations
+  * Add validation framework documentation with error code reference
 
 ## [0.39.1 - 2025-07-07]
 

ai_code_notes/gfql/README.md

@@ -37,6 +37,26 @@ g.chain([n(), e_forward(), n()])      # Pattern matching
 - Prefer `filter_dict` over `query` strings
 - Use appropriate engine: `pandas` (CPU) or `cudf` (GPU)
 
+### Validation (Built-in)
+```python
+# Automatic syntax validation
+chain = Chain([n(), e_forward(hops=-1)])  # Raises GFQLTypeError
+
+# Schema validation at runtime
+g.chain([n({'missing': 'value'})])  # Raises GFQLSchemaError
+
+# Pre-execution validation
+g.chain(ops, validate_schema=True)  # Validate before execution
+
+# Collect all errors
+errors = chain.validate(collect_all=True)
+```
+
+Error types:
+- `GFQLSyntaxError` (E1xx): Structural issues
+- `GFQLTypeError` (E2xx): Type mismatches  
+- `GFQLSchemaError` (E3xx): Missing columns, wrong types
+
 ## 📋 When to Use GFQL
 
 ### Use GFQL When

demos/gfql/gfql_validation_fundamentals.ipynb

@@ -0,0 +1,214 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": "# GFQL Validation Fundamentals\n\nLearn how to use GFQL's built-in validation system to catch errors early and build robust graph applications.\n\n## What You'll Learn\n- How GFQL automatically validates queries\n- Understanding structured error messages with error codes\n- Schema validation against your data\n- Pre-execution validation for performance\n- Collecting all errors vs fail-fast mode\n\n## Prerequisites\n- Basic Python knowledge\n- PyGraphistry installed (`pip install graphistry[ai]`)"
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Setup and Imports\n",
+    "\n",
+    "First, let's import the necessary modules and create sample data."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": "# Core imports\nimport pandas as pd\nimport graphistry\nfrom graphistry.compute.chain import Chain\nfrom graphistry.compute.ast import n, e_forward, e_reverse\n\n# Exception types for error handling\nfrom graphistry.compute.exceptions import (\n    GFQLValidationError,\n    GFQLSyntaxError,\n    GFQLTypeError,\n    GFQLSchemaError,\n    ErrorCode\n)\n\n# Check version\nprint(f\"PyGraphistry version: {graphistry.__version__}\")\nprint(\"\\nValidation is now built-in to GFQL operations!\")"
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Automatic Syntax Validation\n",
+    "\n",
+    "GFQL validates operations automatically when you create them. No need to call separate validation functions!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": "# Example 1: Valid chain creation\ntry:\n    chain = Chain([\n        n({'type': 'customer'}),\n        e_forward(),\n        n()\n    ])\n    print(\"Valid chain created successfully!\")\n    print(f\"Chain has {len(chain.chain)} operations\")\nexcept GFQLValidationError as e:\n    print(f\"Validation error: {e}\")"
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": "# Example 2: Invalid parameter - negative hops\ntry:\n    chain = Chain([\n        n(),\n        e_forward(hops=-1),  # Invalid: negative hops\n        n()\n    ])\nexcept GFQLTypeError as e:\n    print(f\"Caught validation error!\")\n    print(f\"   Error code: {e.code}\")\n    print(f\"   Message: {e.message}\")\n    print(f\"   Field: {e.context.get('field')}\")\n    print(f\"   Suggestion: {e.context.get('suggestion')}\")"
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Understanding Error Codes\n",
+    "\n",
+    "GFQL uses structured error codes for programmatic handling:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Display available error codes\n",
+    "print(\"Error Code Categories:\")\n",
+    "print(\"\\nE1xx - Syntax Errors:\")\n",
+    "print(f\"  {ErrorCode.E101}: Invalid type (e.g., chain not a list)\")\n",
+    "print(f\"  {ErrorCode.E103}: Invalid parameter value\")\n",
+    "print(f\"  {ErrorCode.E104}: Invalid direction\")\n",
+    "print(f\"  {ErrorCode.E105}: Missing required field\")\n",
+    "\n",
+    "print(\"\\nE2xx - Type Errors:\")\n",
+    "print(f\"  {ErrorCode.E201}: Type mismatch\")\n",
+    "print(f\"  {ErrorCode.E204}: Invalid name type\")\n",
+    "\n",
+    "print(\"\\nE3xx - Schema Errors:\")\n",
+    "print(f\"  {ErrorCode.E301}: Column not found\")\n",
+    "print(f\"  {ErrorCode.E302}: Incompatible column type\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Create Sample Data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": "# Create sample data\nnodes_df = pd.DataFrame({\n    'id': ['a', 'b', 'c', 'd', 'e'],\n    'name': ['Alice', 'Bob', 'Charlie', 'David', 'Eve'],\n    'type': ['customer', 'customer', 'product', 'product', 'customer'],\n    'score': [100, 85, 95, 120, 110],\n    'active': [True, True, False, True, False]\n})\n\nedges_df = pd.DataFrame({\n    'src': ['a', 'b', 'c', 'd', 'e'],\n    'dst': ['c', 'd', 'a', 'b', 'c'],\n    'weight': [1.0, 2.5, 0.8, 1.2, 3.0],\n    'edge_type': ['buys', 'buys', 'recommends', 'recommends', 'buys']\n})\n\n# Create graph using canonical graphistry.edges() and graphistry.nodes()\ng = graphistry.edges(edges_df, 'src', 'dst').nodes(nodes_df, 'id')\n\nprint(\"Graph created with:\")\nprint(f\"  Nodes: {len(g._nodes)} (columns: {list(g._nodes.columns)})\")\nprint(f\"  Edges: {len(g._edges)} (columns: {list(g._edges.columns)})\")"
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Schema Validation (Runtime)\n",
+    "\n",
+    "When you execute a chain, GFQL automatically validates against your data schema:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": "# Valid query - columns exist\ntry:\n    result = g.chain([\n        n({'type': 'customer'}),\n        e_forward({'edge_type': 'buys'}),\n        n({'type': 'product'})\n    ])\n    print(f\"Query executed successfully!\")\n    print(f\"   Found {len(result._nodes)} nodes\")\n    print(f\"   Found {len(result._edges)} edges\")\nexcept GFQLSchemaError as e:\n    print(f\"Schema error: {e}\")"
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": "# Invalid query - column doesn't exist\ntry:\n    result = g.chain([\n        n({'category': 'VIP'})  # 'category' column doesn't exist\n    ])\nexcept GFQLSchemaError as e:\n    print(f\"Schema validation caught the error!\")\n    print(f\"   Error code: {e.code}\")\n    print(f\"   Message: {e.message}\")\n    print(f\"   Suggestion: {e.context.get('suggestion')}\")"
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Type Mismatch Detection\n",
+    "\n",
+    "GFQL detects when you use the wrong type of value or predicate for a column:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": "# Type mismatch: string value on numeric column\ntry:\n    result = g.chain([\n        n({'score': 'high'})  # 'score' is numeric, not string\n    ])\nexcept GFQLSchemaError as e:\n    print(f\"Type mismatch detected!\")\n    print(f\"   {e}\")\n    print(f\"\\n   Column type: {e.context.get('column_type')}\")"
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": "# Using predicates\nfrom graphistry.compute.predicates.numeric import gt\nfrom graphistry.compute.predicates.str import contains\n\n# Correct: numeric predicate on numeric column\ntry:\n    result = g.chain([n({'score': gt(90)})])\n    print(f\"Valid: Found {len(result._nodes)} high-scoring nodes\")\nexcept GFQLSchemaError as e:\n    print(f\"Error: {e}\")\n\n# Wrong: string predicate on numeric column\ntry:\n    result = g.chain([n({'score': contains('9')})])\nexcept GFQLSchemaError as e:\n    print(f\"\\nPredicate type mismatch caught!\")\n    print(f\"   {e.message}\")\n    print(f\"   Suggestion: {e.context.get('suggestion')}\")"
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": "## Pre-Execution Validation\n\nYou have two options for validating queries:\n\n1. **Validate-only** (no execution): Use `validate_chain_schema()` to check compatibility without running the query\n2. **Validate-and-run**: Use `g.chain(..., validate_schema=True)` to validate before execution\n\nThis is useful for catching errors early, especially in production systems."
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": "# Pre-validate to catch errors early\nchain_to_test = Chain([\n    n({'missing_col': 'value'}),\n    e_forward({'also_missing': 'value'})\n])\n\n# Method 1: Validate AND run (stops at validation if invalid)\nprint(\"Method 1: Validate-and-run with validate_schema=True\")\ntry:\n    result = g.chain(chain_to_test.chain, validate_schema=True)\n    print(\"Query executed successfully\")\nexcept GFQLSchemaError as e:\n    print(\"Pre-execution validation caught error!\")\n    print(f\"   Error: {e}\")\n    print(\"   (check) No graph operations were performed\")\n    print(\"   (check) Query was rejected before execution\")"
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": "# Method 2: Validate ONLY (no execution)\nprint(\"\\nMethod 2: Validate-only with validate_chain_schema()\")\nfrom graphistry.compute.validate_schema import validate_chain_schema\n\n# Check if chain is compatible with graph schema WITHOUT running it\ntry:\n    validate_chain_schema(g, chain_to_test)\n    print(\"Chain is valid for this graph schema\")\n    print(\"Note: No query execution occurred - only validation!\")\nexcept GFQLSchemaError as e:\n    print(f\"Schema incompatibility detected: {e}\")\n    print(\"   (check) This was validation-only - no query was executed\")\n    print(\"   (check) Use this method to test queries before running them\")"
+  },
+  {
+   "cell_type": "code",
+   "source": "# Example: Demonstrating the difference\nprint(\"=== Demonstrating the difference ===\\n\")\n\n# Create a valid chain\nvalid_chain = Chain([\n    n({'type': 'customer'}),\n    e_forward()\n])\n\n# Validate-only: Just checks, doesn't run\nprint(\"1. Validate-only with validate_chain_schema():\")\ntry:\n    validate_chain_schema(g, valid_chain)\n    print(\"   (check) Validation passed\")\n    print(\"   (check) Query NOT executed\")\n    print(\"   (check) No result object returned\")\nexcept GFQLSchemaError as e:\n    print(f\"   (x) Validation failed: {e}\")\n\n# Validate-and-run: Validates first, then executes if valid\nprint(\"\\n2. Validate-and-run with g.chain(..., validate_schema=True):\")\ntry:\n    result = g.chain(valid_chain.chain, validate_schema=True)\n    print(\"   (check) Validation passed\")\n    print(\"   (check) Query WAS executed\")\n    print(f\"   (check) Result: {len(result._nodes)} nodes, {len(result._edges)} edges\")\nexcept GFQLSchemaError as e:\n    print(f\"   (x) Validation failed: {e}\")\n    print(\"   (x) Query NOT executed\")",
+   "metadata": {},
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Collect All Errors vs Fail-Fast\n",
+    "\n",
+    "By default, validation fails on the first error. You can collect all errors instead:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": "# Create a chain with multiple errors\nproblematic_chain = Chain([\n    n({'missing1': 'value', 'score': 'not-a-number'}),  # 2 errors\n    e_forward({'missing2': 'value'}),  # 1 error  \n    n({'type': gt(5)})  # 1 error: numeric predicate on string column\n])\n\n# Fail-fast mode (default)\nprint(\"Fail-fast mode:\")\ntry:\n    problematic_chain.validate()\nexcept GFQLValidationError as e:\n    print(f\"  Stopped at first error: {e}\")\n\n# Collect-all mode\nprint(\"\\nCollect-all mode:\")\nerrors = problematic_chain.validate(collect_all=True)\nprint(f\"  Found {len(errors)} syntax/type errors\")\n\n# For schema validation\nschema_errors = validate_chain_schema(g, problematic_chain, collect_all=True)\nprint(f\"  Found {len(schema_errors)} schema errors:\")\nfor i, error in enumerate(schema_errors):\n    print(f\"\\n  Error {i+1}: [{error.code}] {error.message}\")\n    if error.context.get('suggestion'):\n        print(f\"    Suggestion: {error.context['suggestion']}\")"
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Error Handling Best Practices"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": "# Comprehensive error handling example\ndef safe_chain_execution(g, operations):\n    \"\"\"Execute chain with proper error handling.\"\"\"\n    try:\n        # Create chain\n        chain = Chain(operations)\n        \n        # Pre-validate if desired\n        # errors = chain.validate_schema(g, collect_all=True)\n        # if errors:\n        #     print(f\"Warning: {len(errors)} schema issues found\")\n        \n        # Execute\n        result = g.chain(operations)\n        return result\n        \n    except GFQLSyntaxError as e:\n        print(f\"Syntax Error [{e.code}]: {e.message}\")\n        if e.context.get('suggestion'):\n            print(f\"  Try: {e.context['suggestion']}\")\n        return None\n        \n    except GFQLTypeError as e:\n        print(f\"Type Error [{e.code}]: {e.message}\")\n        print(f\"  Field: {e.context.get('field')}\")\n        print(f\"  Value: {e.context.get('value')}\")\n        return None\n        \n    except GFQLSchemaError as e:\n        print(f\"Schema Error [{e.code}]: {e.message}\")\n        if e.code == ErrorCode.E301:\n            print(\"  Column not found in data\")\n        elif e.code == ErrorCode.E302:\n            print(\"  Type mismatch between query and data\")\n        return None\n\n# Test with valid query\nprint(\"Valid query:\")\nresult = safe_chain_execution(g, [\n    n({'type': 'customer'}),\n    e_forward()\n])\nif result:\n    print(f\"  Success! Found {len(result._nodes)} nodes\")\n\n# Test with invalid query\nprint(\"\\nInvalid query:\")\nresult = safe_chain_execution(g, [\n    n({'invalid_column': 'value'})\n])"
+  },
+  {
+   "cell_type": "markdown",
+   "source": "## Summary\n\n### Key Takeaways\n\n1. **Automatic Validation**: GFQL validates automatically - no separate validation calls needed\n2. **Structured Errors**: Error codes (E1xx, E2xx, E3xx) help with programmatic handling\n3. **Helpful Messages**: Errors include suggestions for fixing issues\n4. **Two Validation Stages**:\n   - Syntax/Type: During chain construction\n   - Schema: During execution (or pre-execution)\n5. **Flexible Modes**: Choose between fail-fast or collect-all errors\n\n### Quick Reference\n\n```python\n# Automatic syntax validation\nchain = Chain([...])  # Validates syntax/types\n\n# Runtime schema validation  \nresult = g.chain([...])  # Validates against data\n\n# Pre-execution schema validation\nresult = g.chain([...], validate_schema=True)\n\n# Validate chain against graph schema\nfrom graphistry.compute.validate_schema import validate_chain_schema\nvalidate_chain_schema(g, chain)  # Throws GFQLSchemaError if invalid\n\n# Collect all syntax errors\nerrors = chain.validate(collect_all=True)\n\n# Collect all schema errors\nschema_errors = validate_chain_schema(g, chain, collect_all=True)\n```\n\n### Next Steps\n\n- Explore more complex query patterns\n- Learn about GFQL predicates for advanced filtering\n- Use validation in production applications",
+   "metadata": {}
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python",
+   "version": "3.8.0"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

docs/source/api/gfql/validate.rst

@@ -0,0 +1,12 @@
+graphistry.compute.gfql.validate module
+=======================================
+
+.. deprecated:: 2.0.0
+   This external validation module is deprecated. GFQL now has built-in validation that happens automatically during chain construction and execution.
+
+   See :doc:`../../gfql/validation/fundamentals` for the new validation system and :doc:`../../gfql/validation_migration_guide` for migration instructions.
+
+.. automodule:: graphistry.compute.gfql.validate
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/source/gfql/index.rst

@@ -28,3 +28,4 @@ See also:
    :caption: Developer Resources
 
    spec/index
+   validation/index

docs/source/gfql/spec/language.md

@@ -374,4 +374,5 @@ This pattern is essential for extracting specific subsets from complex graph tra
 - {ref}`gfql-spec-python-embedding` - Python implementation details
 - {ref}`gfql-spec-wire-protocol` - JSON serialization format
 - {ref}`gfql-spec-cypher-mapping` - Cypher to GFQL translation with wire protocol
-- [GFQL Quick Reference](../quick.rst) - Comprehensive examples and usage patterns
+- [GFQL Quick Reference](../quick.rst) - Comprehensive examples and usage patterns
+- [GFQL Validation Guide](../validation/fundamentals.rst) - Learn validation basics