feat(gfql): add GFQL validation framework by lmeyerov · Pull Request #699 · graphistry/pygraphistry

lmeyerov · 2025-07-15T09:06:01Z

Summary

This PR enhances GFQL with built-in validation capabilities integrated directly into the AST classes, providing automatic syntax and schema validation with structured error reporting.

Note: This is a stacked PR on top of #698 (GFQL specification documentation)

Changes

Core Validation System

Built-in validation in AST classes - Validation logic integrated into:
- graphistry/compute/ast.py - Enhanced AST nodes with validation
- graphistry/compute/chain.py - Chain class with validate() method and from_json() validation
- graphistry/compute/ASTSerializable.py - JSON serialization with validation support
Structured exceptions (graphistry/compute/exceptions.py):
- GFQLValidationError base class with error codes
- GFQLSyntaxError (E1xx codes) - Structure and parameter errors
- GFQLTypeError (E2xx codes) - Type mismatches
- GFQLSchemaError (E3xx codes) - Schema validation errors
Schema validation (graphistry/compute/validate_schema.py):
- validate_chain_schema() - Validate operations against DataFrame schemas
- Automatic validation with validate_schema=True (default) in g.chain()
Enhanced predicates - All predicate classes updated with validation logic

External Validation Module (Deprecated)

graphistry/compute/gfql/validate.py - Legacy external validation (marked deprecated)
Provided for backwards compatibility during migration

Documentation

docs/source/gfql/validation/ - Complete validation guide:
- fundamentals.rst - Basic concepts, automatic validation, common errors
- llm.rst - LLM integration patterns using JSON serialization
- production.rst - API patterns, testing, security considerations
docs/source/gfql/validation_migration_guide.md - Migration from external to built-in validation
API docs - Reference documentation for validation modules

Interactive Tutorial

demos/gfql/gfql_validation_fundamentals.ipynb - Jupyter notebook demonstrating:
- Automatic validation during chain construction
- Schema validation against DataFrames
- Error handling patterns
- LLM integration examples

Testing

Comprehensive test coverage:
- test_chain_validation.py - Chain validation tests
- test_chain_schema_validation.py - Schema validation tests
- test_chain_schema_prevalidation.py - Pre-execution validation
- test_ast_serializable_validation.py - JSON parsing validation
- test_gfql_exceptions.py - Exception hierarchy tests
- test_validate.py - Validation API tests

Key Features

Automatic Validation - Queries are validated automatically during construction
Structured Error Codes - E1xx (syntax), E2xx (type), E3xx (schema) for programmatic handling
Fail-Fast by Default - Immediate error reporting with option for collect-all mode
LLM-Friendly - JSON serialization with validation for safe AI integration
Schema-Aware - Validates against actual DataFrame columns and types

Breaking Changes

None - The new validation is integrated into existing classes with backwards compatibility.

Testing

Run the test suite:

pytest graphistry/tests/compute/test_*validation*.py -xvs

Dependencies

This PR depends on #698 being merged first (GFQL specification documentation).

lmeyerov · 2025-07-15T10:15:32Z

This needs pretty thorough redoing, e.g., #700

Add comprehensive validation framework for GFQL queries including syntax and schema validation. ## Python Code - `graphistry/compute/gfql/validate.py` - Core validation module with syntax and schema validators - `graphistry/compute/gfql/exceptions.py` - GFQLValidationError exception class - `graphistry/compute/chain_validate.py` - Chain function with validation support - `graphistry/validate/` - General validation utilities - Tests for all validation functionality ## Documentation - `docs/source/gfql/validation/` - Comprehensive validation guide - fundamentals.rst - Basic validation concepts and examples - advanced.rst - Complex query validation patterns - llm.rst - LLM integration patterns - production.rst - Production deployment patterns - API documentation for validation modules - Updated references in spec and main docs ## Notebook - `demos/gfql/gfql_validation_fundamentals.ipynb` - Interactive tutorial This provides a complete framework for validating GFQL queries at both syntax and schema levels, with helpful error messages to guide users. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>

- Added structured error system with E1xx-E3xx error codes - Updated all AST classes to use new validation pattern - Fixed ASTPredicate validate() method conflicts - Added schema validation to filter_by_dict - Created pre-execution validation capability - Updated all documentation and created migration guide Co-authored-by: Claude <noreply@anthropic.com>

- Remove trailing whitespace - Fix unused imports - Add newlines at end of files - Fix monkey-patch syntax for validate_schema - Format code with black Co-authored-by: Claude <noreply@anthropic.com>

- Fix E402 by moving imports before deprecation warning in validate.py - Add missing newlines at end of test files Co-authored-by: Claude <noreply@anthropic.com>

- Change _get_child_validators return type to List[ASTSerializable] - Add cast for ops list in from_json to fix variance issue Co-authored-by: Claude <noreply@anthropic.com>

- Update test to expect GFQLValidationError instead of AssertionError - Fix test for missing 'keep' parameter (now valid with default) Co-authored-by: Claude <noreply@anthropic.com>

- Replace original gfql_validation_fundamentals.ipynb with updated content - Remove duplicate gfql_validation_fundamentals_updated.ipynb - Update notebook title to remove '(Updated)' suffix The original notebook was using the old external validation module, while the updated version demonstrates the new built-in validation system. Co-authored-by: Claude <noreply@anthropic.com>

- bin/ruff: wrapper that runs flake8 (PyGraphistry's linter) - bin/mypy: wrapper that runs typecheck.sh These provide familiar command names while using the project's actual linting and type checking tools. Co-authored-by: Claude <noreply@anthropic.com>

The bin/ruff and bin/mypy wrapper scripts were not needed. Use the existing bin/lint.sh and bin/typecheck.sh instead. Co-authored-by: Claude <noreply@anthropic.com>

Change validate_schema from False to True by default for better UX: - Fail fast with clear error messages - Prevent wasted computation on invalid queries - Consistent with automatic syntax validation - Users can opt out with validate_schema=False if needed This provides dual-layer protection: 1. Pre-execution validation (fast, clear errors) 2. Runtime validation in filter_by_dict (safety net) Co-authored-by: Claude <noreply@anthropic.com>

…on system - Update fundamentals.rst to show automatic validation during chain construction - Update advanced.rst with complex validation patterns using collect-all mode - Update llm.rst with LLM integration patterns using structured error codes - Update production.rst with production-ready patterns and security considerations - Add deprecation notice to api/gfql/validate.rst pointing to new system - All docs now reflect validate_schema=True default and structured error codes 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>

Fix RST syntax error where "Pre-execution Validation" title underline was too short. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>

Remove ✅, ❌, ⚠️, and 💡 emoji characters that cause LaTeX compilation errors in PDF documentation builds. Replace with plain text equivalents. Fixes LaTeX errors: - Unicode character ✅ (U+2705) not set up for use with LaTeX - Unicode character ❌ (U+274C) not set up for use with LaTeX - Unicode character ⚠️ (U+26A0) not set up for use with LaTeX - Unicode character 💡 (U+1F4A1) not set up for use with LaTeX 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>

Remove ✅, ❌, and 💡 emoji characters from gfql_validation_fundamentals.ipynb that were causing LaTeX compilation errors in PDF documentation builds. Replace with plain text equivalents to maintain readability while fixing: - Unicode character ✅ (U+2705) not set up for use with LaTeX - Unicode character ❌ (U+274C) not set up for use with LaTeX - Unicode character 💡 (U+1F4A1) not set up for use with LaTeX 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>

…() in validation notebook Update gfql_validation_fundamentals.ipynb to use the canonical API pattern: - Remove imports of edges and nodes functions - Use graphistry.edges() instead of edges() - Use graphistry.nodes() instead of nodes() - Add comment explaining the canonical usage pattern This follows the recommended PyGraphistry API usage pattern and maintains consistency with documentation examples. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>

- Reverted single to double quote changes in ast.py and chain.py - These changes don't improve functionality and muddy git history - Kept functional changes for validation system intact

- Removed advanced.rst entirely as most content was incorrect - Named operations are unrelated to validation - Temporal predicates like 'after' don't exist - Nested predicates (and_, or_) don't exist - Custom validation not supported in built-in system - No warnings, only exceptions in new validation - Error collection already covered in fundamentals - Schema evolution was just a restatement of schema validation The minimal valid content (pre-execution validation, bounded traversals) is already covered or can be added to fundamentals if needed.

- Added temporal comparison examples using gt() with pd.Timestamp - Emphasized default automatic validation behavior - Made pre-execution validation clearly marked as advanced use - Marked chain_with_validation as deprecated (uses old system) - Removed references to non-existent features

- Removed chain_validate.py and its test entirely - This used the old validation system which is deprecated - New validation is built into chain() by default

Fixed syntax errors from overzealous comma removal: - hyper_dask.py: Added commas after engine: Engine parameters - layout_non_bulk.py: Added comma after engine: Engine parameter 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>

- Use single quotes for getattr/kind comparisons to match master - Keep double quotes for logger.debug calls (correct) - Fix merge operations to use single quotes for 'left' - Fix index-related strings to use single quotes - Preserve all functional validation logic changes

lmeyerov · 2025-07-21T17:57:12Z

@@ -0,0 +1,203 @@
+"""Test script for GFQL validation helpers."""


should move into validate/

lmeyerov · 2025-07-21T17:57:16Z

@@ -0,0 +1,167 @@
+"""Tests for GFQL validation exceptions."""


should move into validate/

lmeyerov · 2025-07-21T17:57:38Z

@@ -0,0 +1,215 @@
+"""Schema validation for GFQL chains without execution."""


should move into validate/

lmeyerov · 2025-07-21T17:59:02Z

@@ -0,0 +1,62 @@
+"""GFQL-specific exceptions for validation and error handling."""


this probably belongs in a models/ folder so always safe to statically import

lmeyerov · 2025-07-21T17:59:33Z

    df = df_to_engine(df, engine_concrete)
    logger.debug('filter_by_dict engine: %s => %s', engine, engine_concrete)

+    from graphistry.compute.exceptions import ErrorCode, GFQLSchemaError


should be top-level import

lmeyerov · 2025-07-21T17:59:56Z

+from typing import Optional, Any, Dict
+
+
+class ErrorCode:


move into a models/?

- Import validate_chain_schema - Add validate_schema: bool = False parameter - Call validation if parameter is True - Add parameter documentation - Total diff: 39 lines (well under 100 line target)

…or type safety

- Moved validate_schema.py to compute/validate/ for better organization - Updated import in chain.py to use new location - Added proper __init__.py for compute/validate module 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>

- Move validate_schema method to Chain class to avoid circular import - Fix Chain validation to properly raise GFQLTypeError/GFQLSyntaxError - Update _validate_fields to raise proper exceptions instead of asserts - Override validate() to collect all errors when collect_all=True - Update from_json to raise GFQLSyntaxError instead of asserts - Pass validate parameter through to ASTObject_from_json All tests now pass with proper error handling and no circular imports.

- Remove string quotes from List['ASTSerializable'] return type annotation - Add proper type casting in validate_schema.py for Chain handling

- Add clear distinction between three validation methods: 1. Validate-only using validate_chain_schema() - no execution 2. Runtime validation (automatic) - validates during execution 3. Validate-and-run using validate_schema=True - pre-execution validation - Add explanatory comments to make the differences clear - Improve error handling examples with clearer output messages 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>

…ion Validation - Update Pre-Execution Validation section to clearly distinguish the two methods: - Method 1: validate_chain_schema() for validation-only (no execution) - Method 2: g.gfql(..., validate_schema=True) for validate-and-run - Add example demonstrating the difference between the two approaches - Update both notebook and RST documentation for consistency - Add visual indicators (✓/✗) to make the distinction clearer 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>

Replace ✓ and ✗ characters with (check) and (x) in gfql_validation_fundamentals.ipynb to fix LaTeX PDF build errors in ReadTheDocs.

The test was importing from graphistry.compute.validate_schema but the module is actually at graphistry.compute.validate.validate_schema 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>

…ESS/ This is a working document for tracking PR stack changes and should not be part of the repository. Moving it to AI_PROGRESS/ which is already gitignored. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>

lmeyerov force-pushed the dev/gfql-validation-stack branch from 18434ef to faf9815 Compare July 15, 2025 10:14

lmeyerov added the WIP label Jul 15, 2025

lmeyerov mentioned this pull request Jul 15, 2025

docs(gfql): add GFQL specification documentation #698

Merged

Base automatically changed from dev/gfql-spec-only to master July 15, 2025 10:26

lmeyerov force-pushed the dev/gfql-validation-stack branch from faf9815 to d91ca95 Compare July 16, 2025 00:29

lmeyerov commented Jul 16, 2025

View reviewed changes

Comment thread docs/source/gfql/spec/python_embedding.md

lmeyerov commented Jul 16, 2025

View reviewed changes

Comment thread demos/gfql/gfql_validation_fundamentals_updated.ipynb Outdated

lmeyerov commented Jul 16, 2025

View reviewed changes

Comment thread docs/source/gfql/spec/python_embedding.md Outdated

lmeyerov added the gpu-ci label Jul 17, 2025

lmeyerov and others added 20 commits July 17, 2025 22:42

fix: clean up linting and type issues in GFQL validation

5e2c6f0

- Remove trailing whitespace - Fix unused imports - Add newlines at end of files - Fix monkey-patch syntax for validate_schema - Format code with black Co-authored-by: Claude <noreply@anthropic.com>

fix: resolve remaining linting issues

59e8624

- Fix E402 by moving imports before deprecation warning in validate.py - Add missing newlines at end of test files Co-authored-by: Claude <noreply@anthropic.com>

fix: resolve mypy type checking issues in chain.py

069fb65

- Change _get_child_validators return type to List[ASTSerializable] - Add cast for ops list in from_json to fix variance issue Co-authored-by: Claude <noreply@anthropic.com>

fix: update test_from_json to handle new validation errors

7138b56

- Update test to expect GFQLValidationError instead of AssertionError - Fix test for missing 'keep' parameter (now valid with default) Co-authored-by: Claude <noreply@anthropic.com>

remove: delete unnecessary convenience scripts

d4500c4

The bin/ruff and bin/mypy wrapper scripts were not needed. Use the existing bin/lint.sh and bin/typecheck.sh instead. Co-authored-by: Claude <noreply@anthropic.com>

docs(gfql): fix title underline length in advanced.rst

ff9c5a9

Fix RST syntax error where "Pre-execution Validation" title underline was too short. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>

style: revert superficial quote changes to maintain code history

ce28a52

- Reverted single to double quote changes in ast.py and chain.py - These changes don't improve functionality and muddy git history - Kept functional changes for validation system intact

cleanup: remove dead code chain_with_validation

24dfd3a

- Removed chain_validate.py and its test entirely - This used the old validation system which is deprecated - New validation is built into chain() by default

docs: remove reference to non-existent and_/or_ predicates in LLM guide

b01cf38

lmeyerov force-pushed the dev/gfql-validation-stack branch from 2f9b821 to d1f1fed Compare July 18, 2025 05:44

lmeyerov removed the WIP label Jul 18, 2025

lmeyerov requested review from aucahuasi and mj3cheun July 18, 2025 06:27

lmeyerov mentioned this pull request Jul 19, 2025

feat(gfql): PR 1.2 - Basic Working DAG execution #706

Closed

lmeyerov commented Jul 21, 2025

View reviewed changes

lmeyerov and others added 15 commits July 21, 2025 13:39

fix: add minimal validate_schema parameter to chain() function

22a1822

- Import validate_chain_schema - Add validate_schema: bool = False parameter - Call validation if parameter is True - Add parameter documentation - Total diff: 39 lines (well under 100 line target)

fix: correct validate_schema default to True as per original design

b48bf22

fix: align Chain class signatures with ASTSerializable parent class f…

0a91d90

…or type safety

fix: add newline to end of __init__.py file (lint fix)

42221ce

fix: remove blank line at end of file (lint fix)

2732975

fix: resolve type annotation and mypy issues in chain validation

d941db2

- Remove string quotes from List['ASTSerializable'] return type annotation - Add proper type casting in validate_schema.py for Chain handling

fix: add newline at end of __init__.py to fix W292 lint error

861735f

fix(docs): replace Unicode checkmarks with LaTeX-safe alternatives

80b42ad

Replace ✓ and ✗ characters with (check) and (x) in gfql_validation_fundamentals.ipynb to fix LaTeX PDF build errors in ReadTheDocs.

docs(changelog): 0.40.0

6ef3d28

lmeyerov merged commit daa88e5 into master Jul 23, 2025
62 of 63 checks passed

lmeyerov deleted the dev/gfql-validation-stack branch July 23, 2025 08:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(gfql): add GFQL validation framework#699

feat(gfql): add GFQL validation framework#699
lmeyerov merged 55 commits into
masterfrom
dev/gfql-validation-stack

lmeyerov commented Jul 15, 2025 •

edited

Loading

Uh oh!

lmeyerov commented Jul 15, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lmeyerov Jul 21, 2025

Uh oh!

lmeyerov Jul 21, 2025

Uh oh!

lmeyerov Jul 21, 2025

Uh oh!

lmeyerov Jul 21, 2025

Uh oh!

lmeyerov Jul 21, 2025

Uh oh!

lmeyerov Jul 21, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		@@ -0,0 +1,203 @@
		"""Test script for GFQL validation helpers."""

		@@ -0,0 +1,167 @@
		"""Tests for GFQL validation exceptions."""

		@@ -0,0 +1,215 @@
		"""Schema validation for GFQL chains without execution."""

		@@ -0,0 +1,62 @@
		"""GFQL-specific exceptions for validation and error handling."""

Uh oh!

Conversation

lmeyerov commented Jul 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Core Validation System

External Validation Module (Deprecated)

Documentation

Interactive Tutorial

Testing

Key Features

Breaking Changes

Testing

Dependencies

Uh oh!

lmeyerov commented Jul 15, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lmeyerov Jul 21, 2025

Choose a reason for hiding this comment

Uh oh!

lmeyerov Jul 21, 2025

Choose a reason for hiding this comment

Uh oh!

lmeyerov Jul 21, 2025

Choose a reason for hiding this comment

Uh oh!

lmeyerov Jul 21, 2025

Choose a reason for hiding this comment

Uh oh!

lmeyerov Jul 21, 2025

Choose a reason for hiding this comment

Uh oh!

lmeyerov Jul 21, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

lmeyerov commented Jul 15, 2025 •

edited

Loading