Playground#45
Merged
Merged
Conversation
…or handling for live agents
- Introduced new schemas for tools, observability spans, events, and runs in graph_schemas.py. - Implemented telemetry recording in graph_service.py to capture run traces and events. - Added methods to retrieve tools and observability data from the graph service. - Created an in-memory telemetry store to manage run traces and records. - Integrated telemetry into the graph execution flow, ensuring observability data is captured and retrievable. - Updated setup_router.py to include new evals router. - Refactored store router to improve memory retrieval logic. - Added dummy graph and store implementations for testing and demonstration purposes.
- Introduced `is_realtime` field in `GraphInfoSchema` to indicate if a graph is a live agent. - Updated `GraphService` to surface the `is_realtime` capability in graph details. - Refactored `DummyThreadNameGenerator` to `DefaultThreadNameGenerator` for clarity and updated related imports. - Created `EvalReportService` to read and process evaluation reports from JSON files. - Added unit tests for `EvalReportService` to ensure correct functionality and data handling. - Implemented a dummy live graph in `graph/live.py` to simulate a realtime agent for testing purposes.
- Introduced a route guard mechanism in `route_guard.py` to ensure all non-public routes are protected by `RequirePermission` dependencies, raising errors at startup for any unprotected routes. - Developed integration tests in `conftest.py` to validate the authorization layer with real dependencies, ensuring cross-user access is properly restricted. - Added negative tests for cross-user data access in `test_isolation_idor.py`, confirming that users cannot access each other's resources. - Created unit tests for CORS policy enforcement in `test_cors_policy.py`, ensuring production configurations fail closed against insecure settings. - Implemented unit tests for media ownership in `test_media_ownership.py`, verifying that file access is restricted to owners. - Enhanced ownership authorization logic with unit tests in `test_ownership_authorization.py`, ensuring proper access control based on ownership. - Developed tests for the ownership resolver in `test_ownership_resolver.py`, validating caching and lookup behavior. - Added rate limit keying tests in `test_rate_limit_keying.py` to prevent spoofing of IP addresses through `X-Forwarded-For` headers. - Created tests for the route guard functionality in `test_route_guard.py`, ensuring that all routes are properly protected and public paths are exempted.
- Refactor ownership resolution to improve type checking for value decoding. - Introduce scope enforcement in permissions, ensuring users have the required scopes for actions. - Implement role-based access control (RBAC) backend, allowing roles to map to specific scopes. - Update route guards to include new public paths for evals. - Enhance error handling and logging in various services, including telemetry and ownership cache eviction. - Add tests for scope enforcement and RBAC functionality, ensuring proper authorization checks. - Clean up code formatting and improve readability in several modules.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces several improvements and new features to the CLI, configuration, and documentation for agentflow, with an emphasis on production-readiness, secure-by-default authorization, and enhanced deployment tooling. The most significant changes are the addition of a scalable, cache-backed object-level authorization system (ownership-based), new Kubernetes manifest generation, improved Dockerfile defaults, and safer config file handling in the CLI.
Authorization and Security Enhancements:
AUTHORIZATION_PLAN.md.agentflow.jsonand CLI config generation so that when JWT or custom auth is enabled, the default authorization backend is now"ownership"(owner-only), making production deployments secure by default.CLAUDE.mdto describe the ownership-based authorization backend, its configuration, and its secure-by-construction enforcement at startup.Deployment Tooling Improvements:
k8s.yaml) via the CLI (--k8sflag), with a Deployment and Service. The generated manifest sets a termination grace period and preStop hook to ensure in-flight agent runs are not killed prematurely during rolling deploys. [1] [2] [3] [4] [5] [6] [7].dockerignorefile alongside the Dockerfile, improving Docker build performance and correctness.CLI and Config Improvements:
initcommand now only overwrites an existingagentflow.jsonif--forceis specified, preserving user edits and preventing accidental config loss. [1] [2]storeandobservabilityfields to the default config template to better reflect typical production setups. [1] [2]Documentation and Versioning:
CLAUDE.md) to reflect the new features, the secure-by-default authorization, and the new version (0.4.0). [1] [2]These changes collectively improve the security, reliability, and deployment experience for agentflow users, especially in production environments.