Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
148 changes: 148 additions & 0 deletions .github/workflows/agent-e2e.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
name: Agent E2E

# Runs the agent e2e suites (e2e/) against the released Agentspan
# server JAR — a full Conductor server with the agent runtime baked in.
# Port of the Python SDK's proven agent-e2e workflow; JS deltas only
# (Node toolchain, jest runner; Python kept solely for mcp-testkit).
#
# These tests call real LLMs via the OPENAI_API_KEY / ANTHROPIC_API_KEY
# repo secrets. Fork PRs cannot see repo secrets, so for them the run
# fails at the silently-empty guard rather than passing vacuously.

on: [pull_request, workflow_dispatch]

concurrency:
group: agent-e2e-${{ github.ref }}
cancel-in-progress: true

env:
AGENTSPAN_VERSION: "0.4.0" # pinned server/CLI release — bump deliberately

jobs:
agent-e2e:
runs-on: ubuntu-latest
timeout-minutes: 45
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
AGENTSPAN_SERVER_URL: http://localhost:8080/api
AGENTSPAN_CLI_PATH: ${{ github.workspace }}/agentspan
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: 22
cache: npm

- name: Set up Python (mcp-testkit only)
uses: actions/setup-python@v5
with:
python-version: '3.12'
# no `cache: pip` — it requires a requirements.txt/pyproject.toml
# to key on, and this repo has neither (Python is mcp-testkit only)

- name: Set up Java
uses: actions/setup-java@v4
with:
distribution: temurin
java-version: '21'

- name: Cache server JAR
id: jar_cache
uses: actions/cache@v4
with:
path: agentspan-server.jar
key: agentspan-server-${{ env.AGENTSPAN_VERSION }}

- name: Download server JAR from release
if: steps.jar_cache.outputs.cache-hit != 'true'
env:
GH_TOKEN: ${{ github.token }}
run: |
gh release download "v${AGENTSPAN_VERSION}" --repo agentspan-ai/agentspan \
--pattern "agentspan-server-${AGENTSPAN_VERSION}.jar" --output agentspan-server.jar

- name: Download CLI binary from release
env:
GH_TOKEN: ${{ github.token }}
run: |
gh release download "v${AGENTSPAN_VERSION}" --repo agentspan-ai/agentspan \
--pattern "agentspan_linux_amd64" --output agentspan
chmod +x agentspan

- name: Install SDK deps and build
run: |
npm ci
npm run build

- name: Install mcp-testkit
run: |
python -m pip install --upgrade pip
pip install mcp-testkit

- name: Start mcp-testkit
run: |
mcp-testkit --transport http --port 3001 &
sleep 2

- name: Start server
run: |
java -jar agentspan-server.jar --server.port=8080 > server.log 2>&1 &

- name: Wait for server health
run: |
for i in $(seq 1 45); do
if curl -sf http://localhost:8080/health | grep -q '"healthy"[[:space:]]*:[[:space:]]*true'; then
echo "server healthy after ~$((i*2))s"; exit 0
fi
sleep 2
done
echo "::error::server failed to become healthy within 90s"
tail -100 server.log
exit 1

- name: Run e2e suites
run: |
mkdir -p results
npm run test:agent-e2e

# The TS suites fail hard when the server is down (no session-skip),
# so this guard is defense-in-depth against a future gate regression
# or a secrets-less fork run skipping everything.
- name: Guard against silently-empty runs
if: always()
run: |
python - <<'EOF'
import glob
import sys
import xml.etree.ElementTree as ET

total = executed = 0
for path in glob.glob("results/junit-*.xml"):
root = ET.parse(path).getroot()
for suite in root.iter("testsuite"):
t = int(suite.get("tests", 0))
sk = int(suite.get("skipped", 0))
total += t
executed += t - sk
print(f"executed {executed}/{total} tests")
sys.exit(0 if executed > 0 else 1)
EOF

- name: Generate HTML report
if: always()
run: |
npx tsx e2e/generate-report.ts results/junit-e2e.xml results/report.html || true

- name: Upload results
if: always()
uses: actions/upload-artifact@v4
with:
name: agent-e2e-results
path: |
results/
server.log
retention-days: 14
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -85,3 +85,7 @@ fabric.properties
.env.development.local

/.idea

# jest-junit output (unit CI uses reports/, agent e2e uses results/)
reports/
results/
44 changes: 43 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,13 +40,51 @@ src/open-api/ # OpenAPI layer
types.ts # Extended types - add custom fields here
src/integration-tests/ # E2E tests against real Conductor server
utils/ # waitForWorkflowStatus, executeWorkflowWithRetry, etc.
src/agents/ # Durable agent layer (merged Agentspan TS SDK)
index.ts # Agent, AgentRuntime, tool, guardrails, handoffs, ...
frameworks/ # LangGraph/LangChain/generic serializers + detection
testing/ # Agent testing toolkit (/agents/testing subpath)
wrappers/ # Vercel AI / LangGraph / LangChain drop-in wrappers
__tests__/ # Colocated jest unit tests (picked up by test:unit)
e2e/ # Agent e2e suites vs live agentspan server (jest.e2e.config.mjs)
cli-bin/ # agentspan CLI helper scripts (Go CLI walk-up probe target)
examples/agents/ # Agent examples (own tsconfig; run via npx tsx)
docs/agents/ # Agent layer documentation
```

### The agent layer (`src/agents/`)

- **Subpath-only exports**: everything agent-flavored ships via `./agents`,
`./agents/testing`, `./agents/vercel-ai`, `./agents/langgraph`,
`./agents/langchain` (see `exports` in package.json + `tsup.config.ts`).
**Never re-export agent symbols from the root** — the agent layer has an
`Action` class that collides with the OpenAPI-generated `Action` type, and
the root re-exports the whole generated surface, which changes with the
server spec.
- Source keeps upstream's ESM-style `.js`-suffixed relative imports (resolved
by `nodenext` for tsc and a `moduleNameMapper` suffix-strip for jest).
- The only coupling to the workflow layer is in `agent-client.ts` and
`worker.ts` (imports from `../sdk` and `../open-api`) — keep it that way.
- Unit tests are colocated in `src/agents/__tests__/` so the existing
`test:unit` glob and per-PR CI matrix pick them up with zero workflow
changes. Agent e2e lives in repo-root `e2e/` and runs only via
`npm run test:agent-e2e` (its own `.github/workflows/agent-e2e.yml` boots a
pinned release server JAR; needs `OPENAI_API_KEY`/`ANTHROPIC_API_KEY`
secrets).
- Agent examples resolve the package name straight to `src/agents` sources via
`examples/agents/tsconfig.json` paths: `npx tsx examples/agents/<file>.ts`.
Framework subdirs (adk/, langgraph/, openai/, vercel-ai/) install their own
deps (`scripts/install-example-deps.sh`); `examples/agents` is excluded from
the root tsconfig.
- `AGENTSPAN_*` env vars (`AGENTSPAN_SERVER_URL`, default
`http://localhost:8080/api`) are the agent layer's config surface — kept
working as-is; `CONDUCTOR_*` aliases are a possible follow-up.

## Commands

```bash
npm test # Unit tests (482+ tests)
npm run build # tsup (ESM + CJS dual output)
npm run build # tsup (root + 5 agent entries, ESM + CJS) + verify:dist
npm run lint # ESLint
npm run generate-openapi-layer # Regenerate from OpenAPI spec

Expand All @@ -55,6 +93,10 @@ CONDUCTOR_SERVER_URL=http://localhost:8080 \
CONDUCTOR_AUTH_KEY=key CONDUCTOR_AUTH_SECRET=secret \
ORKES_BACKEND_VERSION=5 \
npm run test:integration:orkes-v5

# Agent e2e (requires a running agentspan server + LLM keys; CI does this
# against the pinned release JAR — see .github/workflows/agent-e2e.yml)
AGENTSPAN_SERVER_URL=http://localhost:8080/api npm run test:agent-e2e
```

## Post-Change Verification (Required)
Expand Down
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- **Durable AI agents** -- the Agentspan TypeScript SDK (`@conductor-oss/conductor-agent-sdk`) is merged into this package as new subpath exports: `@io-orkes/conductor-javascript/agents` (core: `Agent`, `AgentRuntime`, `tool`, guardrails, handoffs, memory, schedules, streaming, HITL), `/agents/testing`, and framework wrappers `/agents/vercel-ai`, `/agents/langgraph`, `/agents/langchain`. The agent clients are also first-class citizens of the root export (`AgentClient`, `WorkflowClient`, `Schedule`, `ScheduleInfo`, schedule errors) — the root surface grows additively (minor). Docs at [docs/agents/](docs/agents/README.md); 60+ examples at [examples/agents/](examples/agents/).
- **Migration for `@conductor-oss/conductor-agent-sdk` users:** install `@io-orkes/conductor-javascript` and rewrite import specifiers -- `@conductor-oss/conductor-agent-sdk` becomes `@io-orkes/conductor-javascript/agents`, and the wrapper subpaths gain the `/agents` prefix (e.g. `.../vercel-ai` becomes `.../agents/vercel-ai`). APIs, behavior, and `AGENTSPAN_*` env vars are unchanged.
- New optional peer dependencies (all lazily resolved, install only what you use): `zod`, `zod-to-json-schema`, `ai`, `@langchain/core`, `@langchain/langgraph`. `dotenv` becomes a runtime dependency (the agent entry points load it at import time; the package `sideEffects` field allowlists `dist/agents/**` so bundlers keep that bootstrap).
- **Agent clients in `sdk/clients`** -- `AgentClient` and the agent-flavored `WorkflowClient` moved to `src/sdk/clients/agent/` and are exposed by the factory: `OrkesClients.getAgentClient()` / `getAgentWorkflowClient()`. `AgentClient` accepts an optional pre-built client (`new AgentClient({ client })`) to reuse one Conductor client across surfaces. Naming note: `getWorkflowClient()` returns the general-purpose `WorkflowExecutor`; the agent `WorkflowClient` adds the agent-execution 404 fallback and token-usage rollup.
- **`SchedulerClient` pause/resume works on both server families** -- per-schedule pause/resume verbs differ by Conductor family (OSS/embedded map them PUT-only, Orkes GET-only). `pauseSchedule`/`resumeSchedule` now issue PUT first and retry via GET only on HTTP 405 (stateless per call; the new optional `reason` on pause survives the fallback). Previously the GET-only calls failed against OSS/embedded servers. Admin bulk endpoints are unchanged.
- **`SchedulerClient` typed agent-schedule lifecycle** -- absorbed from the deleted agent `ScheduleClient` (below): `save`/`get`/`listForAgent`/`pause`/`resume`/`delete`/`runNow`/`previewNext`/`reconcile` operating on `Schedule`/`ScheduleInfo` with `${agent}-${name}` wire-name prefixing and typed errors (`ScheduleNotFound`, `ScheduleNameConflict`, `InvalidCronExpression`). The constructor also accepts `PromiseLike<Client>` for callers with async client construction.
- **Canonical metrics** -- opt-in harmonized metric surface via `WORKER_CANONICAL_METRICS=true`. See [METRICS.md](METRICS.md) for the full catalog, configuration, and migration guide.
- Bounded `uri` label on `http_api_client_request_seconds`: canonical mode uses path templates (e.g. `/workflow/{workflowId}`) instead of fully-resolved paths, preventing metric cardinality explosion from dynamic IDs.
- `TaskPaused` event type and `PollerOptions.onPaused` callback: emitted when a poll cycle is skipped because the worker is paused. Canonical mode records `task_paused_total`; legacy mode does not (see Implementation Notes in METRICS.md).
Expand All @@ -24,6 +30,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `http_api_client_request` timing is now recorded automatically by `wrapFetchWithRetry` when a metrics collector is active (via `createMetricsCollector()` or `setHttpMetricsObserver`), covering both successful responses and network-error fallback paths. A lightweight request interceptor captures OpenAPI path templates so the canonical `uri` label uses bounded-cardinality templates in all cases. Previously, `recordApiRequestTime` existed but was not wired into the HTTP pipeline -- [details](METRICS.md#implementation-notes).
- Added optional `durationMs` field to `TaskUpdateFailure` event, recording the duration of the last update attempt. Declared optional so existing event listener implementations are unaffected.

### Removed

- **Agent `ScheduleClient` and `SchedulerFetcher`** (agents subpath; never released, no backward compatibility) -- schedules now ride the SDK `SchedulerClient`, re-exported from `@io-orkes/conductor-javascript/agents`. Method names and signatures are unchanged; migration is the import/type rename only (`ScheduleClient` -> `SchedulerClient`). The `schedules.*` namespace, `runtime.schedulesClient()`, `deploy({ schedules })` and `AgentClient.schedule()` are untouched.

### Deprecated

- Legacy metric names remain the default during the transition period. Migration guidance is in [METRICS.md](METRICS.md#migrating-from-legacy-to-canonical).
40 changes: 40 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ If you find [Conductor](https://github.com/conductor-oss/conductor) useful, plea
* [Examples](#examples)
* [API Journey Examples](#api-journey-examples)
* [AI & LLM Workflows](#ai--llm-workflows)
* [Durable AI Agents](#durable-ai-agents)
* [Documentation](#documentation)
* [Support](#support)
* [Frequently Asked Questions](#frequently-asked-questions)
Expand Down Expand Up @@ -537,11 +538,50 @@ See [examples/agentic-workflows/](examples/agentic-workflows/) for all examples.
| [rag-workflow.ts](examples/advanced/rag-workflow.ts) | End-to-end RAG: document indexing → semantic search → LLM answer |
| [vector-db.ts](examples/advanced/vector-db.ts) | Vector DB operations: embedding generation, storage, search |

## Durable AI Agents

The SDK also ships a durable agent authoring layer — long-running, plan-execute,
and event-driven AI agents whose steps run as Conductor workflow tasks, so they
survive restarts and are observable like any workflow. It lives under the
`/agents` subpath export:

```typescript
import { Agent, AgentRuntime } from "@io-orkes/conductor-javascript/agents";

const agent = new Agent({
name: "greeter",
model: "openai/gpt-4o-mini",
instructions: "You are a friendly assistant. Keep responses brief.",
});

const runtime = new AgentRuntime(); // AGENTSPAN_SERVER_URL, default http://localhost:8080/api
try {
const result = await runtime.run(agent, "Hello! What can you do?");
result.printResult();
} finally {
await runtime.shutdown();
}
```

The layer includes tools (`tool()`, with optional Zod schemas), multi-agent
strategies (sequential, parallel, handoffs, swarm), guardrails,
human-in-the-loop, streaming, memory, schedules, and drop-in wrappers for
agents written with the Vercel AI SDK, LangGraph, and LangChain
(`/agents/vercel-ai`, `/agents/langgraph`, `/agents/langchain`), plus a
testing toolkit (`/agents/testing`).

- **Docs:** [docs/agents/](docs/agents/README.md) — start with
[getting-started.md](docs/agents/getting-started.md)
- **Examples:** [examples/agents/](examples/agents/) — 60+ runnable examples,
from a basic agent to the full [kitchen sink](examples/agents/kitchen-sink.ts);
run them with `npx tsx examples/agents/01-basic-agent.ts`

## Documentation

| Document | Description |
|----------|-------------|
| [SDK Development Guide](SDK_DEVELOPMENT.md) | Architecture, patterns, pitfalls, testing |
| [Durable AI Agents](docs/agents/README.md) | Agent authoring layer: tools, guardrails, multi-agent, HITL, framework wrappers |
| [Metrics Reference](METRICS.md) | Legacy and canonical Prometheus metrics with migration guide |
| [Breaking Changes](BREAKING_CHANGES.md) | v3.x migration guide |
| [Workflow Management](docs/api-reference/workflow-executor.md) | Start, pause, resume, terminate, retry, search, signal |
Expand Down
Loading
Loading