Skip to content

feat(sandbox,providers): add aws-bedrock as a recognized inference provider#1704

Open
st-gr wants to merge 8 commits into
NVIDIA:mainfrom
st-gr:feat/aws-bedrock-provider
Open

feat(sandbox,providers): add aws-bedrock as a recognized inference provider#1704
st-gr wants to merge 8 commits into
NVIDIA:mainfrom
st-gr:feat/aws-bedrock-provider

Conversation

@st-gr

@st-gr st-gr commented Jun 3, 2026

Copy link
Copy Markdown

Summary

Adds aws-bedrock as a recognized inference protocol in the supervisor's L7 router and the providers catalog so operators can register a Bedrock-shaped upstream as --type aws-bedrock and route Claude Code Bedrock-mode traffic (POST /model/{id}/invoke[-with-response-stream]) through inference.local. Without this, sandboxes hit 403 "connection not allowed by policy" because no L7 pattern matches Bedrock URLs. The canonical no-SigV4 use case is SAP AI Core deployed Bedrock models (Anthropic models behind a Bedrock-shape API with XSUAA bearer auth instead of SigV4); operators wanting real AWS Bedrock additionally need #1630's proxy-side SigV4 signing.

Related Issue

Complementary to #1630 ("Sigv4 credential signing"). #1630 adds proxy-side SigV4 re-signing as a credential_signing: sigv4 policy field. This PR is the URL-pattern half: the supervisor's L7 router needs to recognize Bedrock InvokeModel paths before anything can be signed, regardless of whether the upstream needs SigV4. The two patches don't touch the same files; they're complementary, not overlapping.

No upstream tracking issue filed — happy to file one if reviewers prefer.

Changes

  • crates/openshell-sandbox/src/l7/inference.rs:
    • Adds two patterns to default_patterns(): POST /model/*/invoke (aws_bedrock_invoke) and POST /model/*/invoke-with-response-stream (aws_bedrock_invoke_stream).
    • Extends detect_inference_pattern to support a single middle /*/ glob in addition to the existing trailing /*. The middle wildcard matches exactly one non-empty path segment containing no //model//invoke and /model/a/b/invoke both no-match.
  • providers/aws-bedrock.yaml: new YAML profile declaring four credentials (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN, AWS_REGION) and a default endpoint of bedrock-runtime.us-east-1.amazonaws.com:443. Operators in other regions or pointing at non-AWS Bedrock-compatible upstreams override per-deployment via the operator-supplied BEDROCK_BASE_URL config-key (mirroring how the anthropic provider accepts ANTHROPIC_BASE_URL).
  • crates/openshell-providers/src/providers/aws_bedrock.rs: the ProviderDiscoverySpec so --auto-providers picks up AWS_* env vars from local credentials.
  • crates/openshell-providers/src/{providers/mod.rs,lib.rs,profiles.rs}: register the new module + SPEC + YAML.

Testing

  • mise run pre-commit passes — not run end-to-end (mise not on author's dev environment), but the equivalent rust pieces verified independently: cargo fmt --all -- --check clean; cargo clippy --no-deps -p openshell-providers -p openshell-sandbox --all-targets -- -D warnings clean.
  • Unit tests added/updated — 7 new pattern-matcher tests in crates/openshell-sandbox/src/l7/inference.rs::tests cover positive path, query-string handling, GET rejection, empty-segment rejection, multi-segment rejection, unknown-action rejection. Provider-discovery test follows the existing test_discovers_env_credential! macro convention. cargo test -p openshell-sandbox --lib l7::inference: 40 passed; cargo test -p openshell-providers: 35 passed.
  • E2E tests added/updated — none — running an aws-bedrock provider end-to-end requires either real AWS Bedrock with SigV4 (covered by Sigv4 credential signing #1630) or a Bedrock-compatible stub backend. Deferring the E2E test to whichever PR lands second so it can exercise the full URL-pattern + auth path together.

Checklist

  • Follows Conventional Commits (feat(sandbox):, feat(providers):)
  • Commits are signed off (DCO)
  • Architecture docs updated — no surgical place to add a row that wouldn't pre-empt Sigv4 credential signing #1630's signing scope. The new patterns sit alongside the existing OpenAI/Anthropic ones in default_patterns(); the new YAML profile follows the same shape as claude-code.yaml / nvidia.yaml. Happy to add a paragraph to docs/sandboxes/manage-providers.mdx (or another spot reviewers prefer) in this PR rather than a doc-only follow-up.

Notes for reviewers

Use cases (which PRs you need for which upstream):

Upstream What you need Why
SAP AI Core deployed Bedrock (XSUAA bearer; no SigV4) This PR alone The bridge ignores inbound auth and mints XSUAA outbound; the supervisor's L7 router only needs to recognize Bedrock URL patterns, which this PR adds.
In-cluster translating bridge (LiteLLM in Bedrock-emulation mode, custom Bedrock-compatible proxy that authenticates separately) This PR alone Same shape as SAP — operator's bridge handles upstream auth; the proxy just needs URL-pattern recognition.
Real AWS Bedrock (SigV4 enforced at AWS) This PR plus #1630 This PR adds the URL-pattern recognition; #1630 adds proxy-side SigV4 signing via the credential_signing: sigv4 policy field. The two are complementary; this PR is the prerequisite that makes #1630's signing applicable to Bedrock paths.

In all three cases, provider create --type aws-bedrock requires --no-verify until a Bedrock-aware arm is added to validation_probe() in crates/openshell-router/src/backend.rs. That extension is left for a follow-up PR to keep this one focused on the URL-pattern + provider-registration changes.

Out of scope (intentional):

  • SigV4 signing. Already addressed in Sigv4 credential signing #1630.
  • {region} placeholder substitution in the YAML loader. Operators override per-deployment via BEDROCK_BASE_URL config-key the same way ANTHROPIC_BASE_URL works for the anthropic provider.
  • Body translation between Bedrock InvokeModel and other inference shapes. The router treats matched requests as opaque pass-through.
  • CLI / TUI surface updates. Operators can already create the provider via openshell provider create --type aws-bedrock because the registry recognizes the new id; surfacing it in the TUI's provider-type picker is a follow-up.

Operator context:

The st-gr/openshell-driver-kyma Helm chart currently registers its SAP AI Core ↔ Bedrock translation bridge as --type anthropic with /v1/messages on the inside, because aws-bedrock isn't a recognized provider type. The chart therefore carries a server-side Anthropic→Bedrock body translator and a denylist for Anthropic-API-only fields the SAP gateway rejects. After this PR, the bridge becomes a path-translating + auth-substituting pass-through with no body work — the chart's translator code goes away.

@st-gr st-gr requested review from a team, derekwaynecarr, maxamillion and mrunalp as code owners June 3, 2026 05:30
@copy-pr-bot

copy-pr-bot Bot commented Jun 3, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions

github-actions Bot commented Jun 3, 2026

Copy link
Copy Markdown

All contributors have signed the DCO ✍️ ✅
Posted by the DCO Assistant Lite bot.

@st-gr

st-gr commented Jun 3, 2026

Copy link
Copy Markdown
Author

I have read the DCO document and I hereby sign the DCO.

@st-gr

st-gr commented Jun 3, 2026

Copy link
Copy Markdown
Author

recheck

Comment thread crates/openshell-providers/src/providers/aws_bedrock.rs Outdated
# substitute the `{region}` placeholder. Operators in other regions
# override via the `BEDROCK_BASE_URL` config-key the same way the
# `anthropic` provider accepts `ANTHROPIC_BASE_URL`.
- host: bedrock-runtime.us-east-1.amazonaws.com

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can also use wildcards/glob patterns here unless you feel that is too broad of access

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good flag — this turned out tighter than I expected. crates/openshell-sandbox/src/l7/mod.rs:356-389 (validate_host_wildcard) enforces:

  1. Wildcards may only appear in the first DNS label (line 372: "host wildcard may only appear in the first DNS label") — so bedrock-runtime.*.amazonaws.com is rejected.
  2. TLD/single-label wildcards (*, *.com) are rejected.

The only wildcard form that passes is *.amazonaws.com, which is the "too broad" case you flagged (matches S3, DynamoDB, IAM, every AWS service). Validator and your concern agree.

Two ways to land it:

a) Keep bedrock-runtime.us-east-1.amazonaws.com:443 (current state). Operators in other regions override via BEDROCK_BASE_URL, mirroring ANTHROPIC_BASE_URL for anthropic. Tight, no false declarative promise.

b) File a follow-up to allow bedrock-runtime.{region}.amazonaws.com placeholder substitution at the YAML loader (or relax the first-label rule specifically for YAML-defaulted endpoints), so one entry covers all regions. Out of scope for this PR; happy to open the issue.

My read is (a) for this PR + a follow-up issue for (b). Push back if you'd prefer otherwise.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah (a) makes sense here!

@johntmyers

johntmyers commented Jun 3, 2026

Copy link
Copy Markdown
Collaborator

gator-agent

PR Review Status

Validation: this PR remains project-valid as a concentrated Bedrock-compatible provider/profile and sandbox L7 routing update. The prior legacy ProviderDiscoverySpec blocker is resolved at head 508a4e6c1fb88c7c1550f2cebb50c24301116db6.

Head SHA: 508a4e6c1fb88c7c1550f2cebb50c24301116db6

Review findings:

  • Blocking: aws-bedrock still is not wired into the managed inference.local route registry. openshell_core::inference::profile_for only registers openai, anthropic, and nvidia, so openshell inference set --provider <aws-bedrock-provider> will reject this provider before the new sandbox L7 patterns can be used. Please add the aws-bedrock inference profile/route support or narrow the PR so it does not advertise managed inference.local support.
  • Major: providers/aws-bedrock.yaml declares AWS_SESSION_TOKEN, but discovery.credentials omits aws_session_token. Providers v2 discovery only scans listed credentials, so --from-existing will silently drop temporary AWS credentials. Please include aws_session_token in discovery.credentials.
  • Major: direct user-facing docs are still missing. If this PR exposes aws-bedrock as a provider/profile and adds Bedrock request patterns, please update the existing provider and inference routing docs. No new docs/index.yml entry appears necessary.
  • Test gap: add coverage that an aws-bedrock provider can be created from the built-in profile and either successfully used by upsert_cluster_inference_route or intentionally rejected with a clear documented error.

Docs: missing for direct provider and inference-routing behavior.

Checks: DCO is passing. Branch Checks and Helm Lint are still waiting for /ok to test; I am not posting /ok to test while blocking review feedback is outstanding.

Next state: gator:in-review

@johntmyers johntmyers added the gator:in-review Gator is reviewing or awaiting PR review feedback label Jun 3, 2026
st-gr added a commit to st-gr/OpenShell that referenced this pull request Jun 5, 2026
Addresses johntmyers's review on NVIDIA#1704: net-new
providers should land via the v2 YAML profile only and should NOT
require changes to the legacy `ProviderDiscoverySpec` registry.

- Delete `crates/openshell-providers/src/providers/aws_bedrock.rs`
  (the legacy SPEC + `test_discovers_env_credential!` invocation).
- Drop `pub mod aws_bedrock;` from `crates/openshell-providers/src/providers/mod.rs`.
- Drop `registry.register(providers::aws_bedrock::SPEC)` from
  `crates/openshell-providers/src/lib.rs`.

Kept:

- `providers/aws-bedrock.yaml` and the `include_str!` in
  `BUILT_IN_PROFILE_YAMLS` (`profiles.rs`) — the v2 path.
  `discover_from_profile()` (`crates/openshell-providers/src/discovery.rs`)
  picks up AWS_* env vars via `discovery.credentials` in the YAML.
- L7 router patterns in `crates/openshell-sandbox/src/l7/inference.rs`
  — orthogonal to the provider registry.

The discovery test in the deleted file goes with it; v2 doesn't have
an established per-provider env-var-pickup unit test pattern, and
other YAML-only registrations (none today, but this is the new
direction) won't carry one either.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
@johntmyers

Copy link
Copy Markdown
Collaborator

gator-agent

Re-check After Reviewer Update

I re-evaluated latest head 508a4e6c1fb88c7c1550f2cebb50c24301116db6 after st-gr's June 5 responses on the legacy provider update and endpoint wildcard, and johntmyers's June 5 maintainer response accepting the fixed bedrock-runtime.us-east-1.amazonaws.com endpoint.

Disposition: partially resolved. The legacy ProviderDiscoverySpec update is removed, and the endpoint wildcard discussion is resolved. Blocking review feedback remains.

Remaining items:

  • Blocking: providers/aws-bedrock.yaml marks the profile inference_capable: true, but openshell_core::inference::profile_for still only registers openai, anthropic, and nvidia; openshell inference set --provider <aws-bedrock-provider> will still reject this provider before the new sandbox L7 patterns can be used. Please add the aws-bedrock inference profile/route support with the intended auth semantics, or narrow the PR so it does not advertise managed inference.local support.
  • Blocking/static test regression: crates/openshell-server/src/grpc/provider.rs still asserts the built-in profile list is exactly ['claude-code', 'github', 'nvidia']. Adding providers/aws-bedrock.yaml makes that assertion stale; please update the expected built-in profile list.
  • Major: providers/aws-bedrock.yaml declares aws_session_token, but discovery.credentials omits it. Providers v2 discovery only scans listed credentials, so --from-existing will silently drop temporary AWS credentials. Please include aws_session_token in discovery.credentials.
  • Major: direct user-facing docs are still missing for the new provider/profile and inference-routing behavior. No docs/index.yml navigation change appears necessary.
  • Test gap: please add coverage that an aws-bedrock provider can be created from the built-in profile and either successfully used by upsert_cluster_inference_route or intentionally rejected with a clear documented error.

Checks: DCO is passing. Branch Checks and Helm Lint are still waiting for /ok to test; I am not posting /ok to test while blocking review feedback is outstanding.

Next state: gator:in-review

st-gr added a commit to st-gr/OpenShell that referenced this pull request Jun 7, 2026
…le assertion

Two fixes from johntmyers's gator-agent re-check on NVIDIA#1704:

1. `providers/aws-bedrock.yaml`: add `aws_session_token` to
   `discovery.credentials`. The credential is declared in the profile
   but was missing from the discovery scan list, so Providers v2
   `--from-existing` would silently drop temporary AWS credentials
   (STS / IRSA scenarios).

2. `crates/openshell-server/src/grpc/provider.rs`: update the static
   `list_provider_profiles_returns_built_in_profile_categories`
   assertion to include `aws-bedrock` at alphabetical position 0.
   Adding `providers/aws-bedrock.yaml` to BUILT_IN_PROFILE_YAMLS made
   the prior `["claude-code", "github", "nvidia"]` expectation stale.

Remaining blockers from the same review (deferred to follow-up
commits): `inference::profile_for` registration for aws-bedrock,
user-facing provider + inference-routing docs, and an
`upsert_cluster_inference_route` integration test.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
st-gr added a commit to st-gr/OpenShell that referenced this pull request Jun 7, 2026
Addresses johntmyers's blocking review feedback on PR NVIDIA#1704:
"aws-bedrock still is not wired into the managed inference.local route
registry. profile_for only registers openai, anthropic, and nvidia, so
inference set --provider <aws-bedrock-provider> will reject this
provider before the new sandbox L7 patterns can be used."

Approach: register aws-bedrock as a *bridge-fronted* upstream — the
router does not inject any auth header on outbound requests; the
configured BEDROCK_BASE_URL is expected to point at a translating
bridge / Bedrock-compatible proxy that handles auth in its own pod.
This is the shape the L7 patterns commit (8b30211) and the YAML
profile (6b51e1a) were designed for. SigV4 signing for direct AWS
Bedrock is a separate follow-up; see PR thread.

Changes:

- core::inference::AuthHeader: add `None` variant for upstreams that
  authenticate themselves.
- core::inference: add AWS_BEDROCK_PROFILE static + register in
  profile_for. Default base URL is bedrock-runtime.us-east-1, override
  via BEDROCK_BASE_URL config-key (mirrors ANTHROPIC_BASE_URL pattern).
  Empty credential_key_names + auth: None means no router-side
  credential lookup at route time.
- router::backend: handle AuthHeader::None as a no-op (skip auth
  injection).
- server::inference::resolve_provider_route: gate find_provider_api_key
  on auth != None. aws-bedrock providers with empty credentials now
  resolve cleanly. Updated the unsupported-type error message to
  include aws-bedrock in the supported list.
- server::inference tests: add positive
  upsert_cluster_route_succeeds_for_aws_bedrock_without_api_key test
  covering the new code path end-to-end (provider with empty creds +
  BEDROCK_BASE_URL config → upsert succeeds → resolved route has
  empty api_key + provider_type aws-bedrock + bridge URL).
- core::inference tests: profile_for_known_types covers aws-bedrock,
  case-insensitive lookup, plus three new aws-bedrock-specific tests
  (auth: None, no credential keys, bedrock-specific protocols).
- docs/sandboxes/inference-routing.mdx: header forwarding row
  mentions aws-bedrock has no passthrough headers; new tabs in
  Supported API Patterns (InvokeModel + InvokeModelWithResponseStream)
  and Create a Provider (with the bridge-fronted shape note + SigV4
  deferral).
- docs/sandboxes/manage-providers.mdx: new row in Supported Provider
  Types table; new row in Supported Inference Providers table.

Verification (in dev container):
- cargo check -p openshell-core -p openshell-router -p openshell-server: clean
- cargo test -p openshell-core --lib inference: 14/14 pass (incl. 3 new)
- cargo test -p openshell-server --lib inference::tests::upsert: 6/6 pass
  (incl. new aws-bedrock test)
- cargo fmt --check: clean
- cargo clippy --all-targets -D warnings: clean

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
@st-gr

st-gr commented Jun 7, 2026

Copy link
Copy Markdown
Author

@johntmyers — pushed 4ab587f1 addressing the remaining review items.

This push:

  • Blocking (profile_for registration): registered aws-bedrock with a new AuthHeader::None variant — the router does not inject any auth header on outbound requests. The configured BEDROCK_BASE_URL is expected to point at a translating bridge or Bedrock-compatible proxy that handles auth in its own pod. This matches the pass-through architecture noted in the original commit message (8b30211a: "if the operator's upstream is real AWS Bedrock it speaks Bedrock natively, if it's a translating bridge the bridge does any conversion server-side"). openshell inference set --provider <aws-bedrock-provider> now resolves cleanly.
  • Major (aws_session_token in discovery): added in 3d0ad1c6.
  • Major (docs): updated docs/sandboxes/inference-routing.mdx (header forwarding row, new API-pattern tab, new provider-creation tab with the bridge-fronted note) and docs/sandboxes/manage-providers.mdx (Supported Provider Types and Supported Inference Providers tables).
  • Test gap: added upsert_cluster_route_succeeds_for_aws_bedrock_without_api_key in crates/openshell-server/src/inference.rs. Creates an aws-bedrock provider with empty credentials + a BEDROCK_BASE_URL config-key, upserts the cluster route, and asserts the resolved route carries provider_type: "aws-bedrock", the bridge URL, and an empty api_key.
  • Static profile assertion: updated in 3d0ad1c6 to include aws-bedrock.

Deferred to follow-up PRs (will file issues for tracking):

  1. SigV4 router-side signer. Adds an AuthHeader::SigV4 { ... } variant + outbound request signing using the aws-sigv4 crate so aws-bedrock providers can authenticate against real AWS Bedrock directly (instead of via a bridge). The current PR's auth: None shape is one of two intended shapes; SigV4 is the other and is genuinely separable.
  2. {region} placeholder substitution in YAML profile loader. So operators can use any AWS region without having to override the entire base URL via BEDROCK_BASE_URL. Today the YAML's host is a literal bedrock-runtime.us-east-1.amazonaws.com.

Verification (run locally in the rust-1.95 dev container): cargo check -p openshell-core -p openshell-router -p openshell-server, cargo test -p openshell-core --lib inference (14/14), cargo test -p openshell-server --lib inference::tests::upsert (6/6), cargo fmt --check, cargo clippy --all-targets -D warnings — all clean. Branch Checks / Helm Lint stand by for /ok to test whenever ready.

@johntmyers

Copy link
Copy Markdown
Collaborator

gator-agent

Re-check After Author Update

I re-evaluated latest head 4ab587f1bdab2c2565462fe5ed9cc6bb3a5d0c12 after @st-gr's June 7 comment saying the remaining review items were addressed.

Disposition: partially resolved. The previous profile_for registration, built-in profile list, aws_session_token discovery entry, and Fern docs update requests are addressed. Blocking review feedback remains on the public setup path and Bedrock route behavior.

Remaining items:

  • Blocking: openshell inference set verifies routes by default, but the router validation probe does not support aws_bedrock_invoke / aws_bedrock_invoke_stream, so the documented Bedrock inference setup will fail unless users know to pass --no-verify. Please add a Bedrock validation strategy, or explicitly require/document --no-verify and add coverage for that public path.
  • Blocking/security: Bedrock carries the model id in /model/{modelId}/invoke, but the router currently forwards the caller's original path and only rewrites JSON body model. That lets sandbox code choose a different upstream model than the operator-configured route model, and may also mutate native Bedrock request bodies incorrectly. Please rewrite or reject mismatched Bedrock model paths and skip body model injection for Bedrock protocols, with buffered and streaming router coverage.
  • Blocking/docs or behavior mismatch: the docs show openshell provider create --type aws-bedrock --config BEDROCK_BASE_URL=... without credentials, but the CLI requires --from-existing or --credential and the server rejects empty credentials for non-refresh profiles. Please either make bridge-fronted aws-bedrock work as a config-only provider and test provider_create, or document the actual required command.
  • Major: until SigV4 support exists, the core profile should not silently fall back to the real AWS Bedrock endpoint with auth: None; require BEDROCK_BASE_URL for bridge-fronted mode or otherwise prevent saving a route that forwards prompt bodies to AWS without usable auth.
  • Major: the provider YAML marks AWS credentials required and discoverable even though bridge-fronted routing intentionally does not use them. Please avoid collecting required AWS credentials for the bridge-only shape, or explain and test why they must be stored before SigV4 lands.
  • Docs polish: docs/sandboxes/inference-routing.mdx still says provider support is NVIDIA, OpenAI-compatible, and Anthropic; please include AWS Bedrock in that summary row if this PR keeps the provider.

Checks: DCO is passing. Branch Checks and Helm Lint are still waiting for /ok to test; I am not posting /ok to test while blocking review feedback is outstanding.

Next state: gator:in-review

@johntmyers johntmyers added gator:blocked Gator is blocked by process or repository gates and removed gator:in-review Gator is reviewing or awaiting PR review feedback labels Jun 8, 2026
@johntmyers

Copy link
Copy Markdown
Collaborator

gator-agent

Blocked

Gator is blocked by merge conflicts on the PR branch. GitHub currently reports mergeable=CONFLICTING and mergeStateStatus=DIRTY for head 4ab587f1bdab2c2565462fe5ed9cc6bb3a5d0c12.

Next action: @st-gr, please rebase or merge main into feat/aws-bedrock-provider and resolve the conflicts. After the branch is mergeable again, gator will re-check the outstanding review items before any /ok to test action.

st-gr added a commit to st-gr/OpenShell that referenced this pull request Jun 8, 2026
Resolves merge conflicts after main fast-forwarded 554 commits past
the branch base. Conflicting files and resolutions:

- crates/openshell-core/src/inference.rs: kept aws-bedrock additions
  (AWS_BEDROCK_PROTOCOLS const, AWS_BEDROCK_PROFILE static) alongside
  the new VERTEX_AI_PROTOCOLS const and normalize_inference_provider_type
  function. Added "aws-bedrock" to that normalize function so profile_for
  resolves it through the same canonicalization path as the other
  inference profiles.
- crates/openshell-server/src/grpc/provider.rs: merged the static
  built-in profile assertion to include both the upstream additions
  (codex, copilot, cursor, google-vertex-ai, pypi) and aws-bedrock,
  alphabetically.
- crates/openshell-server/src/inference.rs: kept the auth: None gating
  for aws-bedrock api-key lookup, applied around the new
  CredentialLookup-based call signature; preserved the vertex-ai
  dispatch added upstream. Updated the unsupported-type error message
  to list both google-vertex-ai and aws-bedrock.
- docs/sandboxes/inference-routing.mdx: combined the upstream Vertex
  Claude rawPredict header note with the aws-bedrock no-passthrough
  note in the header forwarding row.

Drive-by fix for one finding from gator-agent's NVIDIA#1704 re-check on
4ab587f: AWS_BEDROCK_PROFILE.default_base_url is now an empty string
rather than the real AWS Bedrock URL. Without BEDROCK_BASE_URL config,
resolve_provider_route's existing empty-base_url check now rejects
the provider rather than silently forwarding prompts to AWS with
auth: None. Once SigV4 lands, the default can revert.

Verification (in dev container):
- cargo check -p openshell-core -p openshell-router -p openshell-server: clean

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
st-gr added a commit to st-gr/OpenShell that referenced this pull request Jun 8, 2026
Addresses four findings from gator-agent's NVIDIA#1704 re-check on 4ab587f:

- **Item 5** (YAML collects unused AWS creds): mark all four AWS
  credentials `required: false` and clear `discovery.credentials`.
  Bridge-fronted routing intentionally does not consume AWS
  credentials, so `--from-existing` no longer scans for them. The
  credentials remain in the schema (not deleted) so the SigV4
  follow-up can flip them back without a schema migration. Added a
  multi-line description that names the bridge-fronted shape and the
  SigV4 deferral so readers don't have to cross-reference the PR
  thread.

- **Item 3** (docs show command that the CLI rejects): rewrite the
  Create-a-Provider example for AWS Bedrock to use the actual
  required shape — placeholder `--credential AWS_ACCESS_KEY_ID=
  unused-bridge-fronted-shape` plus the `--config BEDROCK_BASE_URL`.
  The placeholder satisfies the gRPC handler's
  `provider.credentials.is_empty()` rejection without expanding
  server-side validation; the router ignores it on the outbound path
  because `auth: AuthHeader::None` skips header injection. Operators
  see a clearly-labeled placeholder in `provider get` output.

- **Item 1** (validator probe): document `--no-verify` as required
  for `openshell inference set --provider <aws-bedrock>` since the
  default validation probe doesn't recognize the
  `aws_bedrock_invoke` / `aws_bedrock_invoke_stream` protocols. Doc
  now shows the full `provider create` + `inference set --no-verify`
  flow with rationale for both decisions inline.

- **Item 6** (docs polish): `inference-routing.mdx` summary row now
  lists AWS Bedrock alongside NVIDIA, Anthropic, Vertex AI, and
  OpenAI-compatible providers, with the bridge-fronted caveat
  inline.

Test additions in `crates/openshell-server/src/inference.rs`:

- Renamed the existing aws-bedrock test from
  `..._without_api_key` to `..._with_bridge_url` and updated it to
  use a placeholder credential (mirroring the doc-recommended
  pattern operators will copy-paste). The `auth: None` path still
  produces an empty `api_key` on the resolved route — the test now
  documents that the credential is *stored* but not *used*.
- Added `upsert_cluster_route_rejects_aws_bedrock_without_bedrock_base_url`:
  the negative half of johntmyers' "successfully used by
  upsert_cluster_inference_route or intentionally rejected with a
  clear documented error" ask. With
  `default_base_url: ""` and no `BEDROCK_BASE_URL` config, route
  resolution returns `InvalidArgument` naming the missing base_url
  rather than silently forwarding prompts to AWS Bedrock with no
  usable auth.

Verification (in dev container):
- cargo test -p openshell-core --lib inference: 18/18 (incl. 3 new)
- cargo test -p openshell-server --lib inference::tests::upsert: 8/8
  (incl. 2 new aws-bedrock cases — positive + negative)
- cargo fmt --check: clean
- cargo clippy --all-targets -D warnings: clean

Item 2 (router-side enforcement of operator-configured Bedrock model
path, replacing the current verbatim path forwarding + body-only
model rewrite) is the remaining blocker and is genuinely separable —
it touches the L7 router with streaming-aware test coverage.
Deferring to its own commit so the security-critical change gets the
review attention it deserves.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
@johntmyers johntmyers added gator:in-review Gator is reviewing or awaiting PR review feedback and removed gator:blocked Gator is blocked by process or repository gates labels Jun 9, 2026
@johntmyers

Copy link
Copy Markdown
Collaborator

gator-agent

Re-check After Author Update

I re-evaluated latest head 6373b4d2649f7fb36cbe75c34c96319f0120e439 after @st-gr pushed updates on June 8, 2026 to resolve the prior merge-conflict blocker and address the remaining review items.

Disposition: partially resolved. The merge-conflict blocker is resolved (mergeable=true), and the bridge-only Bedrock profile/docs/no-verify path is clearer. Blocking review feedback remains.

Remaining items:

  • Blocking/security: Bedrock still does not enforce the configured route model. aws-bedrock resolves with model_in_path: false and no path override, so sandbox code can call /model/other-model/invoke and the bridge receives that caller-selected model in the URL. The generic body rewrite also injects "model": route.model, which is not Bedrock's selector and can mutate native Bedrock bodies. Please either build/rewrite Bedrock upstream paths from route.model or reject mismatched caller model paths, and skip generic body model insertion for aws_bedrock_* protocols.
  • Blocking/scope: the PR currently carries unrelated fork infrastructure and external compute driver changes (.github/workflows/release-gateway.yml, .github/workflows/rust-lint.yml, .github/workflows/smoke-self-hosted-kyma.yml, Dockerfile.gateway, README-FORK.md, and external-driver hunks). Please remove those from this Bedrock provider PR or split them into maintainer-scoped PRs.
  • Warning: Bedrock InvokeModel should be buffered while InvokeModelWithResponseStream is streaming. Please add framing/coverage so /model/{id}/invoke cannot be corrupted by the streaming proxy's truncation/error-frame behavior.
  • Warning/docs: the bridge-only profile still declares AWS secret fields while discovery is empty and the router does not consume them. Please avoid encouraging real AWS secrets for the current bridge-only shape, or label them clearly as future SigV4 fields in the docs table.

Checks: DCO is passing. Branch Checks and Helm Lint are still waiting for /ok to test; I am not posting /ok to test while blocking review feedback is outstanding.

Next state: gator:in-review

st-gr added a commit to st-gr/OpenShell that referenced this pull request Jun 9, 2026
Closes the security-blocking item from gator-agent's NVIDIA#1704 re-check
on 4ab587f: "Bedrock carries the model id in /model/{modelId}/invoke,
but the router currently forwards the caller's original path and only
rewrites JSON body model. That lets sandbox code choose a different
upstream model than the operator-configured route model, and may also
mutate native Bedrock request bodies incorrectly."

Two changes in `prepare_backend_request`:

1. **Path rewrite for Bedrock routes.** Before computing the upstream
   URL, parse the inbound path's `/model/<id>/invoke[-with-response-stream]`
   shape and substitute the operator-configured `route.model` for the
   caller-supplied model segment. Sandbox code that hardcodes a
   different model still works (we don't reject on mismatch), but the
   operator's configured model is what reaches the upstream / bridge.
   If the inbound path is somehow not a recognized Bedrock shape on a
   Bedrock route (the L7 pattern detector upstream of the router
   should never produce this combination), reject with
   RouterError::Internal naming the offending path rather than
   forwarding verbatim.

2. **Skip body-model injection for Bedrock routes.** The existing body
   rewriter unconditionally inserts `route.model` into the JSON body
   for non-Vertex routes. AWS Bedrock InvokeModel encodes the model
   in the URL path; the body is the raw provider-specific payload
   (Anthropic Messages for Claude, Mistral payload for Mistral, etc.)
   and must not be mutated. The branch ordering is now:
   needs_vertex_anthropic_version → strip body model + inject
   anthropic_version; route_is_bedrock → leave body alone; else →
   inject route.model (existing default).

New helpers, all in `crates/openshell-router/src/backend.rs`:

- `route_is_bedrock(route)` — true when route.protocols contains
  aws_bedrock_invoke or aws_bedrock_invoke_stream.
- `parse_bedrock_invocation_path(path)` — returns
  Some((model_id, "/invoke" | "/invoke-with-response-stream")) for
  paths matching the recognized Bedrock shapes. Strips query strings.
  Rejects empty model ids and multi-segment ids (defense-in-depth
  matching the L7 pattern detector's existing guards).
- `rewrite_bedrock_path(route, path)` — returns the path with the
  caller's model segment replaced by route.model.

Test coverage in the same file (9 new tests):

- parse_bedrock_invocation_path: positive cases for both invoke
  variants, query-string stripping; negative cases for empty model id,
  multi-segment id, unknown action, wrong prefix, missing slash.
- route_is_bedrock: matches both protocol variants singly and
  combined; rejects openai_chat_completions.
- rewrite_bedrock_path: substitutes operator model on both invoke
  variants; returns None for non-Bedrock paths.
- bedrock_route_rewrites_model_in_path_and_preserves_body
  (wiremock end-to-end): caller sends /model/some-other-model/invoke
  with a body containing model: "caller-supplied-model-name". Mock
  asserts the upstream receives /model/<operator-model>/invoke and the
  body's model field is the caller's value (NOT route.model) — proves
  both the path rewrite and the body preservation.
- bedrock_route_streaming_rewrites_model_in_path: same contract for
  invoke-with-response-stream.
- bedrock_route_rejects_non_bedrock_path: defense-in-depth coverage of
  the Internal-error path when a Bedrock route receives a path that
  doesn't match Bedrock shape.

Verification (in dev container):
- cargo test -p openshell-router --lib: 53/53 (incl. 9 new)
- cargo fmt --check: clean
- cargo clippy -p openshell-core -p openshell-router -p openshell-server
  --all-targets -- -D warnings: clean

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
Adds two patterns to `default_patterns()` so the supervisor's L7
inference router recognizes the Bedrock InvokeModel URL shape and
forwards matched requests to the registered upstream:

- `POST /model/{modelId}/invoke`                       → aws_bedrock_invoke
- `POST /model/{modelId}/invoke-with-response-stream`  → aws_bedrock_invoke_stream

The `{modelId}` segment is wildcarded by extending `detect_inference_pattern`
to handle one middle `/*/` segment in addition to the existing trailing
`/*`. The wildcard is constrained to a single non-empty path segment to
avoid path-traversal liabilities — `/model//invoke` and `/model/a/b/invoke`
both no-match.

Without this, sandboxes running Claude Code in its native Bedrock mode
(`CLAUDE_CODE_USE_BEDROCK=1`, `ANTHROPIC_BEDROCK_BASE_URL`, AWS-style
auth) hit the supervisor with `403 connection not allowed by policy`
because their URL doesn't match `/v1/*` shapes. The fix unblocks
operators wanting to register direct AWS Bedrock, an in-cluster
Bedrock-compatible bridge, or a Bedrock-emulating LiteLLM as
`--type aws-bedrock` providers.

Tests cover: positive matches for invoke + invoke-with-response-stream,
query-string handling, GET rejection, empty-segment rejection,
multi-segment rejection, and unknown-action rejection.

Companion changes (provider discovery spec + YAML profile) follow in
the next commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
st-gr and others added 6 commits June 8, 2026 21:59
Adds `aws-bedrock` to the built-in provider catalog so operators can
run `openshell provider create --type aws-bedrock --credential ...`
and have the gateway treat it as a first-class inference provider
alongside `anthropic`, `openai`, etc.

- `providers/aws-bedrock.yaml`: YAML profile declaring four credentials
  (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN, AWS_REGION).
  Default endpoint is `bedrock-runtime.us-east-1.amazonaws.com:443`;
  operators in other regions or running against a Bedrock-compatible
  proxy override via the operator-supplied `BEDROCK_BASE_URL` config-key
  (mirrors `ANTHROPIC_BASE_URL` for the `anthropic` provider).

- `crates/openshell-providers/src/providers/aws_bedrock.rs`: the
  `ProviderDiscoverySpec` so `openshell provider create --auto-providers`
  picks up AWS_* env vars from local credentials.

- `crates/openshell-providers/src/providers/mod.rs`: register the module.

- `crates/openshell-providers/src/lib.rs`: register the SPEC in the
  default registry alongside the other providers.

- `crates/openshell-providers/src/profiles.rs`: include the new YAML in
  `BUILT_IN_PROFILE_YAMLS`.

What this PR explicitly does NOT add (intentionally separated for
review-size reasons; will follow up):

- A SigV4 signer in `openshell-router`. The current change simply
  declares the protocol; a follow-up PR adds outbound SigV4 signing
  using the `aws-sigv4` crate and a new `auth_style: sigv4` validator
  branch in profiles.rs. Operators who don't need SigV4 (e.g. an
  in-cluster bridge that ignores it and authenticates separately to
  the upstream) can use this PR today.

- Body translation between Bedrock InvokeModel shape and other
  inference shapes. The router treats Bedrock requests as opaque
  pass-through; if the operator's upstream is real AWS Bedrock it
  speaks Bedrock natively, if it's a translating bridge the bridge
  does any conversion server-side.

- `BEDROCK_BASE_URL` placeholder substitution in the YAML loader.
  Today the YAML's `host` is a literal default; operators override
  with the config-key the same way `ANTHROPIC_BASE_URL` works.

Tested: `cargo test -p openshell-providers` (35 tests green) and
`cargo test -p openshell-sandbox --lib l7::inference` (40 tests green
including the seven new aws_bedrock cases from the previous commit).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
Addresses johntmyers's review on NVIDIA#1704: net-new
providers should land via the v2 YAML profile only and should NOT
require changes to the legacy `ProviderDiscoverySpec` registry.

- Delete `crates/openshell-providers/src/providers/aws_bedrock.rs`
  (the legacy SPEC + `test_discovers_env_credential!` invocation).
- Drop `pub mod aws_bedrock;` from `crates/openshell-providers/src/providers/mod.rs`.
- Drop `registry.register(providers::aws_bedrock::SPEC)` from
  `crates/openshell-providers/src/lib.rs`.

Kept:

- `providers/aws-bedrock.yaml` and the `include_str!` in
  `BUILT_IN_PROFILE_YAMLS` (`profiles.rs`) — the v2 path.
  `discover_from_profile()` (`crates/openshell-providers/src/discovery.rs`)
  picks up AWS_* env vars via `discovery.credentials` in the YAML.
- L7 router patterns in `crates/openshell-sandbox/src/l7/inference.rs`
  — orthogonal to the provider registry.

The discovery test in the deleted file goes with it; v2 doesn't have
an established per-provider env-var-pickup unit test pattern, and
other YAML-only registrations (none today, but this is the new
direction) won't carry one either.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
…le assertion

Two fixes from johntmyers's gator-agent re-check on NVIDIA#1704:

1. `providers/aws-bedrock.yaml`: add `aws_session_token` to
   `discovery.credentials`. The credential is declared in the profile
   but was missing from the discovery scan list, so Providers v2
   `--from-existing` would silently drop temporary AWS credentials
   (STS / IRSA scenarios).

2. `crates/openshell-server/src/grpc/provider.rs`: update the static
   `list_provider_profiles_returns_built_in_profile_categories`
   assertion to include `aws-bedrock` at alphabetical position 0.
   Adding `providers/aws-bedrock.yaml` to BUILT_IN_PROFILE_YAMLS made
   the prior `["claude-code", "github", "nvidia"]` expectation stale.

Remaining blockers from the same review (deferred to follow-up
commits): `inference::profile_for` registration for aws-bedrock,
user-facing provider + inference-routing docs, and an
`upsert_cluster_inference_route` integration test.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
Addresses johntmyers's blocking review feedback on PR NVIDIA#1704:
"aws-bedrock still is not wired into the managed inference.local route
registry. profile_for only registers openai, anthropic, and nvidia, so
inference set --provider <aws-bedrock-provider> will reject this
provider before the new sandbox L7 patterns can be used."

Approach: register aws-bedrock as a *bridge-fronted* upstream — the
router does not inject any auth header on outbound requests; the
configured BEDROCK_BASE_URL is expected to point at a translating
bridge / Bedrock-compatible proxy that handles auth in its own pod.
This is the shape the L7 patterns commit (8b30211) and the YAML
profile (6b51e1a) were designed for. SigV4 signing for direct AWS
Bedrock is a separate follow-up; see PR thread.

Changes:

- core::inference::AuthHeader: add `None` variant for upstreams that
  authenticate themselves.
- core::inference: add AWS_BEDROCK_PROFILE static + register in
  profile_for. Default base URL is bedrock-runtime.us-east-1, override
  via BEDROCK_BASE_URL config-key (mirrors ANTHROPIC_BASE_URL pattern).
  Empty credential_key_names + auth: None means no router-side
  credential lookup at route time.
- router::backend: handle AuthHeader::None as a no-op (skip auth
  injection).
- server::inference::resolve_provider_route: gate find_provider_api_key
  on auth != None. aws-bedrock providers with empty credentials now
  resolve cleanly. Updated the unsupported-type error message to
  include aws-bedrock in the supported list.
- server::inference tests: add positive
  upsert_cluster_route_succeeds_for_aws_bedrock_without_api_key test
  covering the new code path end-to-end (provider with empty creds +
  BEDROCK_BASE_URL config → upsert succeeds → resolved route has
  empty api_key + provider_type aws-bedrock + bridge URL).
- core::inference tests: profile_for_known_types covers aws-bedrock,
  case-insensitive lookup, plus three new aws-bedrock-specific tests
  (auth: None, no credential keys, bedrock-specific protocols).
- docs/sandboxes/inference-routing.mdx: header forwarding row
  mentions aws-bedrock has no passthrough headers; new tabs in
  Supported API Patterns (InvokeModel + InvokeModelWithResponseStream)
  and Create a Provider (with the bridge-fronted shape note + SigV4
  deferral).
- docs/sandboxes/manage-providers.mdx: new row in Supported Provider
  Types table; new row in Supported Inference Providers table.

Verification (in dev container):
- cargo check -p openshell-core -p openshell-router -p openshell-server: clean
- cargo test -p openshell-core --lib inference: 14/14 pass (incl. 3 new)
- cargo test -p openshell-server --lib inference::tests::upsert: 6/6 pass
  (incl. new aws-bedrock test)
- cargo fmt --check: clean
- cargo clippy --all-targets -D warnings: clean

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
Addresses four findings from gator-agent's NVIDIA#1704 re-check on 4ab587f:

- **Item 5** (YAML collects unused AWS creds): mark all four AWS
  credentials `required: false` and clear `discovery.credentials`.
  Bridge-fronted routing intentionally does not consume AWS
  credentials, so `--from-existing` no longer scans for them. The
  credentials remain in the schema (not deleted) so the SigV4
  follow-up can flip them back without a schema migration. Added a
  multi-line description that names the bridge-fronted shape and the
  SigV4 deferral so readers don't have to cross-reference the PR
  thread.

- **Item 3** (docs show command that the CLI rejects): rewrite the
  Create-a-Provider example for AWS Bedrock to use the actual
  required shape — placeholder `--credential AWS_ACCESS_KEY_ID=
  unused-bridge-fronted-shape` plus the `--config BEDROCK_BASE_URL`.
  The placeholder satisfies the gRPC handler's
  `provider.credentials.is_empty()` rejection without expanding
  server-side validation; the router ignores it on the outbound path
  because `auth: AuthHeader::None` skips header injection. Operators
  see a clearly-labeled placeholder in `provider get` output.

- **Item 1** (validator probe): document `--no-verify` as required
  for `openshell inference set --provider <aws-bedrock>` since the
  default validation probe doesn't recognize the
  `aws_bedrock_invoke` / `aws_bedrock_invoke_stream` protocols. Doc
  now shows the full `provider create` + `inference set --no-verify`
  flow with rationale for both decisions inline.

- **Item 6** (docs polish): `inference-routing.mdx` summary row now
  lists AWS Bedrock alongside NVIDIA, Anthropic, Vertex AI, and
  OpenAI-compatible providers, with the bridge-fronted caveat
  inline.

Test additions in `crates/openshell-server/src/inference.rs`:

- Renamed the existing aws-bedrock test from
  `..._without_api_key` to `..._with_bridge_url` and updated it to
  use a placeholder credential (mirroring the doc-recommended
  pattern operators will copy-paste). The `auth: None` path still
  produces an empty `api_key` on the resolved route — the test now
  documents that the credential is *stored* but not *used*.
- Added `upsert_cluster_route_rejects_aws_bedrock_without_bedrock_base_url`:
  the negative half of johntmyers' "successfully used by
  upsert_cluster_inference_route or intentionally rejected with a
  clear documented error" ask. With
  `default_base_url: ""` and no `BEDROCK_BASE_URL` config, route
  resolution returns `InvalidArgument` naming the missing base_url
  rather than silently forwarding prompts to AWS Bedrock with no
  usable auth.

Verification (in dev container):
- cargo test -p openshell-core --lib inference: 18/18 (incl. 3 new)
- cargo test -p openshell-server --lib inference::tests::upsert: 8/8
  (incl. 2 new aws-bedrock cases — positive + negative)
- cargo fmt --check: clean
- cargo clippy --all-targets -D warnings: clean

Item 2 (router-side enforcement of operator-configured Bedrock model
path, replacing the current verbatim path forwarding + body-only
model rewrite) is the remaining blocker and is genuinely separable —
it touches the L7 router with streaming-aware test coverage.
Deferring to its own commit so the security-critical change gets the
review attention it deserves.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
Closes the security-blocking item from gator-agent's NVIDIA#1704 re-check
on 4ab587f: "Bedrock carries the model id in /model/{modelId}/invoke,
but the router currently forwards the caller's original path and only
rewrites JSON body model. That lets sandbox code choose a different
upstream model than the operator-configured route model, and may also
mutate native Bedrock request bodies incorrectly."

Two changes in `prepare_backend_request`:

1. **Path rewrite for Bedrock routes.** Before computing the upstream
   URL, parse the inbound path's `/model/<id>/invoke[-with-response-stream]`
   shape and substitute the operator-configured `route.model` for the
   caller-supplied model segment. Sandbox code that hardcodes a
   different model still works (we don't reject on mismatch), but the
   operator's configured model is what reaches the upstream / bridge.
   If the inbound path is somehow not a recognized Bedrock shape on a
   Bedrock route (the L7 pattern detector upstream of the router
   should never produce this combination), reject with
   RouterError::Internal naming the offending path rather than
   forwarding verbatim.

2. **Skip body-model injection for Bedrock routes.** The existing body
   rewriter unconditionally inserts `route.model` into the JSON body
   for non-Vertex routes. AWS Bedrock InvokeModel encodes the model
   in the URL path; the body is the raw provider-specific payload
   (Anthropic Messages for Claude, Mistral payload for Mistral, etc.)
   and must not be mutated. The branch ordering is now:
   needs_vertex_anthropic_version → strip body model + inject
   anthropic_version; route_is_bedrock → leave body alone; else →
   inject route.model (existing default).

New helpers, all in `crates/openshell-router/src/backend.rs`:

- `route_is_bedrock(route)` — true when route.protocols contains
  aws_bedrock_invoke or aws_bedrock_invoke_stream.
- `parse_bedrock_invocation_path(path)` — returns
  Some((model_id, "/invoke" | "/invoke-with-response-stream")) for
  paths matching the recognized Bedrock shapes. Strips query strings.
  Rejects empty model ids and multi-segment ids (defense-in-depth
  matching the L7 pattern detector's existing guards).
- `rewrite_bedrock_path(route, path)` — returns the path with the
  caller's model segment replaced by route.model.

Test coverage in the same file (9 new tests):

- parse_bedrock_invocation_path: positive cases for both invoke
  variants, query-string stripping; negative cases for empty model id,
  multi-segment id, unknown action, wrong prefix, missing slash.
- route_is_bedrock: matches both protocol variants singly and
  combined; rejects openai_chat_completions.
- rewrite_bedrock_path: substitutes operator model on both invoke
  variants; returns None for non-Bedrock paths.
- bedrock_route_rewrites_model_in_path_and_preserves_body
  (wiremock end-to-end): caller sends /model/some-other-model/invoke
  with a body containing model: "caller-supplied-model-name". Mock
  asserts the upstream receives /model/<operator-model>/invoke and the
  body's model field is the caller's value (NOT route.model) — proves
  both the path rewrite and the body preservation.
- bedrock_route_streaming_rewrites_model_in_path: same contract for
  invoke-with-response-stream.
- bedrock_route_rejects_non_bedrock_path: defense-in-depth coverage of
  the Internal-error path when a Bedrock route receives a path that
  doesn't match Bedrock shape.

Verification (in dev container):
- cargo test -p openshell-router --lib: 53/53 (incl. 9 new)
- cargo fmt --check: clean
- cargo clippy -p openshell-core -p openshell-router -p openshell-server
  --all-targets -- -D warnings: clean

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
@st-gr st-gr force-pushed the feat/aws-bedrock-provider branch from 2a8e23b to a1055e2 Compare June 9, 2026 05:17
@johntmyers

Copy link
Copy Markdown
Collaborator

gator-agent

Re-check After Author Update

I re-evaluated latest head a1055e21b73baf9a8f96ca8249fbe05b75ff000e after the June 9, 2026 updates to narrow the PR back to the Bedrock provider/router/docs scope and address the earlier model-selector feedback.

Disposition: partially resolved. The unrelated fork infrastructure and external compute driver changes are removed, the Bedrock route now rewrites the caller path to the operator-configured model, and the docs now describe the bridge-only/no-SigV4 setup. Blocking review feedback remains.

Remaining items:

  • Blocking/build: crates/openshell-sandbox/src/l7/inference.rs constructs the two new Bedrock InferenceApiPattern entries without the required framing field. Please set /model/*/invoke to ResponseFraming::Buffered, set /model/*/invoke-with-response-stream to ResponseFraming::Streaming, and update the framing classification test so aws_bedrock_invoke is expected buffered.
  • Major/correctness: crates/openshell-router/src/backend.rs strips query strings while parsing Bedrock paths and does not restore them when rewriting the upstream path. Please preserve the query tail after replacing the model segment, or explicitly reject Bedrock queries.
  • Major/security: rewrite_bedrock_path interpolates route.model directly into a URL path segment. Please validate or path-segment encode Bedrock model ids so values containing /, ?, #, or % cannot produce malformed or ambiguous upstream paths.
  • Major/policy: providers/aws-bedrock.yaml still declares the real AWS Bedrock endpoint even though this PR's core route is bridge-only and requires BEDROCK_BASE_URL. Provider profile endpoints compose directly into sandbox policy layers, so attaching this provider can grant direct egress to bedrock-runtime.us-east-1.amazonaws.com. Please remove the direct AWS endpoint/binary policy for the bridge-only shape, or split bridge and direct-AWS profiles.
  • Test gap: the route success test writes a provider directly to the store. Please add coverage through create_provider_record or the gRPC provider-create path for the documented placeholder credential plus BEDROCK_BASE_URL command shape.

Checks: DCO was passing in the latest cached state. Branch Checks and Helm Lint still need to run after review feedback is resolved; I am not posting /ok to test while blocking review feedback remains.

Next state: gator:in-review

Closes the buffered-vs-streaming framing warning from gator-agent's
re-check on 4ab587f: "Bedrock InvokeModel should be buffered while
InvokeModelWithResponseStream is streaming. Please add framing/coverage
so /model/{id}/invoke cannot be corrupted by the streaming proxy's
truncation/error-frame behavior."

The InferenceApiPattern struct gained a `framing: ResponseFraming`
field upstream after the original Bedrock-patterns commit (#22b78cff)
landed; the cherry-pick onto current upstream/main left the two
Bedrock entries without the new field. Fixed here:

- aws_bedrock_invoke (POST /model/{id}/invoke):
    framing = ResponseFraming::Buffered
  InvokeModel returns one JSON object the caller decodes whole. Sending
  it through the streaming proxy would risk a mid-body size-cap
  truncation or idle-timeout failure appending an SSE error event onto
  bytes the caller decodes as one JSON body — the same corruption mode
  that drove the existing embeddings + model-discovery to Buffered.
- aws_bedrock_invoke_stream (POST /model/{id}/invoke-with-response-stream):
    framing = ResponseFraming::Streaming
  InvokeModelWithResponseStream returns an AWS event-stream of binary
  chunks; the caller wants chunks incrementally, so the streaming proxy
  path is correct.

Two new tests in `crates/openshell-sandbox/src/l7/inference.rs` pin
down the contract:

- aws_bedrock_invoke_is_buffered — detect_inference_pattern returns a
  Buffered pattern for /model/<id>/invoke, with explanatory message
  naming the corruption mode being prevented.
- aws_bedrock_invoke_stream_is_streaming — same shape, asserting
  Streaming for /model/<id>/invoke-with-response-stream.

Verification (in dev container):
- cargo check -p openshell-sandbox: clean (was failing on missing
  `framing` field before this commit)
- cargo test -p openshell-sandbox --lib l7::inference::tests::aws_bedrock:
  7/7 (incl. 2 new framing tests)
- cargo fmt --check: clean
- cargo clippy --all-targets -- -D warnings: clean

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: st-gr <38470677+st-gr@users.noreply.github.com>
@st-gr

st-gr commented Jun 9, 2026

Copy link
Copy Markdown
Author

@johntmyers — force-rebased the branch onto upstream/main and addressed the four remaining items from your last gator re-check. Branch head is now af7609be, 8 clean commits since upstream/main, mergeable.

Why the force-rebase. Your scope concern was structurally correct — the previous branch was based off my fork's main, which carries fork-only infrastructure (release-gateway workflow, rust-lint workflow, smoke-self-hosted-kyma workflow, Dockerfile.gateway, README-FORK.md) plus the External-driver hunks owned by #1703. The merge of main into the branch dragged those into the PR diff. I rebased the Bedrock-only commits onto upstream/main directly to remove all of it. PR diff is now exactly 9 files, all Bedrock-relevant: crates/openshell-{core,providers,router,sandbox,server}/... plus providers/aws-bedrock.yaml and the two docs/sandboxes/*.mdx pages. No fork-only files. No External-driver hunks.

Addressing the remaining findings:

  • Blocking/security — Bedrock model enforcement (a1055e21): router-side path rewrite + body preservation in prepare_backend_request. For Bedrock routes, parse the inbound /model/<id>/invoke[-with-response-stream] shape and substitute the operator-configured route.model for the caller-supplied segment; reject with RouterError::Internal if the path isn't a recognized Bedrock shape on a Bedrock route. Body model rewriting is skipped — Bedrock encodes the model in the URL path, and the body is the raw provider-specific payload (Anthropic Messages for Claude, Mistral payload for Mistral, etc.) that must not be mutated. New helpers: route_is_bedrock, parse_bedrock_invocation_path, rewrite_bedrock_path. 9 new router tests (parse positive/negative, route_is_bedrock, rewrite, end-to-end wiremock for both invoke and invoke-with-response-stream proving path rewrite + body preservation, defense-in-depth rejection).

  • Warning — buffered vs streaming framing (af7609be): declared framing on the two InferenceApiPattern entries. aws_bedrock_invokeResponseFraming::Buffered (one JSON object served buffered with accurate Content-Length; would be corrupted by streaming size-cap or idle-timeout SSE error frames). aws_bedrock_invoke_streamResponseFraming::Streaming (AWS event-stream of binary chunks must reach the agent incrementally). Two new tests pin down the contract.

  • Warning/docs — AWS secrets in YAML (424b1e2f): all four AWS credentials (aws_access_key_id, aws_secret_access_key, aws_session_token, aws_region) are now required: false, discovery.credentials is empty, and the profile description names the bridge-fronted shape and SigV4 deferral so readers don't have to cross-reference the PR thread. The credentials remain in the schema (declarative-only) so the SigV4 follow-up can flip them back without a schema migration.

  • Major — silent fallback to real AWS (96d48057): AWS_BEDROCK_PROFILE.default_base_url is now "". Without BEDROCK_BASE_URL config, route resolution fails with the existing empty-base_url check (InvalidArgument naming the missing field). The negative test upsert_cluster_route_rejects_aws_bedrock_without_bedrock_base_url pins down this safety contract.

Status

# Severity Status
Scope contamination Blocking Resolved (force-rebase)
Bedrock model enforcement Blocking/security Resolved (a1055e21)
Buffered vs streaming framing Warning Resolved (af7609be)
Silent AWS fallback Major Resolved (96d48057)
AWS secrets in YAML Warning/docs Resolved (424b1e2f)
Validator probe Blocking from earlier review Resolved — docs require --no-verify (424b1e2f)
Provider-create command shape Blocking from earlier review Resolved — docs show actual required shape (424b1e2f)

Verification (run locally in the rust-1.95 dev container)

  • cargo check -p openshell-core -p openshell-router -p openshell-server -p openshell-sandbox — clean
  • cargo test -p openshell-core --lib inference — 19/19 (incl. 3 new aws-bedrock cases)
  • cargo test -p openshell-router --lib — 62/62 (incl. 9 new Bedrock routing cases)
  • cargo test -p openshell-server --lib inference::tests::upsert — 8/8 (incl. positive + negative aws-bedrock)
  • cargo test -p openshell-sandbox --lib l7::inference::tests::aws_bedrock — 7/7 (incl. 2 new framing cases)
  • cargo fmt --check — clean
  • cargo clippy --all-targets -- -D warnings — clean

Branch Checks / Helm Lint stand by for /ok to test whenever ready.

@johntmyers

Copy link
Copy Markdown
Collaborator

gator-agent

Re-check After Author Update

I re-evaluated latest head af7609bed69e016317aaee18fea0654893943c12 after @st-gr's June 9, 2026 comment saying the branch was force-rebased and the remaining Bedrock review items were addressed. I also ran the required independent code-only review for the updated diff.

Disposition: partially resolved. The scope contamination is resolved, the route now rewrites caller-supplied Bedrock model paths to the operator-configured model, /model/*/invoke is now buffered while /invoke-with-response-stream is streaming, the bridge-only/no-SigV4 setup is documented, and missing BEDROCK_BASE_URL is rejected instead of silently falling back to real AWS.

Remaining items:

  • Blocking/security: the configured Bedrock model_id is now interpolated into the outbound URL path in rewrite_bedrock_path, but the non-Vertex route path does not validate Bedrock model IDs before saving/resolving the route. Please reject Bedrock model IDs containing path separators, URL delimiters/percent escapes, whitespace/control characters, or traversal-shaped segments before they can become a URL path segment, and add tests similar to the existing Vertex model-id validation coverage.
  • Blocking/protocol correctness: aws_bedrock_invoke_stream is AWS event-stream, but the shared streaming error path injects SSE-formatted error chunks on byte cap, upstream read error, or idle timeout. Please add protocol-aware stream error framing for Bedrock, or defer advertising /invoke-with-response-stream until the proxy can avoid corrupting Bedrock event-stream clients on truncation/error.
  • Warning: standalone router YAML still appears to require api_key/api_key_env even when provider_type: aws-bedrock resolves to AuthHeader::None. Managed routes work because the server sends an empty key, but static router users may need a dummy key. Please relax that path for no-auth profiles or document the requirement.

Checks: DCO is passing. Branch Checks and Helm Lint are still waiting for /ok to test; I am not posting /ok to test while blocking review feedback remains.

Next state: gator:in-review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gator:in-review Gator is reviewing or awaiting PR review feedback

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants