Skip to content

CLI: 'preflight' vs 'provider-preflight' naming is confusing #4

Description

@rendrag-git

UX problem

Two subcommands with similar names and overlapping semantics:

  • `oc-bench preflight --suite ... --models ...` — benchmark-run preflight (predates M3). Verifies a suite + model config will run end-to-end safely.
  • `oc-bench provider-preflight --profile X --provider vllm --base-url Y --route-model Z` — provider-route preflight (M3). Runs four gates against a provider config.

A user reading `--help` has to read both descriptions to know which to invoke. The names don't telegraph the difference.

Options

  1. Rename `preflight` → `run-preflight` and `provider-preflight` → `provider-check` (or similar). Breaking change.
  2. Subcommand groups: `oc-bench preflight run` and `oc-bench preflight provider`. Cleaner but bigger restructure.
  3. Leave as-is, sharpen `--help` text: change the two help strings so the difference is obvious from `oc-bench --help` alone. Lowest effort, lowest payoff.

Recommendation

Option 3 first (1-line fix). Revisit option 2 when the next CLI surface change lands.

Severity

P3 — neither command misbehaves; just easy to invoke the wrong one.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions