feat(docs): add volcano docs documentation search subsystem

[tkkhq]

Summary

Adds a volcano docs command group so the CLI — and primarily the AI agent driving it — can search and read the official Volcano documentation on demand, looking things up when it needs more context or wants to validate assumptions against the docs.

Docs are pulled from a configurable GitHub source (default: Kong/volcano-hosting /docs on main), cached locally, and searched with a small, dependency-free in-house BM25 index. Inspired by tobi/qmd but intentionally lightweight and Volcano-specific (no vector DB / LLM reranking / sqlite / CGO). Machine-readable --json output is a first-class, versioned contract for agents.

Commands

volcano docs sync                                   # fetch/refresh local cache
volcano docs search "reset password email"          # ranked BM25 search + snippets
volcano docs search "service key" --topic authentication -n 5
volcano docs get authentication/password-reset.md   # read a whole doc
volcano docs get "authentication/password-reset.md#how-it-works"  # read one section
volcano docs list --topic storage

Group flags: --json, --offline, --repo, --ref, --path. Reads bootstrap the cache once unless --offline; they never touch the network when a cache exists, and mark the cache stale after 7 days while still serving it.

Design

• internal/docs — source resolution/precedence, transactional cache (atomic snapshot publish + manifest), GitHub sync (commit-pinned, blob-SHA reuse, truncated-tree fallback to contents walk, bounded-concurrency downloads, path/size limits), fence-aware markdown section parser (GitHub anchors, breadcrumbs, line ranges), in-memory BM25 with title/heading/path/phrase/technical-token boosts.
• internal/cmd/docs — cobra command group + versioned JSON envelope (schema_version=1, source/cache/data|error, DOCS_* error codes, stdout JSON-only + nonzero exit on error), wired into root.go.
• Config/runtime seams — config.DocsSource persistence and runtime.Deps test seams (DocsCacheDir, DocsGitHubAPIURL, DocsRawBaseURL, Now).

Source precedence: flags → VOLCANO_DOCS_{REPO,REF,PATH} → config.DocsSource → compiled defaults. Each source is cached separately under os.UserCacheDir()/volcano/docs/v1/<sha256 key>.

Note on private repos

The docs repo is currently private, so downloads default to the authenticated GitHub contents API (Accept: application/vnd.github.raw) rather than raw.githubusercontent.com. GITHUB_TOKEN is sent only to api.github.com, never to raw/injected hosts. The raw-host path remains as an optional override for public sources/tests.

Indexing tradeoff

The BM25 index is built in memory per search invocation (no persisted index). Since each command is a one-shot process, subsequent searches rebuild it from the on-disk cache — ~70 ms warm for the current ~800 KB / 96-file corpus (~6.7 MB in memory), so persisting it isn't worth the added format/versioning/invalidation complexity. If the corpus grows substantially or we add a long-lived server/MCP mode, the index would simply stay resident across requests.

Testing

• go test ./... — all packages pass (httptest-backed coverage for SHA reuse, deletion, truncation fallback, partial-failure cache preservation, offline behavior, staleness, JSON success/error contracts; plus parser and BM25 ranking unit tests).
• make lint — 0 issues.
• Live validated against Kong/volcano-hosting: synced 96 docs, ranked search + section-level get + offline reads all working.

Docs

Adds docs/docs-search.md and a README link.

[Copilot]

Pull request overview

Adds a configurable volcano docs subsystem for caching, searching, listing, and reading Volcano documentation.

Changes:

• Adds transactional GitHub-backed documentation synchronization and caching.
• Implements Markdown parsing and BM25 section search.
• Adds Cobra commands, versioned JSON output, tests, and user documentation.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 14 comments.

Show a summary per file

File	Description
README.md	Links the documentation search guide.
docs/docs-search.md	Documents commands, JSON output, and configuration.
internal/runtime/runtime.go	Adds docs-specific runtime seams.
internal/config/config.go	Adds persisted docs source configuration.
internal/docs/cache.go	Implements snapshot-based cache storage.
internal/docs/errors.go	Defines docs errors and machine codes.
internal/docs/parse.go	Parses Markdown into searchable sections.
internal/docs/parse_test.go	Tests section parsing and anchors.
internal/docs/search.go	Implements BM25 indexing and snippets.
internal/docs/search_test.go	Tests ranking, filtering, and tokens.
internal/docs/service.go	Coordinates cache, search, list, and retrieval.
internal/docs/source.go	Resolves and validates documentation sources.
internal/docs/source_test.go	Tests source precedence and validation.
internal/docs/sync.go	Synchronizes GitHub documents into snapshots.
internal/docs/sync_test.go	Tests synchronization and cache behavior.
internal/cmd/docs/docs.go	Defines the command group and JSON envelope.
internal/cmd/docs/subcommands.go	Implements docs subcommands and rendering.
internal/cmd/docs/docs_test.go	Tests command and JSON contracts.
internal/cmd/root/root.go	Registers the docs command.
internal/cmd/root/root_test.go	Verifies docs appears in root help.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.