Plan review: add compatibility fallback design + test vectors#3847
Plan review: add compatibility fallback design + test vectors#3847nguyenhoangduc0707-lang wants to merge 14 commits into
Conversation
Adds a design document describing a JSON-first then bullets heuristic for plan review menus and example test vectors. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…(JSON-first then bullets heuristic) Implements parsePlanReviewOptions and mocha/ts-node tests using existing test vectors. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… + id-extraction tests\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…mode flow\n\nProvides TypeScript and Go examples and testing notes.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…val flow (placeholder path)\n\nIncludes unified-diff style patch and notes for maintainers.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…\nRuns npm ci and npm test to validate prototype behavior.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Summary of updates:\n- Added TypeScript prototype parser (JSON->YAML->lists) with normalization and tests (6 passing locally).\n- Added integration docs and a generic patch for maintainers.\n- Added GitHub Actions CI workflow that runs npm ci && npm test on PRs.\n\nRequest:\nPlease review and advise where to wire parsePlanReviewOptions into the CLI codebase; if you enable the workflow it will validate tests automatically. If you prefer, I can produce a targeted patch given the exact file path. Thanks! |
…llback in tests via options\n\n- Feature flag: COPILOT_PLAN_FALLBACK=1 or options.enableFallback=true; disabled by default\n- Sanitizes ids/labels to prevent injection and limit length\- Tests updated to enable fallback explicitly\\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
@maintainers Xin chào — đề nghị review và canary rollout cho PR này. Tóm tắt nhanh:\n\n- Thay đổi: thêm prototype parser (JSON/YAML/Lists), sanitization, feature-flag (OFF theo mặc định), tests (6 passing) và CI workflow.\n- An toàn: fallback chỉ bật khi COPILOT_PLAN_FALLBACK=1 hoặc options.enableFallback=true; mặc định tắt.\n\nĐề xuất canary rollout (bằng tiếng Việt):\n1) Tạo branch canary/plan-review-fallback và merge PR vào branch đó để thử nghiệm nội bộ.\n2) Bật biến môi trường cho workflow CI trên branch canary (thêm vào workflow job �nv: COPILOT_PLAN_FALLBACK: '1') để chạy test vector với fallback ON.\n3) Thử nghiệm với một vài backend 'strict OpenAI-compatible' để xác nhận menu hiển thị đúng và không có hành vi thực thi không mong muốn.\n4) Nếu ổn, cân nhắc thêm feature-flag runtime (setting) để admin bật cho người dùng cụ thể.\n\nSnippet để bật flag trong workflow (thêm vào job trong .github/workflows/...):\n |
|
@Admin Hello — request for review and canary rollout guidance. Summary of updates:\n\n- Prototype parser: JSON -> YAML -> numbered/bulleted lists fallback, with sanitization and feature-flag off by default.\n- Tests: 6 passing locally; GitHub Actions CI added to run npm ci && npm test.\n- Safety: parser only used when COPILOT_PLAN_FALLBACK=1 or options.enableFallback=true; otherwise existing flow (function_call) remains primary and a minimal Accept/Request changes fallback is returned.\n\nSuggested canary rollout steps:\n1) Merge this branch into a 'canary/plan-review-fallback' branch for internal testing.\n2) Enable COPILOT_PLAN_FALLBACK=1 in the CI workflow for that branch to validate behavior with fallback on.\n3) Test against strict OpenAI-compatible backends to verify menu rendering and confirm no unintended execution.\n4) If stable, introduce a runtime feature-flag to toggle for users or orgs and roll out gradually.\n\nIf you want, I can produce a direct patch for the exact file that builds plan-approval menus — just provide the path. Thanks! |
|
Additional technical details and security checks:\n\n- Parsing algorithm (deterministic order):\n 1) Prefer structured metadata.function_call.arguments (JSON) if present.\n 2) Fenced code blocks: try JSON, then YAML.\n 3) Inline JSON objects/arrays.\n 4) Bare YAML docs (---).\n 5) Numbered lists (1., 2., ...).\n 6) Bulleted lists (-, *, +).\n 7) Minimal fallback if none found.\n\n- Sanitization & safety:\n - IDs are sanitized (trim, collapse spaces to _, remove unsafe chars, max 50 chars).\n - Labels/descriptions stripped of control chars, collapsed whitespace, max 200 chars.\n - No model text is executed; parsed items are display-only (labels only).\n - Feature-flag OFF by default; enable with COPILOT_PLAN_FALLBACK=1 or parsePlanReviewOptions(..., { enableFallback: true }).\n\n- Tests/CI: 6 unit tests added; GitHub Actions workflow included to run |
…n.md\n\nIncludes tasks, detailed steps, acceptance criteria, estimated time, and a comprehensive security checklist for tokens/sessions.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ature flags)\n\nDo NOT commit real secrets; use GitHub Secrets or a vault.
…for YAML parsing to avoid unsafe tags
…lnerabilities; update lockfile
…remediations, roadmap)\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Security audit report added: SECURITY_AUDIT_REPORT.md in this branch. Summary: dependency override for serialize-javascript applied, parser hardened (JSON_SCHEMA, size cap 50KB, items cap 50), secretlint + husky pre-commit, truffleHog CI, CodeQL workflow, and fuzz tests (11 passing). Please review the report and advise next steps for pinning action SHAs and canary rollout. @Admin |
1 participant
This PR adds a design document and test vectors for a compatibility fallback for plan review menus when using strict OpenAI-compatible backends that do not provide function_call/tool metadata.
It specifies a JSON-first parsing strategy followed by YAML and numbered/bulleted-list heuristics, and includes representative test cases to guide implementation.
If maintainers like this design, next steps are: