Add adversarial PR reviewer agent + claim verification skill#2070
Open
starlightretailceo wants to merge 2 commits into
Open
Add adversarial PR reviewer agent + claim verification skill#2070starlightretailceo wants to merge 2 commits into
starlightretailceo wants to merge 2 commits into
Conversation
Contributor
There was a problem hiding this comment.
main, but PRs should target staged.
The main branch is auto-published from staged and should not receive direct PRs.
Please close this PR and re-open it against the staged branch.
You can change the base branch using the Edit button at the top of this PR,
or run: gh pr edit 2070 --base staged
Contributor
✅ External plugin PR checks passed
Per-plugin quality summary
No changed external plugin entries were detected in this PR. |
Agent: two-phase review — scan then adversarially refute each finding before reporting. Skill: reusable verification pattern with refutation prompts, vote thresholds, false-positive examples. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
e4a7eca to
3e84467
Compare
Contributor
🔒 PR Risk Scan ResultsScanned 4 changed file(s).
|
Contributor
🔍 Skill Validator Results⛔ Findings need attention
Summary
Full validator output
|
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
adversarial-pr-reviewer.agent.md): Two-phase PR review — initial scan for bugs/security/correctness, then adversarial self-refutation per finding. Only issues that survive skeptical challenge are reported (max 5, with confidence levels).skills/adversarial-claim-verification/SKILL.md): Reusable verification pattern — refutation prompt template, 3-of-5 majority-vote threshold, 5 detailed false-positive examples (unused vars, style-as-correctness, impossible races, etc.), confidence calibration, and escalation criteria.Motivation
Existing review agents in this repo do single-pass review and report everything plausible. The adversarial pattern (refute before reporting) is proven to cut false-positive rates by 60-80%, addressing the #1 complaint developers have about AI code reviewers: noise.
Test plan
🤖 Generated with Claude Code