Automate Copilot re-review requests and add review loop#590
Automate Copilot re-review requests and add review loop#590
Conversation
The respond-to-copilot command was single-pass and told the user to manually click the re-request button in GitHub UI. Copilot re-reviews can actually be requested via API using the full bot account name `copilot-pull-request-reviewer[bot]`. Changes: - Rewrite respond-to-copilot command with loop structure that runs until Copilot has no new comments - Re-request reviews via API, poll for arrival (15s intervals, 10min timeout), then check for new comments - Track comment IDs across rounds to distinguish new from addressed - Update github-cli rules to replace all "no API for re-reviews" guidance with the working API call (8 locations) - Trim command from 145 to 88 lines Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR updates internal Claude/GitHub CLI guidance to support an automated “fix → re-request Copilot review → poll → repeat” loop, replacing prior manual “click 🔄” instructions and refreshing stale references.
Changes:
- Rewrites the
/respond-to-copilotcommand to iterate until no new Copilot comments appear (or polling times out). - Updates Copilot re-review instructions to use
copilot-pull-request-reviewer[bot]via the REST API. - Refreshes multiple workflow examples/checklists in the GitHub CLI rules to reflect the new re-review approach.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
.claude/rules/github-cli.md |
Updates Copilot re-review workflow and examples to use the bot login + adds polling guidance. |
.claude/commands/respond-to-copilot.md |
Reworks the command into a multi-round loop with steps for reply/resolve/re-request/poll. |
You can also share your feedback on Copilot code review. Take the survey.
| ```bash | ||
| gh api repos/{owner}/{repo}/pulls/{pr_number}/comments/{comment_id}/replies \ | ||
| -method POST -f body="Your response" | ||
| gh api repos/chenders/deadonfilm/pulls/{pr_number}/comments | jq '.[] | {id, body, path, line}' |
There was a problem hiding this comment.
Not implementing — the agent processing these comments uses contextual analysis to determine which are Copilot comments vs human comments. Pre-filtering in the jq command would lose context about the full review thread (e.g., human replies to Copilot comments).
- Standardize placeholders to {PR} and {COMMENT_ID} across command doc
- Fix polling to filter by copilot-pull-request-reviewer login and use
baseline count comparison instead of total review count
- Add note about dual Copilot logins (Copilot vs copilot-pull-request-reviewer[bot])
- Update multi-round example to filter comments by both Copilot logins
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Updates the repository’s Claude/Copilot operational docs to support an automated “fix → reply → resolve → re-request Copilot review → poll → repeat” loop, and replaces stale guidance that re-reviews must be triggered manually in the GitHub UI.
Changes:
- Rewrite
/respond-to-copilotdocumentation to run in rounds until Copilot produces no new comments (or a timeout). - Replace “click 🔄” re-review instructions with
gh api .../requested_reviewerscalls targetingcopilot-pull-request-reviewer[bot]. - Refresh multiple workflow examples/checklists in the GitHub CLI rules to match the new re-review approach.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
.claude/rules/github-cli.md |
Updates Copilot re-review instructions, examples, and checklist to use an API-driven re-request + polling workflow. |
.claude/commands/respond-to-copilot.md |
Reworks the command doc into a multi-round loop with explicit re-request + polling steps. |
You can also share your feedback on Copilot code review. Take the survey.
All jq filters for Copilot reviews now use the full bot login `copilot-pull-request-reviewer[bot]` instead of the truncated version. Also removed orphaned bullet point in re-review section. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Updates the repo’s internal GitHub CLI + Claude command docs to support an automated “fix → reply → resolve → re-request Copilot review → poll → repeat” loop, using the copilot-pull-request-reviewer[bot] account for re-reviews instead of manual UI steps.
Changes:
- Document re-requesting Copilot re-reviews via
POST /pulls/{PR}/requested_reviewerswithcopilot-pull-request-reviewer[bot]. - Rewrite
/respond-to-copilotto be explicitly multi-round with polling and completion criteria. - Refresh workflow examples/quick-reference to reflect the new API-based re-review approach.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
.claude/rules/github-cli.md |
Replaces manual “click 🔄” instructions with API-based re-review requests, polling guidance, and updated examples. |
.claude/commands/respond-to-copilot.md |
Refactors the command doc into a per-round loop that re-requests Copilot review and repeats until no new comments. |
You can also share your feedback on Copilot code review. Take the survey.
| 2. **Fetch all review comments** | ||
| - Run `gh api repos/{owner}/{repo}/pulls/{pr_number}/comments` to get all review comments | ||
| - Filter for comments from `github-actions[bot]` or `copilot` that represent Copilot suggestions | ||
| - Also check `gh pr view [PR] --json reviews` for review-level comments | ||
|
|
||
| 3. **Analyze each comment** | ||
| For each Copilot comment, evaluate: | ||
| - **Validity**: Is the suggestion technically correct? | ||
| - **Relevance**: Does it apply to the actual code context? | ||
| - **Value**: Would implementing it improve code quality, security, performance, or maintainability? | ||
| - **Scope**: Is it within the scope of this PR, or is it unrelated cleanup? | ||
| - **Trade-offs**: Are there downsides to the suggestion (complexity, over-engineering, etc.)? | ||
|
|
||
| 4. **Categorize suggestions** | ||
| - **Will implement**: Valid, valuable, and within scope | ||
| - **Won't implement**: Invalid, not valuable, or has significant trade-offs | ||
| - **Needs discussion**: Unclear or requires user input | ||
|
|
||
| **IMPORTANT: Never defer work or create issues without explicit user approval.** If a suggestion is valid but you believe it's out of scope: | ||
| - First, attempt to implement it if it's reasonably small | ||
| - If it's too large, ask the user: "This suggestion would require significant work. Should I implement it now, or would you prefer to defer it to a separate PR?" | ||
| - Only create tracking issues if the user explicitly asks for deferral | ||
|
|
||
| 5. **Make changes for accepted suggestions** | ||
| - Implement the changes for suggestions you've decided to accept | ||
| - Run tests to ensure changes don't break anything: `npm test && cd server && npm test` | ||
| - Run quality checks: `npm run lint && npm run type-check` | ||
|
|
||
| 6. **Commit and push changes before responding** | ||
| - Stage and commit with a message like: "Address Copilot review feedback" | ||
| - Push the changes to update the PR | ||
| - Note the commit SHA for use in responses: `git rev-parse --short HEAD` | ||
|
|
||
| 7. **Respond to each comment on GitHub** | ||
| Use `gh api` to reply to each comment: | ||
| ```bash | ||
| gh api repos/{owner}/{repo}/pulls/{pr_number}/comments/{comment_id}/replies \ | ||
| -method POST -f body="Your response" | ||
| gh api repos/chenders/deadonfilm/pulls/{PR}/comments | jq '.[] | {id, body, path, line}' | ||
| ``` |
There was a problem hiding this comment.
Not implementing. PRs in this repo rarely exceed 30 review comments. The agent processing these instructions can handle pagination if needed — this is guidance for an AI agent, not a script to be copy-pasted verbatim.
| 8. **Resolve implemented threads** (use PRRT* thread IDs, not PRRC* comment IDs) | ||
|
|
||
| 9. **Notify user to request Copilot re-review** | ||
| If any changes were committed and pushed, tell the user: | ||
| ```bash | ||
| # Get thread IDs | ||
| gh api graphql -f query='query { repository(owner: "chenders", name: "deadonfilm") { pullRequest(number: {PR}) { reviewThreads(first: 50) { nodes { id isResolved comments(first: 1) { nodes { body } } } } } } }' | ||
|
|
||
| > "Changes pushed. To trigger a Copilot re-review, click the 🔄 re-request button next to Copilot's name in the Reviewers section on the PR page." | ||
| # Resolve | ||
| gh api graphql -f query='mutation { resolveReviewThread(input: {threadId: "PRRT_..."}) { thread { isResolved } } }' | ||
| ``` |
There was a problem hiding this comment.
Not implementing. The agent matches threads to comments by comparing the body text preview, which works reliably in practice. Adding comment node IDs would add complexity without solving a real problem.
| 3. **Check for new comments** — If there are no new unaddressed comments since the last round, the loop is done. Report the final status and stop. | ||
|
|
||
| 8. **REQUIRED: Resolve implemented comment threads** | ||
| 4. **Analyze each new comment** | ||
| - Validity: Is the suggestion technically correct? | ||
| - Value: Would it improve code quality? | ||
| - Scope: Is it within the scope of this PR? |
There was a problem hiding this comment.
Not implementing. Copilot always posts inline review comments, not just review-level bodies. The comment-based loop detection has worked correctly across multiple rounds on this PR.
| **There is no API or CLI for re-reviews.** See [GitHub community discussion #186152](https://github.com/orgs/community/discussions/186152). | ||
| ```bash | ||
| # Baseline before re-request | ||
| gh api repos/chenders/deadonfilm/pulls/123/reviews --jq '[.[] | select(.user.login == "copilot-pull-request-reviewer[bot]")] | length' |
There was a problem hiding this comment.
Not implementing. PRs in this repo never approach 30 reviews. Adding --paginate would slow down the 15-second polling loop for no practical benefit.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Updates the repository’s Claude/Copilot operational docs to automate Copilot re-review requests and formalize a multi-round “fix → reply → resolve → re-request → poll” loop until no new Copilot comments appear.
Changes:
- Rewrites
/respond-to-copilotinstructions to run in rounds with explicit re-request + polling behavior. - Replaces prior “click 🔄 in UI” guidance with a REST API re-request using
copilot-pull-request-reviewer[bot]. - Refreshes multiple stale Copilot re-review references across the GitHub CLI rule doc.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
.claude/rules/github-cli.md |
Updates re-review guidance, examples, and quick-reference commands to use API-based Copilot re-requests. |
.claude/commands/respond-to-copilot.md |
Replaces single-pass guidance with a multi-round loop, including re-request + polling steps and termination criteria. |
| **Re-reviews:** Use the bot account name `copilot-pull-request-reviewer[bot]` (not `Copilot`) to re-request reviews: | ||
|
|
||
| ```bash | ||
| gh api repos/chenders/deadonfilm/pulls/123/requested_reviewers \ | ||
| -X POST -f 'reviewers[]=copilot-pull-request-reviewer[bot]' |
There was a problem hiding this comment.
The doc now instructs re-requesting reviews via copilot-pull-request-reviewer[bot], but the later “Reading Copilot Comments” examples still filter reviews/comments to "Copilot" only (.user.login == "Copilot" and .author.login == "Copilot"). After a re-request, this will miss bot-authored comments/reviews; update those examples to include the bot login (or check both logins consistently).
| # Request Copilot review (initial or re-review) | ||
| gh api repos/OWNER/REPO/pulls/PR/requested_reviewers \ |
There was a problem hiding this comment.
There’s an internal inconsistency: earlier sections describe using Copilot for the initial review and copilot-pull-request-reviewer[bot] for re-reviews, but this Quick Reference labels the bot login as working for “initial or re-review”. Please reconcile these instructions (either document both supported forms explicitly or make all snippets use the same reviewer identity) to avoid copy/paste mistakes.
| # Request Copilot review (initial or re-review) | |
| gh api repos/OWNER/REPO/pulls/PR/requested_reviewers \ | |
| # Request Copilot review (initial) | |
| gh api repos/OWNER/REPO/pulls/PR/requested_reviewers \ | |
| -X POST -f 'reviewers[]=Copilot' | |
| # Request Copilot re-review | |
| gh api repos/OWNER/REPO/pulls/PR/requested_reviewers \ |
| ```bash | ||
| gh api repos/{owner}/{repo}/pulls/{pr_number}/comments/{comment_id}/replies \ | ||
| -method POST -f body="Your response" | ||
| gh api repos/chenders/deadonfilm/pulls/{PR}/comments | jq '.[] | {id, body, path, line}' |
There was a problem hiding this comment.
The “Fetch all review comments” example only outputs {id, body, path, line}, but later Notes say to filter by Copilot login variants when needed and to track comment IDs across rounds. Without including user.login (and ideally created_at) in the output, it’s harder to distinguish Copilot vs human comments and to debug “new comment” detection. Consider adding user: { login } (and optionally created_at) to the jq projection.
| gh api repos/chenders/deadonfilm/pulls/{PR}/comments | jq '.[] | {id, body, path, line}' | |
| gh api repos/chenders/deadonfilm/pulls/{PR}/comments | jq '.[] | {id, body, path, line, user: { login: .user.login }, created_at}' |
Summary
/respond-to-copilotcommand to loop until Copilot has no new comments, instead of single-passcopilot-pull-request-reviewer[bot]github-cli.mdrulesChanges
.claude/commands/respond-to-copilot.md(145 → 88 lines):copilot-pull-request-reviewer[bot]API (not the shortCopilotname which only works for initial reviews).claude/rules/github-cli.md:Test plan
🤖 Generated with Claude Code