Skip to content

Commit 98abd24

Browse files
abeltranoCopilot
andcommitted
Add opt-in executive summary and focus areas to review-pull-request
Adds a new `executive_summary` parameter to the `review-pull-request` template (`auto` default, plus `always` and `never`). When triggered, the template produces a holistic change narrative and a short suggested-focus-areas list to help a reviewer orient themselves on broad-scope or large-diff PRs and decide where to concentrate attention. The narrative and focus areas are surfaced via the report's Problem Statement section in document mode and presented to the reviewer before findings walk-through in action mode. They are never auto-posted to the GitHub review body. Phase 2 per-file coverage is preserved; focus areas advise reviewer attention only and never reduce coverage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 562c9ec commit 98abd24

1 file changed

Lines changed: 106 additions & 4 deletions

File tree

templates/review-pull-request.md

Lines changed: 106 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ params:
2222
additional_protocols: "Optional — specific protocols to apply (e.g., memory-safety-c, thread-safety)"
2323
context: "What this PR does, which system it affects, any known concerns"
2424
output_mode: "Output mode — 'document' (produce investigation report) or 'action' (post review comments via gh CLI)"
25+
executive_summary: "Whether to include a holistic change narrative explaining what the PR is actually doing — 'auto' (default; produced for broad-scope or large PRs based on diff size, breadth, and impact signals), 'always', or 'never'. Blank or invalid values are treated as 'auto'."
2526
input_contract: null
2627
output_contract:
2728
type: investigation-report
@@ -51,6 +52,8 @@ isolation, but the diff, commit history, linked issues, and CI status.
5152

5253
**Output Mode**: {{output_mode}}
5354

55+
**Executive Summary Mode**: {{executive_summary}}
56+
5457
## Instructions
5558

5659
### Phase 1: Gather PR Context
@@ -78,6 +81,83 @@ isolation, but the diff, commit history, linked issues, and CI status.
7881
protocols are relevant (e.g., `memory-safety-c` for C,
7982
`thread-safety` for concurrent code). Apply these in Phase 2.
8083

84+
5. **Establish Change Narrative** (when applicable per `executive_summary`).
85+
86+
**Normalization.** If `executive_summary` is blank, unset, or not one
87+
of `auto`, `always`, or `never`, treat it as `auto`. This guards
88+
against missing values when the template is packaged as a Copilot
89+
prompt file or agentic workflow.
90+
91+
**Determine scope:**
92+
- `never` — skip the rest of this step.
93+
- `always` — produce the narrative.
94+
- `auto` (default) — produce the narrative when **any** of these
95+
triggers fires:
96+
- **Size signals**: more than ~300 lines changed (additions +
97+
deletions, excluding lockfiles, generated code, and vendored
98+
deps), OR more than ~15 source files modified (same exclusions).
99+
If exact filtered counts are unavailable, estimate
100+
conservatively rather than spending excessive effort calculating.
101+
- **Breadth signals**: the PR touches multiple unrelated subsystems
102+
(e.g., distinct top-level code roots that don't simply mirror
103+
each other like `src/` and `tests/`).
104+
- **Impact signals**: the PR introduces public API or interface
105+
changes, schema or data-model changes, auth / permission /
106+
security changes, CI or deployment / infrastructure changes, or
107+
broad dependency or configuration changes.
108+
- **Description-gap signals**: the PR description is sparse
109+
(< 2 short paragraphs) AND total change exceeds 100 LOC.
110+
111+
**Skip condition (`auto` only):** even when a trigger fires, skip
112+
the narrative if the PR description already provides a clear,
113+
accurate change overview AND the diff does not reveal materially
114+
different scope from what the description claims.
115+
116+
**Compose narrative** (1–3 paragraphs, ≤ ~300 words). Answer:
117+
- **What** the PR changes functionally (user-visible behavior, API
118+
surface, data model)
119+
- **How** it changes things architecturally (key restructurings,
120+
new modules, removed code paths)
121+
- **Why**, per the PR description and linked issues — state the
122+
author's stated motivation; do not speculate beyond it
123+
- **Scope boundaries** — what is *not* changed despite appearing
124+
in the diff (generated files, formatting-only edits, vendored deps)
125+
126+
Ground every narrative claim in the diff, PR description, linked
127+
issues, or context files you read. Omit ungrounded claims.
128+
129+
**Compose suggested focus areas** (1–5 bulleted items; omit element
130+
when none warrant calling out). Produce only when the narrative was
131+
produced above; otherwise omit. For each item state:
132+
- The specific file path(s) or module — do not generalize
133+
- The motivating signal: an **impact signal** that fired in the
134+
trigger evaluation above (public API, schema, auth, CI/deploy,
135+
broad deps), OR an observable **concentration of complexity** in
136+
the diff (e.g., a single file with disproportionate change density
137+
or intricate logic)
138+
- The **risk angle** in one short phrase (correctness, safety,
139+
security, or test coverage)
140+
141+
Identify focus areas only when at least one of the conditions above
142+
holds. If neither an impact signal fired nor concentration of
143+
complexity is observable, omit the focus-areas element rather than
144+
fabricating items. Focus areas are advisory pointers for reviewer
145+
attention; they do **not** reduce required Phase 2 coverage — every
146+
changed file remains subject to analysis regardless of whether it
147+
appears in the focus list.
148+
149+
**Format requirement.** If either the narrative or the focus areas
150+
were produced, the report MUST use the **full** investigation-report
151+
format (so the `Problem Statement` section is present), regardless
152+
of finding count or severity. This overrides the format's default
153+
abbreviated-vs-full selection rule.
154+
155+
Internally record whether the narrative was produced and, in `auto`
156+
mode, which trigger fired (or why it was skipped). In document mode,
157+
surface this briefly in the Coverage section in Phase 5. In action
158+
mode, the narrative (if produced) is shown to the user during step 1
159+
of the action flow; no Coverage-section recording applies.
160+
81161
### Phase 2: Analyze Changes
82162

83163
Apply the **anti-hallucination protocol** throughout — base your review
@@ -159,19 +239,22 @@ PR review concepts to report sections:
159239

160240
| Report Section | PR Review Content |
161241
|---|---|
162-
| Executive Summary | Overall verdict + key findings |
163-
| Problem Statement | What the PR claims to change and why |
242+
| Executive Summary | Overall verdict + key findings (2–4 sentences per format) |
243+
| Problem Statement | What the PR claims to change and why. When the change narrative was produced in Phase 1, lead this section with the holistic narrative, then the suggested focus areas (if any were produced), then state the author's motivation. |
164244
| Investigation Scope | Files changed, diff size, linked issues |
165245
| Findings | Per-file findings with severity |
166246
| Root Cause Analysis | Omit or use for systemic patterns |
167247
| Remediation Plan | Suggested fixes ordered by priority |
168248
| Prevention | Process suggestions (testing, CI checks) |
169249
| Open Questions | Ambiguities in the PR or linked issues |
170-
| Coverage | Files examined, search method, exclusions, limitations |
250+
| Coverage | Files examined, search method, exclusions, limitations. In `auto` mode for `executive_summary`, briefly note whether the change narrative was produced and the triggering condition (or reason for skipping). |
171251

172252
#### Action Mode (`output_mode: action`)
173253

174254
1. **Present findings** to the user using the document structure above.
255+
If a change narrative or suggested focus areas were produced in
256+
Phase 1, present them (from `Problem Statement`) before walking
257+
through individual findings.
175258
2. **Ask the user to confirm** which findings to post as review comments.
176259
Present each finding (or batch by file) and ask:
177260
- Post this comment? (yes / skip / edit)
@@ -185,7 +268,17 @@ PR review concepts to report sections:
185268
only; code references, file paths, and quoted reviewer text are
186269
exempt.
187270
3. **Post confirmed findings** as inline review comments using a JSON
188-
payload file so `comments` is sent as an array, not a string:
271+
payload file so `comments` is sent as an array, not a string.
272+
273+
**Constructing the `body` field**: the review `body` is the
274+
overall verdict summary (findings + recommended review event).
275+
Do NOT prepend the Phase 1 change narrative or suggested focus
276+
areas — the PR author already wrote the PR description, and
277+
reviewer feedback should focus on the verdict, not on re-describing
278+
the change. These are reviewer-internal grounding artifacts; do not
279+
add them to the body unless the user explicitly requests it as part
280+
of confirming the review.
281+
189282
```
190283
cat > review.json <<'EOF'
191284
{
@@ -225,6 +318,12 @@ PR review concepts to report sections:
225318
action mode, you may post an `APPROVE`, `REQUEST_CHANGES`, or `COMMENT`
226319
review only after explicit user confirmation, and you must not merge.
227320
- Do NOT modify the PR branch or push commits.
321+
- Do NOT speculate in the change narrative or suggested focus areas
322+
beyond what is visible in the diff, PR description, or linked
323+
issues — including author intent that is not explicitly stated.
324+
- Do NOT use suggested focus areas to reduce Phase 2 coverage. Focus
325+
areas advise reviewer attention only; every changed file remains
326+
subject to analysis.
228327

229328
## Quality Checklist
230329

@@ -238,4 +337,7 @@ Before finalizing, verify:
238337
- [ ] Linked issues were checked against the actual changes
239338
- [ ] CI status was noted (or stated as unavailable)
240339
- [ ] At least 3 findings have been re-verified against the diff
340+
- [ ] In `auto` mode for `executive_summary`, the decision to produce or skip the change narrative was made deliberately; in document mode, the decision is also recorded in the Coverage section
341+
- [ ] If the change narrative was produced, every claim in it is grounded in the diff, PR description, linked issues, or context files (no speculation about author intent)
342+
- [ ] If suggested focus areas were produced, each item is anchored to an observable impact signal that fired or to concentration of complexity in the diff (no fabrication)
241343
- [ ] In action mode: user confirmation was obtained before every post

0 commit comments

Comments
 (0)