learn-from-pr

Learn From PR

Extracts lessons learned from a completed PR to improve repository documentation and agent capabilities.

Inputs

Input	Required	Source
PR number or Issue number	Yes	User provides (e.g., "PR #33352" or "issue 33352")

Outputs

1. Learning Analysis - Structured markdown with:

   • What happened (problem, attempts, solution)
   • Fix location analysis (attempted vs actual)
   • Failure modes identified
   • Prioritized recommendations

2. Actionable Recommendations - Each with:

   • Category, Priority, Location, Specific Change, Why It Helps

Completion Criteria

The skill is complete when you have:

• Gathered PR diff and metadata
• Analyzed fix location (attempted vs actual)
• Identified failure modes
• Generated at least one concrete recommendation
• Presented findings to user

When to Use

• After agent failed to find the right fix
• After agent succeeded but took many attempts
• After agent succeeded quickly (to understand what worked)
• When asked "what can we learn from PR #XXXXX?"

When NOT to Use

• Before PR is finalized (use pr-finalize first)
• For trivial PRs (typo fixes, simple changes)
• When no agent was involved (nothing to analyze)

---

Workflow

Step 1: Gather Data

# Required: Get PR info
gh pr view XXXXX --json title,body,files
gh pr diff XXXXX

Analyze the PR to extract learning:

1. PR discussion - Comments reveal what was tried
2. Commit history - Multiple commits may show iteration
3. Code complexity - Non-obvious fixes suggest learning opportunities
4. Similar past issues - Search for related bugs

Focus on: "What would have helped an agent find this fix faster?"

Step 2: Fix Location Analysis

Critical question: Did agent attempts target the same files as the final fix?

# Where did final fix go?
gh pr view XXXXX --json files --jq '.files[].path' | grep -v test

Scenario	Implication
Same files	Agent found right location
Different files	Major learning opportunity - document why

If different files: Answer these questions:

• Why did agent think that was the right file?
• What search would have found the correct file?

Step 3: Analyze Outcome

Determine which scenario applies and look for the relevant patterns:

Scenario A: Agent Failed

Pattern	Indicator
Wrong file entirely	All attempts in File A, fix in File B
Tunnel vision	Only looked at file mentioned in error
Trusted issue title	Issue said "crash in X" so only looked at X
Pattern not generalized	Fixed one instance, missed others
Didn't search codebase	Never found similar code patterns
Missing platform knowledge	Didn't know iOS/Android/Windows specifics
Wrong abstraction layer	Fixed handler when problem was in core
Misread error message	Error pointed to symptom, not cause
Incomplete context	Didn't read enough surrounding code
Over-engineered	Complex fix when simple one existed

Scenario B: Agent Succeeded Slowly (many attempts)

Pattern	Indicator
Correct file, wrong approach	Found right file but tried wrong fixes first
Needed multiple iterations	Each attempt got closer but wasn't quite right
Discovery was slow	Eventually found it but search was inefficient
Missing domain knowledge	Had to learn something that could be documented

Key question: What would have gotten agent to the solution faster?

Scenario C: Agent Succeeded Quickly

Pattern	Indicator
Good search strategy	Found right file immediately
Understood the pattern	Recognized similar issues from past
Documentation helped	Existing docs pointed to solution
Simple, minimal fix	Didn't over-engineer

Key question: What made this work? Should we reinforce this pattern?

Step 4: Find Improvement Locations

# Discover where agent guidance lives
find .github/instructions -name "*.instructions.md" 2>/dev/null
find .github/skills -name "SKILL.md" 2>/dev/null
ls docs/design/ 2>/dev/null
ls .github/copilot-instructions.md 2>/dev/null

Location	When to Add Here
.github/instructions/*.instructions.md	Domain-specific AI guidance (testing patterns, platform rules)
.github/skills/*/SKILL.md	Skill needs new step, checklist, or improved workflow
/docs/design/*.md	Detailed architectural documentation
.github/copilot-instructions.md	General AI workflow guidance
Code comments	Non-obvious code behavior

Step 5: Generate Recommendations

For each recommendation, provide:

1. Category: Instruction file / Skill / Architecture doc / Inline comment / Linting issue
2. Priority: High (prevents class of bugs) / Medium (helps discovery) / Low (nice to have)
3. Location: Exact file path
4. Specific Change: Exact text to add
5. Why It Helps: Which failure mode it prevents

Prioritization factors:

• How common is this pattern?
• Would future agents definitely hit this again?
• How hard is it to implement?

Pattern-to-Improvement Mapping (Failures/Slow Success):

Pattern	Likely Improvement
Wrong file entirely	Check /docs/design/ for component relationships
Tunnel vision	Instruction file: "Always search for pattern across codebase"
Missing platform knowledge	Platform-specific instruction file
Wrong abstraction layer	Reference /docs/design/HandlerResolution.md
Misread error message	Code comment explaining the real cause
Over-engineered	Skill enhancement: "Try simplest fix first"

Pattern-to-Improvement Mapping (Quick Success - reinforce):

Pattern	Improvement
Good search strategy	Document the search pattern that worked in skills
Documentation helped	Note which docs were valuable, ensure they stay updated
Recognized pattern	Add to instruction files as known pattern

Step 6: Present Findings

Present your analysis covering:

• What happened and what made it hard
• Where agent looked vs actual fix location
• Which patterns applied and evidence
• Prioritized recommendations with full details (category, priority, location, exact change, why it helps)

---

Error Handling

Situation	Action
PR not found	Ask user to verify PR number
No agent involvement evident	Ask user if they still want analysis
Can't determine failure mode	State "insufficient data" and what's missing

Constraints

• Analysis only - Don't apply changes (use learn-from-pr agent for that)
• Actionable recommendations - Every recommendation must have specific file path and text
• Don't duplicate - Check existing docs before recommending new ones
• Focus on high-value learning - Skip trivial observations
• Respect PR scope - Don't recommend improvements unrelated to the PR's learnings

---

Examples

Example: Wrong File Entirely

PR #33352 - TraitCollectionDidChange crash on MacCatalyst

What happened:

• Issue title: "ObjectDisposedException in ShellSectionRootRenderer"
• Agent made 11 attempts, ALL in ShellSectionRootRenderer.cs
• Actual fix was in PageViewController.cs

Failure Mode: Trusted issue title instead of searching for pattern.

Recommendation:

• Category: Instruction File
• Location: .github/instructions/ios-debugging.instructions.md
• Change: "When fixing iOS crashes, search for the PATTERN across all files, not just the file named in the error"
• Why: Prevents tunnel vision on named file

Example: Slow Success

PR #34567 - CollectionView scroll position not preserved

What happened:

• Agent took 5 attempts to find fix
• First 3 attempts were in wrong layer (handler vs core)
• Eventually found it after reading more context
• Final fix was simple once the right layer was identified

Pattern: Wrong abstraction layer - fixed handler when problem was in core.

Recommendation:

• Category: Architecture Doc
• Location: .github/architecture/handler-vs-core.md
• Change: Document layer responsibilities - handlers map properties, core handles behavior
• Why: Helps agent identify correct layer faster

Example: Quick Success

PR #35678 - Button disabled state not updating

What happened:

• Agent found fix in 1 attempt
• Searched for "IsEnabled" pattern across codebase immediately
• Found similar past fix in another control and applied same approach
• Simple, minimal change

Pattern: Good search strategy - recognized pattern from similar code.

Recommendation:

• Category: Skill Enhancement
• Location: .github/skills/try-fix/SKILL.md
• Change: Add to search strategy: "Search for same property pattern in other controls"
• Why: Reinforces successful discovery technique

---

Integration

• pr-finalize → Use first to verify PR is ready
• learn-from-pr skill → Analysis only (this skill)
• learn-from-pr agent → Analysis + apply changes

For automated application of recommendations, use the learn-from-pr agent instead.