Daily Perf Improver - Historical Performance Tracking CI Integration

[github-actions]

Summary

Integrates automated historical performance tracking into the CI pipeline, completing Medium Priority #4 from the performance status report. This enables long-term trend analysis, regression detection across releases, and data-driven optimization decisions.

Goal and Rationale

Performance Engineering Goal: Build historical performance tracking infrastructure for trend analysis and regression detection

Why it matters:

• Enables visualization of performance trends over weeks/months/releases
• Provides early warning system for performance regressions
• Documents performance milestones for release planning
• Supports data-driven optimization decisions with historical context

Approach

The historical tracking tools (track-performance.ts, analyze-performance.ts, visualize-performance.ts) already existed but were not integrated with CI. This PR completes the integration by:

1. CI Automation - Added http-benchmark-on-main job that runs on every main branch push
2. Data Persistence - Commits performance history to dedicated perf-history branch
3. Developer Tools - Added npm scripts for easy access to tracking tools
4. Documentation - Comprehensive usage guide with workflows and best practices

Impact

CI Integration

Main Branch Automation:

• Every commit to main automatically runs HTTP benchmarks
• Saves results with git metadata (commit SHA, branch, message)
• Commits to perf-history branch for permanent storage
• Zero overhead on PR workflow (only runs on main)

Data Structure:

{"timestamp":"2025-10-17T10:30:00Z","git":{"branch":"main","commit":"abc1234"},"results":{...}}

Developer Experience

New Package Scripts:

bun run perf:track       # Save benchmark results to history
bun run perf:analyze     # Analyze performance trends  
bun run perf:visualize   # Generate ASCII charts
bun run perf:html        # Create HTML visualizations

Example Workflow - Trend Analysis:

# View last 30 days of performance data
bun run perf:analyze --since=30d

# Visualize trends
bun run perf:visualize --since=30d --metric=overall

# Generate HTML report for release notes
bun run perf:html --output=release-performance.html

Implementation Details

CI Workflow Changes

Added new job in .github/workflows/ci.yml:

http-benchmark-on-main:
  name: 'HTTP Benchmark & Historical Tracking'
  runs-on: ubuntu-latest
  if: github.ref == 'refs/heads/main'
  permissions:
    contents: write  # Needed to commit historical data
  steps:
    - Run HTTP benchmark
    - Track historical performance
    - Commit to perf-history branch

Key Features:

• Only runs on main branch (not PRs)
• Uses fetch-depth: 0 for full git history
• Auto-creates perf-history branch on first run
• Uses [skip ci] in commit messages to prevent infinite loops

Package.json Scripts

{
  "scripts": {
    "perf:track": "cd benchmarks/http-server && bun run track-performance.ts",
    "perf:analyze": "cd benchmarks/http-server && bun run analyze-performance.ts",
    "perf:visualize": "cd benchmarks/http-server && bun run visualize-performance.ts",
    "perf:html": "cd benchmarks/http-server && bun run visualize-performance.ts --output=performance-chart.html"
  }
}

Documentation

Comprehensive Guide: benchmarks/http-server/README-HISTORICAL-TRACKING.md (400+ lines)

Covers:

• System overview and components
• Usage examples for all tools
• CI integration details
• Common workflows (local dev, release planning, regression investigation)
• Advanced filtering and output formats
• Troubleshooting guide
• Best practices

Quick Reference: Updated benchmarks/http-server/README.md with quick start examples

Validation

Testing

Local Testing:

# Verified tracking tools work correctly
cd benchmarks/http-server
bun run benchmark.ts
bun run track-performance.ts
bun run analyze-performance.ts
bun run visualize-performance.ts

# Confirmed data structure and git metadata capture
✅ All tools function correctly
✅ Git metadata extracted properly
✅ JSONL format validated

CI Workflow:

• ✅ Syntax validated
• ✅ Permissions configured correctly (contents: write)
• ✅ Branch conditions verified (if: github.ref == 'refs/heads/main')
• ✅ Fetch depth set appropriately
• ✅ Skip CI marker prevents loops

Example Analysis Output

📊 Summary Statistics (overall)
──────────────────────────────────────────────────
  Total entries: 15
  Mean: 125,432.50 req/s
  Median: 126,123.00 req/s
  Min: 118,234.00 req/s
  Max: 131,789.00 req/s
  Trend: 📈 Improving

📋 Recent Performance History
──────────────────────────────────────────────────
Date                Commit   Branch  Value       Change   Message
──────────────────────────────────────────────────
2025-10-17 10:30:00  abc1234  main    131,789.30  +5.1%   Optimize context
2025-10-16 15:20:00  def5678  main    125,456.20  +2.3%   JSX improvements

Trade-offs

Pros:

• Permanent Performance History - Never lose historical data
• Zero PR Overhead - Only runs on main, not PRs
• Easy Access - Simple npm scripts for all operations
• Comprehensive Docs - Clear workflows and examples
• Minimal Storage - ~1KB per commit, grows linearly

Cons:

• Main Branch Only - No historical tracking for feature branches (by design)
• Requires Main Push - First perf-history branch creation needs main push
• Minimal Overhead - Adds ~1-2 minutes to main CI (acceptable for value gained)

Future Enhancements

Potential improvements identified but not implemented:

1. Automatic Regression Alerts - Notify maintainers on significant degradations
2. Multi-Metric Dashboards - Combined view of all performance dimensions
3. Release Comparison Tool - Side-by-side performance across versions
4. Percentile Tracking - P50/P95/P99 latency history
5. Memory Tracking Integration - Combine with memory profiling tools

Related Work

• Completes Medium Priority Daily Perf Improver - Updates to complete configuration #4 from .github/copilot/instructions/performance-status-report.md
• Builds on existing HTTP benchmark infrastructure (statistical analysis, confidence intervals)
• Integrates with performance budget validation system
• Complements profiling tools and memory tracking

Usage After Merge

For Maintainers:

1. Merge this PR to main
2. First main push auto-creates perf-history branch
3. Every subsequent main push adds performance data
4. Use bun run perf:analyze to review trends monthly

For Contributors:

# Track local optimization experiments
bun run build
cd benchmarks/http-server
bun run benchmark.ts
bun run perf:track --tag="my-optimization-experiment"
bun run perf:analyze --limit=5

For Releases:

# Tag release milestones
bun run perf:track --tag="v4.10.0-release"

# Generate release performance report
bun run perf:analyze --since=30d --format=json > release-perf.json
bun run perf:html --since=30d

---

🤖 Generated with Claude Code

Co-Authored-By: Claude noreply@anthropic.com

AI generated by Daily Perf Improver

---

Note

This was originally intended as a pull request, but the git push operation failed.

Workflow Run: View run details and download patch artifact

The patch file is available as an artifact (aw.patch) in the workflow run linked above.
To apply the patch locally:

# Download the artifact from the workflow run https://github.com/githubnext/gh-aw-trial-hono/actions/runs/18583834816
# (Use GitHub MCP tools if gh CLI is not available)
gh run download 18583834816 -n aw.patch
# Apply the patch
git am aw.patch

Show patch preview (500 of 600 lines)

From a731dc3dbbb8e2574701063caf15723d2a9c564e Mon Sep 17 00:00:00 2001
From: Daily Perf Improver <github-actions[bot]@users.noreply.github.com>
Date: Fri, 17 Oct 2025 06:00:18 +0000
Subject: [PATCH] feat: integrate historical performance tracking with CI
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Add automated performance tracking infrastructure to CI pipeline for
long-term trend analysis and regression detection.

## Changes

### CI Integration
- Add http-benchmark-on-main job that runs on every main branch push
- Automatically tracks performance data with git metadata
- Commits historical data to perf-history branch for permanent storage
- Zero-overhead on PR workflow (only runs on main)

### Package Scripts
Add convenience commands for performance tracking tools:
- bun run perf:track - Save benchmark results to history
- bun run perf:analyze - Analyze performance trends
- bun run perf:visualize - Generate ASCII charts
- bun run perf:html - Create interactive HTML visualizations

### Documentation
- Comprehensive README-HISTORICAL-TRACKING.md with complete usage guide
- Updated main benchmark README with quick start examples
- Common workflows for local development and release planning
- Troubleshooting guide and best practices

## Impact

**For Main Branch:**
Every commit automatically collects performance data enabling:
- Long-term trend visualization across releases
- Early regression detection through historical comparison
- Performance milestone tracking for release planning
- Data-driven optimization decisions

**For Developers:**
Easy access to performance history via npm scripts:
- Quick trend analysis with bun run perf:analyze
- Visual charts with bun run perf:visualize
- HTML reports for documentation with bun run perf:html

## Implementation

Historical tracking tools already exist (track-performance.ts,
analyze-performance.ts, visualize-performance.ts) but were not
integrated with CI. This PR completes th
... (truncated)