===============
Getting Started
===============

In this tutorial you will add two workflow files to a GitHub repository and
get a formatted benchmark comparison table posted on every pull request.

The end result is a PR comment like this
(`live
example from eOn <https://github.com/TheochemUI/eOn/pull/321#issuecomment-3979562277>`_):

.. raw:: html

   <details>
   <summary><strong>Example PR comment (click to expand)</strong></summary>
   <div style="border:1px solid var(--sd-color-muted); border-radius:6px; padding:16px; margin:8px 0; background:var(--sd-color-card-background, #fff);">
   <h3 style="margin-top:0;">Benchmark Results</h3>
   <div style="background:#dff6dd; border-left:4px solid #2da44e; padding:8px 12px; border-radius:4px; margin-bottom:12px;">
   <strong>1 benchmark(s) improved</strong>
   </div>
   <table style="width:100%; border-collapse:collapse; margin-bottom:12px;">
   <tr style="border-bottom:2px solid #d0d7de;">
     <th></th><th style="text-align:right;">Count</th>
   </tr>
   <tr><td>&#x1F7E2; Improved</td><td style="text-align:right;">1</td></tr>
   <tr><td>&#x26AA; Unchanged</td><td style="text-align:right;">7</td></tr>
   </table>
   <h4>Improvements</h4>
   <table style="width:100%; border-collapse:collapse; margin-bottom:12px;">
   <tr style="border-bottom:2px solid #d0d7de;">
     <th></th><th>Benchmark</th><th style="text-align:right;">Before</th>
     <th style="text-align:right;">After</th><th style="text-align:right;">Ratio</th>
   </tr>
   <tr>
     <td>&#x1F7E2;</td>
     <td><code>bench.TimeFoo.time_foo</code></td>
     <td style="text-align:right;">7.67&plusmn;0ms</td>
     <td style="text-align:right;">5.94&plusmn;0ms</td>
     <td style="text-align:right;">0.77x</td>
   </tr>
   </table>
   <details style="margin-bottom:8px;">
   <summary>7 unchanged benchmark(s)</summary>
   <table style="width:100%; border-collapse:collapse; margin-top:8px;">
   <tr style="border-bottom:2px solid #d0d7de;">
     <th>Benchmark</th><th style="text-align:right;">Before</th>
     <th style="text-align:right;">After</th><th style="text-align:right;">Ratio</th>
   </tr>
   <tr>
     <td><code>bench.TimeBar.time_bar</code></td>
     <td style="text-align:right;">38.2&plusmn;0ms</td>
     <td style="text-align:right;">35.6&plusmn;0ms</td>
     <td style="text-align:right;">~0.93x</td>
   </tr>
   <tr><td colspan="4" style="color:#666;">...</td></tr>
   </table>
   </details>
   <details style="margin-bottom:8px;">
   <summary>Details</summary>
   <ul style="margin-top:8px;">
     <li><strong>Base:</strong> <code>d7b3a604</code></li>
     <li><strong>Head:</strong> <code>b22af558</code></li>
     <li><strong>Runner:</strong> <code>ubuntu-22.04</code></li>
   </ul>
   </details>
   <details>
   <summary>Raw asv-spyglass output</summary>
   <pre style="margin-top:8px; font-size:0.85em; overflow-x:auto;">
   All benchmarks:

   | Change | Before   | After    | Ratio | Benchmark                       |
   |--------|----------|----------|-------|---------------------------------|
   |        | 38.2±0ms | 35.6±0ms |  0.93 | bench.TimeBar.time_bar          |
   | -      | 7.67±0ms | 5.94±0ms |  0.77 | bench.TimeFoo.time_foo          |
   </pre>
   </details>
   </div>
   </details>

Prerequisites
-------------

- A GitHub repository with ASV benchmarks (``asv.conf.json`` + ``benchmarks/``)

- ``uv`` available on CI runners (via ``astral-sh/setup-uv@v5``)

Step 1: Create the Benchmark Workflow
-------------------------------------

Create ``.github/workflows/benchmark.yml`` and paste this entire block:

.. code:: yaml

    name: Benchmark PR
    on:
      pull_request:
        branches: [main]

    jobs:
      # Run ASV for base and PR commits in parallel
      benchmark:
        strategy:
          fail-fast: false
          matrix:
            include:
              - label: main
                sha: ${{ github.event.pull_request.base.sha }}
              - label: pr
                sha: ${{ github.event.pull_request.head.sha }}
        runs-on: ubuntu-latest
        steps:
          - uses: actions/checkout@v4
            with:
              fetch-depth: 0  # full history needed for checkout

          # Stash benchmarks/ before checkout -- the PR branch may have
          # added or changed benchmark files that don't exist on main
          - name: Checkout target
            run: |
              mkdir -p /tmp/_bench_preserve
              cp -r benchmarks/ /tmp/_bench_preserve/
              git checkout -f ${{ matrix.sha }}
              git clean -fd
              rm -rf benchmarks
              cp -r /tmp/_bench_preserve/benchmarks/ benchmarks/

          - name: Run benchmarks
            run: |
              asv machine --yes
              asv run --record-samples --set-commit-hash ${{ matrix.sha }}

          - uses: actions/upload-artifact@v4
            with:
              name: bench-${{ matrix.label }}
              path: .asv/results/

      # Download both results, compare, bundle into one artifact
      combine:
        needs: benchmark
        runs-on: ubuntu-latest
        steps:
          - uses: astral-sh/setup-uv@v5

          - uses: actions/download-artifact@v4
            with:
              name: bench-main
              path: results/
          - uses: actions/download-artifact@v4
            with:
              name: bench-pr
              path: results/

          # Compare the two result files and write metadata
          - name: Compare
            run: |
              BASE_PREFIX="${{ github.event.pull_request.base.sha }}"
              PR_PREFIX="${{ github.event.pull_request.head.sha }}"
              BASE_FILE=$(find results -name "${BASE_PREFIX:0:8}*.json" | head -1)
              PR_FILE=$(find results -name "${PR_PREFIX:0:8}*.json" | head -1)
              uvx --from "git+https://github.com/airspeed-velocity/asv_spyglass.git" \
                asv-spyglass compare "$BASE_FILE" "$PR_FILE" \
                --label-before main --label-after pr > results/comparison.txt
              echo "main_sha=${{ github.event.pull_request.base.sha }}" > results/metadata.txt
              echo "pr_sha=${{ github.event.pull_request.head.sha }}" >> results/metadata.txt

          - uses: actions/upload-artifact@v4
            with:
              name: benchmark-results
              path: results/

Step 2: Create the Commenter Workflow
-------------------------------------

Create ``.github/workflows/benchmark_comment.yml`` and paste this block:

.. code:: yaml

    name: Comment benchmark results

    # Runs after the benchmark workflow finishes.
    # Separate workflow so fork PRs get write access via workflow_run.
    on:
      workflow_run:
        workflows: ["Benchmark PR"]  # must match the name: above exactly
        types: [completed]

    jobs:
      comment:
        if: >-
          github.event.workflow_run.event == 'pull_request' &&
          github.event.workflow_run.conclusion == 'success'
        runs-on: ubuntu-latest
        permissions:
          pull-requests: write   # post/update PR comments
          issues: write          # required for the comment API
          actions: read          # download artifacts from other runs

        steps:
          - name: Set up uv
            uses: astral-sh/setup-uv@v5

          - name: Download benchmark artifact
            uses: actions/download-artifact@v4
            with:
              name: benchmark-results
              path: results
              run-id: ${{ github.event.workflow_run.id }}
              github-token: ${{ secrets.GITHUB_TOKEN }}

          # This is the only step that uses asv-perch
          - name: Post benchmark comment
            uses: HaoZeke/asv-perch@v1
            with:
              github-token: ${{ secrets.GITHUB_TOKEN }}
              comparison-text-file: results/comparison.txt
              metadata-file: results/metadata.txt
              regression-threshold: '10'
              runner-info: ubuntu-latest

Step 3: Open a Pull Request
---------------------------

Commit both files, push the branch, and open a PR. The benchmark workflow
runs first. When it succeeds, the commenter workflow starts automatically
and posts the comparison table on the PR.

Push again to the same PR and the existing comment updates in place.

Verify It Works
---------------

After both workflows finish:

1. The PR has a comment with a benchmark comparison table.

2. The Actions tab step summary shows the same table.

3. The workflow logs show ``regression-detected: true`` or ``false``.

If no comment appears, check the commenter workflow logs. The three most
common issues:

- The ``workflows:`` trigger name does not match the benchmark workflow ``name:``.

- The benchmark workflow did not succeed.

- The artifact name does not match between upload and download.

Add a Badge
-----------

Show that your project tracks benchmarks with ``asv-perch``:

.. code:: markdown

    [![Benchmarks](https://img.shields.io/badge/benchmarks-asv--perch-orange)](https://github.com/HaoZeke/asv-perch)

Next Steps
----------

`Multi-way comparison <./multi_way_comparison.rst>`_
    compare a
    baseline against multiple build configs

`Cross-environment comparison <./cross_environment.rst>`_
    compare
    the same commit across different environments

`Pre-computed output <../howto/precomputed_output.rst>`_
    skip
    asv-spyglass entirely by handing the action a text file

`Configuration reference <../reference/configuration.rst>`_
    all
    inputs, outputs, and YAML pipeline options