Skip to main content

The pull-request agent

The pull-request agent reviews a pull request when you ask for a review, answers questions about the pull request, and fixes its open feedback when you ask for a fix. It reads the change, the code around it, the issues the pull request closes, the comments so far and CI's results. It posts its review as a GitHub review on the pull request. It changes the pull request's code only when you ask it to fix the pull request.

Commands​

Mention @ai-auto-swe in a comment on the pull request or in a review thread, and write what you want in your own words. The agent decides which request in the table below the comment makes. It changes code only for the auto-swe:review-and-fix label, or for a comment that it classifies as a request for a fix with high confidence. When the confidence is lower, it answers instead. The answer says what it would change and how to ask for the fix with @ai-auto-swe fix. The agent acts only on a mention or a label from someone with write access to the repository, and ignores everyone else.

The agent reacts 👀 to your comment when it starts on it. When you comment while the agent works on an earlier request, your comment gets 👀 at once, and the agent starts on it after the earlier request. The 👀 comes before the agent checks your access, so a comment it then ignores also gets one.

You writeThe agent
The auto-swe:review label, added to a pull requestReviews the pull request, drafts included
@ai-auto-swe alone, or a request for a review, such as @ai-auto-swe review or @ai-auto-swe can you take a look?Reviews the pull request, drafts included
A request to fix, such as @ai-auto-swe fix, @ai-auto-swe fix this in a review thread, or @ai-auto-swe please address the nitsFixes the open feedback, or what you name, pushes its commits, and answers your comment with what it did
A request to review and fix, such as @ai-auto-swe review and fix or @ai-auto-swe review this and fix whatever you findFixes the open feedback, reviews the pull request and fixes its own findings, pushes its commits, and posts a review with each fix marked
The auto-swe:review-and-fix label, added to a pull requestThe same as @ai-auto-swe review and fix
@ai-auto-swe <question>, in a comment on the pull requestAnswers in a comment
@ai-auto-swe <question>, in a review threadAnswers in the thread
@ai-auto-swe feedback <text>Records your feedback and reacts 👍. It does not reply

The agent does not review a pull request on its own when you push. Ask again with @ai-auto-swe review after a change; the new review says what changed since the commit it reviewed last.

What a review looks like​

The review's body has, in order:

  • a tally, such as No blocking issues or 2 important;
  • up to five lines on what the agent checked and how, one of them on CI;
  • what the pull request misses or adds beyond its linked issues or its description, when there is something;
  • the findings and worth-a-look items that are not inline comments.

The review does not describe the change.

Each finding has a severity and a category:

SeverityMeaning
🔴 ImportantFix it before merging. It changes stored data, a report, an event, a response to a caller, whether a run succeeds or how long it takes, or it is a security defect
🟡 NitChanges none of those, such as wording, naming or dead code; the repository limits how many are posted
🟣 Pre-existingA bug in code the pull request touches but did not introduce

A finding on lines of the diff is an inline comment. The comment has:

  • a failure scenario;
  • the evidence, which is code the agent read or a command it ran;
  • sometimes, a suggested change in a code block.

The suggested change is a plain code block rather than a GitHub suggestion, because applying a GitHub suggestion creates a commit whose default message is not a Conventional Commit.

Any other finding is listed in the review's body. A review never approves or requests changes, so it does not block merging.

A 💭 Worth a look item is not a finding. It is one of:

  • a risk that depends on what the agent cannot read, such as live data, an external API's behavior or a person's decision. A failure the code shows is a finding, even when its size depends on such data;
  • a change with a concrete benefit, such as an existing helper the code duplicates.

Like a finding, it is an inline comment when it is about lines of the diff.

A defect with a failure scenario is a finding, even when its fix needs a choice between behaviors, such as failing or falling back.

Fixing​

A request to fix, such as @ai-auto-swe fix, fixes the pull request's open feedback. The open feedback is:

  • every finding and worth-a-look item in the agent's latest review, of any severity, that no later commit fixes;
  • every unresolved review thread in which a person asks for a change;
  • every comment on the pull request in which a person asks for a change that no later commit or reply settles.

The agent also fixes a worth-a-look item when the change is small and it is sure the change is right. It leaves a point that someone disputes or said to leave. Your comment can also narrow or add to what the agent fixes, for example @ai-auto-swe fix the nit on src/a.py too, and leave the rename.

Before it pushes, github_push looks for reviews that people submitted after the request arrived, and for comments they posted or edited after it, such as a reply saying to leave a point. The agent's prompt may not show them, or may show a comment's earlier text. When github_push finds one, it pushes nothing and returns those comments, with their threads, to the agent. The agent applies the same rules to them and calls github_push again with their ids.

A request to review and fix, such as @ai-auto-swe review and fix, fixes the open feedback. The agent then reviews the pull request as it does for a review request, and fixes that review's findings too. The auto-swe:review-and-fix label does the same.

When a finding has more than one reasonable fix, such as raising an error or returning a default, the agent does not choose. It lists the options and the one it recommends, and you answer with @ai-auto-swe fix <your choice>, for example in that finding's thread. When the repository's rules settle the choice, it fixes the finding and cites the rule.

When it fixes, the agent:

  1. edits the code, then runs the repository's formatter, linter, type checker and the tests of the changed code, as the repository's guidance names them for a commit. It does not run a check again after a timeout stops it, and its post names each check that did not run;
  2. commits its change in its sandbox, in one commit or several, and pushes the commits to the pull request's branch. Each pushed commit's author is the App, and its message follows the repository's commit message convention. CI runs on them as on any push;
  3. posts one GitHub review, so you get one notification:
    • fix answers your comment under a 🔧 Fix heading. The answer is a reply in your comment's review thread or, for a comment on the pull request, the review's text. It says what it fixed, what needs your choice, and what it did not fix and why;
    • a request to review and fix opens its review with a 🔧 Review and fix line on what it pushed, then marks each finding and each 💭 Worth a look item ✅ Fixed, with what changed, or ❓ Needs your call, with the options. For an item it did not fix, the review says why. The review's tally counts the fixed items separately from the open ones;
    • for both commands, the review holds a one-sentence reply, naming the commit, in each earlier review thread that a commit fixed. The agent does not reply in a thread it leaves as it was; it lists that thread in its summary instead. GitHub shows these replies in the review next to the code they are on. To answer one, reply in its original thread;
  4. resolves the review threads whose feedback is fixed, including the threads of its own ✅ Fixed comments and threads it had already answered. Unresolve a thread to dispute a fix.

It pushes nothing, and says why in its answer or review, when:

  • there is no open feedback;
  • a check fails on its change and it cannot fix the failure;
  • someone pushed to the branch after the agent's run started. Ask again;
  • its commits would together change more files than the repository's max_changed_files (20 by default), or would change a GitHub Actions workflow under .github/workflows/, a symbolic link or a submodule.

The agent never force-pushes and never pushes to another branch. It does not run on a pull request from a fork.

Turning it on for a repository​

An organization admin installs the GitHub App AI Auto SWE on the repository. No other setup is needed. The agent then answers commands on that repository's pull requests.

Tuning it​

These files in the repository change what the agent does. Each is optional.

  • REVIEW.md, at the repository root or in .auto-swe/: your review rules. They are added to the agent's default rules and take precedence where they conflict. Keep the file short. For example:

    # Review rules

    ## What counts as important here
    - A query not scoped to the caller's tenant.
    - A migration that is not backward compatible.

    ## Always check
    - A new API route has an integration test.

    ## Skip
    - Generated code under `src/gen/`.
  • .auto-swe/config.yaml: settings, under the agent's name. For example:

    pull-request:
    max_nits: 2 # nits posted per review; default 5
    max_pre_existing: 1 # default 2
    max_considerations: 0 # worth-a-look items; default 3
    skip_paths: ["**/*.lock", "src/gen/**"]
    max_changed_files: 10 # files a fix's commits may change; default 20
    egress: [nexus.sulzer-us.com] # hosts the sandbox can reach, besides the defaults

    Before it pushes, github_push checks each commit's message. It reads .pre-commit-config.yaml from the pull request's base branch, so a pull request cannot change its own check. When the file has hooks whose stages list commit-msg, github_push writes a pre-commit config with only those hooks to the sandbox, runs pre-commit install-hooks with it, then runs its hooks on the message. Hooks without stages are not run, because they also check the staged files. A local hook still runs the checkout's files, such as its commitlint config. When a hook fails, github_push pushes nothing and returns the hook's output to the agent, which rewords the commit. When install-hooks fails, the call fails and the agent does not reword the commit.

    A repository without such hooks can set commit_message_check, a shell command that github_push runs in the checkout's root with each message on stdin, in place of the hooks. commit_message_check_setup runs before it, such as a command that installs the check's tools, and fails the call when it exits nonzero.

    egress adds hosts that the agent's commands can reach, such as a package registry your repository installs from. The sandbox already reaches the hosts in sandbox.egress in the agent's agent.yaml, and the hosts of its credentials. egress accepts only the hosts in sandbox.approved_repo_egress in the same file. To add a host, open a pull request in ppm-data-platform that adds it there.

    Adding a registry that needs a credential, such as a private Nexus repository, does not give the sandbox that credential. Code that runs in the sandbox can send your repository's contents to each host you add; see What the sandbox can reach.

  • .auto-swe/setup.sh: a script that installs tools the agent needs, such as a JDK. It runs with bash before the agent starts, with the checkout's path as its argument. It runs again in later reviews until it succeeds, unless a run does not finish. It has 15 minutes and the same hosts as the agent. Install tools in it, and leave your dependencies to the agent: the agent reviews later pushes in the same sandbox, so dependencies the script installs are not updated for them. If the script fails or does not finish, the agent is told and reviews anyway, and the review says that the script failed.

  • CLAUDE.md, .claude/rules/ and .claude/skills/: your general instructions for coding agents. The agent reviews against them, and cites the rule a convention finding breaks.

The agent reads REVIEW.md, .auto-swe/config.yaml and .auto-swe/setup.sh from the pull request's base branch, so a change to any of them applies to reviews after it merges.

Limits​

  • The agent runs in a sandbox with Python, uv, Node, npm, git and the GitHub CLI. It also has Pants for a repository with a pants.toml. For a repository with a pnpm-lock.yaml or a packageManager field in package.json, it has corepack's pnpm and Yarn. It has no credentials for private package registries; see What the sandbox can reach.
  • It does not run your full test, lint or type-check suites. It reads CI's results instead, and says so when CI has not finished. When it fixes, it runs the checks only on the code it changed.
  • Each command the agent runs in the sandbox ends after 5 minutes.
  • After 30 minutes, the agent is told to finish and post what it has. Its last commands can run up to 15 minutes past that.
  • An answer to a question and a fix have the same model-call limit as a review.

Cost​

Each review ends with the number of model calls it made and an estimate of its cost in dollars. Every step run's trace in LangSmith shows the same figures; see Tokens and cost.

Giving feedback​

LangSmith records your feedback on the trace of the step run it is about.

  • 👍 or 👎 on any comment the agent posts. Each starts with both, so you can rate it in one click. The agent posts these comments:
    • an inline comment of a review
    • an answer in the conversation
    • a reply in a thread
  • A reply in the thread of one of the agent's comments. Mention @ai-auto-swe only if you want an answer.
  • @ai-auto-swe feedback <text> for anything else about the agent's last run.