Skip to content

Add a github.submit_patch action that opens a pull request - #7475

Open
OrlaM wants to merge 3 commits into
mozilla:masterfrom
OrlaM:ao/bugbug-6/github-submit-patch
Open

OrlaM wants to merge 3 commits into
mozilla:masterfrom
OrlaM:ao/bugbug-6/github-submit-patch

Conversation

@OrlaM

@OrlaM OrlaM commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #7057.

Adds github.submit_patch, the GitHub equivalent of phabricator.submit_patch. The agent records the action during the run. When it is applied, hackbot-api pushes the run's commits to a hackbot/<run_id> branch of the agent's configured GitHub repository and opens a draft PR against its default branch. The PR's url goes in the action result, so later actions can reference it.

How it works

hackbot-api has no checkout or git binary, so it can't git push. This follows the Phabricator diff and the try push: the git work happens in the agent's container, and hackbot-api only calls the service's API.

  • Agent side: publish_changes writes changes/github_commits.json when a github.submit_patch is recorded, next to the existing changes artifacts. build_github_commits in changes.py builds it from the commits since the published base. It includes uncommitted work, through the same WIP commit the other artifacts use, and records for each commit its message, author and changed files.
  • Apply side: github_handler.SubmitPatchHandler replays those commits through GitHub's Git Data API (blobs, trees, commits, then a branch ref) and opens the draft PR. Commits keep their original author, and the GitHub App commits and signs them. The installation token is narrowed to the one repository, with contents: write and pull_requests: write.
  • Target repository: the artifact is written by the agent, so it never decides where hackbot writes. Each agent names its one repository in AgentSpec.github_repository (owner/repo). The handler refuses an agent without one, and an artifact whose repo_url names any other repository, before minting credentials. Every API call uses the configured name.
  • Workflow files: a run that changes anything under .github/workflows/ is refused before minting credentials. The branch lives in the repository itself, not a fork, so the PR's workflows would run with that repository's secrets.
  • Retries: a failed action can be applied again, so the handler moves the run's branch if it already exists and reuses a PR already open from it.
  • Patch types: github.submit_patch is added to hackbot-api's PATCH_ACTION_TYPES, so a run that leaves changes without submitting them is still flagged. The runtime gets its own GitHub PATCH_ACTION_TYPES, kept separate from the Phabricator one so that a GitHub submission doesn't also build a Phabricator diff.
  • Docs and UI: a catalog row and an explanation in docs/hackbot/actions.md, and a pull request preview (title and body) on the run page.

Tests

  • hackbot-api: pytest tests/ passes (533). The new test_github_handler.py drives githubkit against a fake api.github.com built on httpx.MockTransport. It covers:

    • the full flow, an empty body, and a retry that reuses the branch and PR;
    • an agent with no github_repository, an artifact naming another repository, and a case-only difference in the name;
    • workflow file changes, including deletions;
    • a non-GitHub repository, a missing artifact, GitHub rejecting a call, and the App not being configured.

    The refusal tests also check that no GitHub request is made.

  • hackbot-runtime: pytest tests/ passes (357). It covers the tool, build_github_commits against real git repos (multiple commits, uncommitted work, deletions, file modes, binaries, renames, paths with spaces, a commit that changes no files, no changes) and publish_changes gating.

  • pre-commit passes on the changed files.

This hasn't been run against a real GitHub repository yet.

Risks and follow-ups

  • No agent sets github_repository yet, so the action is refused for every agent until one is configured. bug-fix would also need prompt changes before using it.
  • github_repository repeats the repository in the agent's [source].repo_url. If the two drift apart, submissions fail with a mismatch error.
  • The PR always targets the default branch; there is no base-branch parameter.
  • Agents can't change GitHub workflow files through this action, by design.
  • Each changed file in each commit is a separate API call, made one at a time as GitHub recommends for creating content. Very large changes will be slow.
  • A path that isn't valid UTF-8 makes the artifact build fail. That gets logged and the artifact is skipped, so applying the action then fails.

@OrlaM
OrlaM marked this pull request as ready for review November 12, 2026 21:39
@OrlaM
OrlaM requested a review from a team as a code owner November 12, 2026 21:39
Copilot AI balanced review requested due to automatic review settings November 12, 2026 21:39

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The agent-writable artifact controls both the target repository and credential installation, allowing a compromised run to create branches and PRs in other repositories where the App is installed.

2 open issues
What changed in this PR

Adds github.submit_patch, the GitHub equivalent of Hackbot’s Phabricator submission action, addressing #7057. Agents record a submission during a run; hackbot-api later recreates the commits and opens a draft pull request.

Changes:

  • Adds the recording tool and GitHub-specific commit artifact generation.
  • Registers an apply-side handler that creates commits, manages the run’s branch, and reuses an existing PR on retries.
  • Adds runtime and API tests, action documentation, and a UI preview.
File Description
services/​hackbot-ui/​components/​RunDetail.tsx Previews the PR title and body.
services/​hackbot-api/​tests/​test_github_handler.py Tests GitHub API calls, retries, and failures.
services/​hackbot-api/​app/​action_handlers/​registry.py Registers the handler and patch action type.
services/​hackbot-api/​app/​action_handlers/​github_handler.py Recreates commits and opens draft PRs.
libs/​hackbot-runtime/​tests/​test_github_actions.py Tests action recording and tool exposure.
libs/​hackbot-runtime/​tests/​test_context.py Tests artifact publishing and commit naming.
libs/​hackbot-runtime/​tests/​test_changes.py Tests commit serialization and file changes.
libs/​hackbot-runtime/​hackbot_runtime/​context.py Publishes GitHub artifacts for recorded submissions.
libs/​hackbot-runtime/​hackbot_runtime/​changes.py Builds the GitHub commit payload.
libs/​hackbot-runtime/​hackbot_runtime/​actions/​github.py Defines the submission recording tool.
libs/​hackbot-runtime/​hackbot_runtime/​actions/​claude_sdk.py Exposes the tool through the Claude SDK integration.
libs/​hackbot-runtime/​hackbot_runtime/​actions/​__init__.py Exports GitHub actions.
docs/​hackbot/​actions.md Documents the action and submission flow.

Review effort: Balanced

The GitHub counterpart of phabricator.submit_patch. The agent records the
submission during the run; when it is applied, hackbot-api pushes a branch
carrying the run's commits to the source repository and opens a draft pull
request against its default branch.

hackbot-api has no git checkout to apply the run's patch in, so
publish_changes also captures each commit (message, author and changed
files) as changes/github_commits.json, from the published base. The handler
recreates those commits through GitHub's Git Data API on a hackbot/<run_id>
branch, using the GitHub App with a token narrowed to that repository.
Retrying reuses the branch and any pull request already open from it.

github.submit_patch counts as a patch action in hackbot-api, so a run that
leaves changes without one is still reported, and the run page previews the
pull request's title and body.

Fixes mozilla#7057
Drop the trailing NUL explicitly and pair headers with paths using
itertools.batched(strict=True), so output that doesn't hold one path
per entry raises instead of being silently mispaired. Handle a commit
that changes no files, where diff-tree prints nothing.
…changes

The target repository came from the agent-written commits artifact, so a
compromised run could open pull requests on any repository the GitHub App
is installed on. Each agent now names its one repository in
AgentSpec.github_repository. The handler refuses agents without one and
artifacts naming any other repository, before minting credentials, and
uses the configured name for every call.

Also refuse runs that change files under .github/workflows/. Pushing them
would need workflows: write, and since the branch lives in the repository
itself, the pull request's workflows would run with its secrets.
@OrlaM
OrlaM force-pushed the ao/bugbug-6/github-submit-patch branch from d1a6807 to f2d7761 Compare November 13, 2026 16:47

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a github.submit_patch action to open a PR

2 participants