Repository navigation
fix(success): keep GraphQL queries for associated PRs below the timeout - #1335
Open
barredterra wants to merge 2 commits into
Open
barredterra wants to merge 2 commits into
barredterra wants to merge 2 commits into
Conversation
For large releases, the success step sent one getAssociatedPRs query for each 100 commits, all at the same time. GitHub can fail these parallel queries with a timeout or a 502 error. Run the chunks one after another. Closes semantic-release#1017
A single getAssociatedPRs query for 100 commits can take longer than GitHub's GraphQL timeout of about 10 seconds, even when it runs alone. The cost grows with the number of PRs for each commit, because the query loads all PR fields for each link. On backport branches, one commit can link to many PRs. Load the PRs in two steps: 1. Get only the PR numbers for each commit (50 commits per query). 2. Remove duplicates and load the full PR fields for 10 PRs per query. Also load related issues 10 per query, because they use the same PR fields. This also fixes the pagination of associated PRs, which used an undefined variable.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1017
Problem
For large releases, the
successstep can fail with502 Bad Gatewayor504fromPOST /graphql. The release and the tag already exist at that point, so no comments are posted and the next run hides the failure.GitHub stops a GraphQL request after about 10 seconds (see #1017 (comment)). One
getAssociatedPRsquery asks for 100 commits ×associatedPullRequests(first: 100)with all PR fields. Some of these fields are expensive (mergeable,changedFiles,commits.totalCount). The cost grows with the number of PRs for each commit. On backport branches, one commit can link to many PRs.Measured on a public repository with a backport branch:
502/504after ~11 s, also when it runs alone200, ~10 s200, ~5 s: 1,890 links, but only 64 unique PRsThus, smaller chunks alone do not fix the problem. The same PR is loaded again for each commit that links to it.
Fix
Load the associated PRs in two steps:
buildRelatedIssuesQuery(10 PRs per query).Other changes:
inChunksruns the chunks one after another, not in parallel.issueobject forsuccessCommentConditionkeeps the same fields.associatedPullRequestsused an undefined variable (response). This is fixed.Result on 200 commits of the same backport branch: 4 number queries (max 3.2 s) and 8 detail queries for 73 unique PRs (max 1.8 s). All return
200.Tests
overwriteRoutes: trueand replaced the first mock. It now checks that a PR from the second page gets a comment.