Repository navigation
docs: troubleshoot Browser checks that fail intermittently - #818
Conversation
Add an "Intermittent failures" section to the Browser checks troubleshooting page: hard waits, load/networkidle with hanging third-party scripts, addLocatorHandler for optional popups, shared beforeAll contexts, strict mode and hidden duplicates, redirect races, browser launch errors, and location-specific failures. Stop recommending waitForLoadState() for page readiness in the timing accordion. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
- Browser check troubleshooting, Error Recovery: replace the try/catch pop-up snippet with page.addLocatorHandler(), as the new intermittent failures section recommends. - learn/playwright/selectors: use .first() only when the first match is the one you want; hidden copies make it pick an invisible element. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tested runtimeId 2025.04 with checkly test; the run log names the runtime. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hlenke
left a comment
There was a problem hiding this comment.
Ready to merge · Every runner claim and Playwright API in the new section checks out against the runner source and Playwright's docs. The four wording items below are optional.
Overlap. #813 also edits detect/synthetic-monitoring/browser-checks/troubleshooting.mdx. The hunks don't overlap: #813 adds a "Missing traces or videos" accordion after Performance and Timeout Issues, and this PR edits the Timing and Race Conditions list and appends a section at the end. Either order merges cleanly; I'd land #813 first. Once both land, the page says twice that pages from browser.newContext() have no video: #813's item 3 and line 179 here. Consider shortening line 179 to a link to that accordion, or keep it, since it explains the beforeAll case.
CI: all checks pass. Mergeable: clean. main has not changed the touched files since the merge base.
Inline: 4 wording.
What I checked
- Read the full diff from the merge base
e0e0277f, every line of both files, and the whole troubleshooting page andlearn/playwright/selectors.mdxat the PR head. - Local app unavailable (BUSY: another Tilt on the ports), so I verified against source instead of end-to-end runs. The PR description reports runs of every snippet with Playwright 1.58.2 and a production
--verboserun for the runtime log line. I didn't repeat those. - Runner settings in the intro (one worker, no Playwright retries, default test timeout, per-run browser,
retain-on-failurefor scheduled runs): monorepomainpackages/playwright-check-runner/src/browser/browser-runner.tsandpackages/check-runtimes-common/src/browser/getPlaywrightTestConfig.js. The test timeout is taken from the check'splaywrightConfig.timeoutwhen set, which is why I suggest mentioning it on line 94.playwright-support.mdxliststimeoutas a supported global option. - Playwright APIs:
page.addLocatorHandler()added in v1.42 and runs before actionability-checked actions and auto-waiting assertions (https://playwright.dev/docs/api/class-page#page-add-locator-handler).networkidleis DISCOURAGED and means no network connections for at least 500 ms (same page,waitUntil).locator.filter({ visible })added in v1.51 (https://playwright.dev/docs/release-notes).getByRoleexcludes hidden elements by default. - Runtime mapping:
platform/runtimes/runtime-specification.mdxlists Playwright 1.42.1 for 2024.02 and 1.51.1 for 2025.04. - Code samples: read every block for valid Playwright. No
awaiton locators, locator-based actions,@playwright/teststyle. Single quotes and no semicolons match the rest of the page. - Links and anchors:
#viewing-trace-files,#avoid-network-interception-in-synthetic-monitoring,#handling-multiple-matching-elements, and#overriding-runtimes-for-specific-checksexist onmain.scripts/pr_impact.py: no findings. - Not verified: the
browserType.launchcause on Checkly's runners, beyond the author's simulated run.
What changed and why
People often ask why a Browser check fails only sometimes, usually with a timeout. The troubleshooting page had one-line tips, and one of them (use
waitForLoadState()for page readiness) is part of the problem. Popups that appear only sometimes (page.addLocatorHandler()), sharedbeforeAllcontexts, redirect races and browser launch errors weren't covered anywhere.load/networkidlewith hanging third-party scripts, optional popups withpage.addLocatorHandler(), sharedbeforeAllcontexts, strict mode violations and hidden duplicates, redirects that race with the next step,browserType.launch: Target page, context or browser has been closed, and failures from one location only.learn/playwrightarticles instead of repeating them.runtimeIdandnpx checkly test --verbose, check the runtime in the run log, and compare traces. Links to the runtimes page for the web app.waitForLoadState()for page readiness; it points to a web-first assertion and the new section.try { click({ timeout: 5000 }) } catchis replaced withpage.addLocatorHandler(), as the new section recommends. The try/catch version fails when the popup appears after the catch window.learn/playwright/selectors.mdx, Position-Based Selection: the advice to add.first()"almost every time" is replaced. Use.first()only when you know the first match is the one you want; hidden copies make a CSS or text locator's.first()pick an invisible element. Otherwise make the locator more specific (example:getByRolewith a name).How each claim was verified
The Browser check runner can't run in an isolated local devenv. So each claim was tested with real Playwright 1.58.2, the version in runtime 2026.04, against small local test pages. The runs used the config Checkly's own code generates for a scheduled Browser check on monorepo
main(e086a77755):getPlaywrightTestConfig: workers 1, trace and videoretain-on-failure, default test timeout;browser-runner.ts: retries 0, global timeout 240 s.Every code sample in the PR was run as written, with only the URLs pointing at local pages.
workers: 1, notimeoutunless the check sets one) andbrowser-runner.ts(retries: 0,timeout: inputConfig.timeout). Runs time out at 30000 msplaywright testprocess in a new temp directory (browser-runner.ts)retain-on-failureExpected: "Ready"/Received: "Loading". A wait longer than the test timeout fails withpage.waitForTimeout: Test timeout of 30000ms exceeded.. ThetoHaveTextfix passesloadpage.gotofails withpage.goto: Test timeout of 30000ms exceeded.and the call log linenavigating to "…", waiting until "load". Run:waitForLoadState('load')reports"domcontentloaded" event firedbefore it times outnetworkidlenever settles on a polling pagepage.waitForLoadState: Test timeout of 30000ms exceeded.. 500 ms definition and "discouraged" from the Playwright docstrace.networkhas the script request with no response (status -1)waitUntil: 'domcontentloaded'+ assertion, orpage.route(...abort), fix itintercepts pointer eventslocator.click: Test timeout …with<div id="overlay">…</div> intercepts pointer eventsaddLocatorHandlerfixes it, and does nothing when no popupaddLocatorHandlerneeds Playwright 1.42 (runtime 2024.02+)beforeAllcontext leaks state between tests and has no video1 itemsinstead of0 items; the failed result has a trace but no video. With thepagefixture both tests pass and a failing test has a videoError: locator.click: Error: strict mode violation: locator('.submit') resolved to 2 elements:.first()can pick a hidden copyelement is not visiblegetByRole, scoping, andfilter({ visible: true })fix itgetByRoleignores the hidden duplicate.filter({ visible })needs Playwright 1.51, so runtime 2025.04 per the release notes and the runtime specification pagepage.gotostarted just before fails withpage.goto: net::ERR_ABORTED at …. A click just before leaves the next assertion failing. Whether it fails depends on timing: in a delay sweep, 950 ms passed and 990 ms failedtoHaveURL/waitForURLbefore the next step fixes itbrowserType.launch: Target page, context or browser has been closedhappens before test code runs, no videoRetryServicereschedules the whole check, and each run is a new browser process. Retries docsaddLocatorHandlersnippet closes the popup when it's there and does nothing when it isn't.optional-popupcovers the Log in button: popup shown before the action (handler closes it, click lands), no popup (click lands), popup appearing 6 s after load, past the old 5 s catch window (handler closes it, click lands)<div class="optional-popup">…</div> intercepts pointer events.first()on a CSS or text locator can pick a hidden copy and time outlocator('.submit').first().click()on a page with a hidden mobile-menu button first, andgetByText(…).first().click()on hidden calendar entries: both resolve to the hidden element and time out withelement is not visiblegetByRole('button').first()example and the newgetByRole('button', { name: 'Submit' })example work on that pagegetByRoleskips the hidden copy and clicks the visible button. The article's filtering approach (filter({ visible: true })) also picks the visible entryruntimeId; the run log names itruntimeId: '2025.04'andnpx checkly test --verbose. Log:Creating runtime version 2025.04 using Node.js 22; the script logged Playwright 1.51 and Chromium 134 (2025.04's versions). The same check on 2026.04 logsCreating runtime version 2026.04 using Node.js 24mint dev(both changed pages) and the docs checks (frontmatter, anchors, links, redirects, broken links) all passNot tested / left out
git apply --checkof docs: when Browser checks and Playwright Check Suites keep traces and videos #813's diff succeeds on top of this branch.communicate/alerts/retries.mdx, which this PR only links to.🤖 Generated with Claude Code