Repository navigation
docs: AEO metadata pass, llms.txt page refresh, content quality CI check - #861
Dgiordano33 wants to merge 3 commits into
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018yu1ndW3a5Ap8jMTZhiWf6
…archable Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018yu1ndW3a5Ap8jMTZhiWf6
…rk orphan pages hidden Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018yu1ndW3a5Ap8jMTZhiWf6
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
hlenke
left a comment
There was a problem hiding this comment.
Needs changes · Regenerate sitemap.xml so static-docs-checks passes, and key the content-quality baseline on something other than line numbers, because as written it fails unrelated PRs (for example #847) the moment they add a line above an existing violation.
The metadata pass is a clear improvement: most of the rewritten descriptions are tighter and match their pages. Three of them still promise more than the page says (inline below).
Merge readiness
- CI:
static-docs-checksfails (blocker). Thegenerate-sitemapstep reportssitemap.xml is out of date: addingchangelog/changelogto the navigation (the hidden,searchablegroup) makes the generator emithttps://www.checklyhq.com/docs/changelog/changelog/. Runnpm run generate-sitemapand commit the result. The new content-quality step itself passed (137 accepted baseline violations, 0 new). - CI:
preview-checksfails. 1 of the 5 Checkly checks against the preview failed. The same job also fails on #857, so it is probably not caused by this PR, but the log does not say which check failed; it's worth a look in the test session before merging. - Mergeable, and
mainhas not moved since the merge base (84c88b6e). - Overlap. #774 (sentence case for titles) edits the
titleline right next to thedescriptionlines this PR changes on about a dozen of the same files (cli/importing,cli/installation,communicate/alerts/channels,communicate/status-pages/subscriber-notifications, thecomparisons/*/overviewpages,concepts/*), so whichever lands second will likely need a rebase. #830 (draft) touches the private-location pages,platform/dynamic-secret-scrubbing.mdxandplatform/secrets.mdx. #849 also editsdocs.json. #847 editsintegrations/alerts/webhooks.mdx, which has three line-keyed baseline entries; see the blocker on the script.
Not on a specific line
- Sibling now contradicts the refreshed page.
ai/overview.mdx:77still says "anllms.txtindex lists them all". This PR's own description says/llms.txtno longer lists every page, and/docs/llms.txtlinks section indexes for some sections. Update that sentence in the same PR. hidden: trueis more than a marker. The PR description says it "doesn't change rendering", which is true, but Mintlify adds anoindexmeta tag tohidden: truepages (Mintlify: hidden pages). Pages that are only missing from the navigation have nonoindex. For these eight orphans that is probably what you want (none have inbound links from other pages), so this is just worth stating in the description.- Optional:
comparisons/overview.mdxkeeps the description'Display inline code and code blocks', which looks copied from a component page. You're already touching that frontmatter, so you could fix it here.
Inline comments: 1 blocker, 5 important, 3 wording.
What I checked
- Read the full diff from the merge base
84c88b6e4c49c45591d9063abd3f94711ce50261to1e615f2e: the script, workflow step,docs.json,ai/llms-txt.mdx, and all 65 frontmatter edits. I read the baseline JSON entries that are keyed by line. gh pr checks 861;gh run view --log-failedfor runs 44432904247 (sitemap step) and 44432904223 (npx checkly testagainst the preview: 1 failed, 4 passed); compared withgh pr checks 857.gh pr view 861 --json mergeable,mergeStateStatus;git log 84c88b6e..origin/main(empty);gh pr list --json number,filesfor overlap.pr_impact.py: no removed pages or anchors;communicate/alerts/channelsanddetect/assertionsare hubs, but only their descriptions changed.- Script logic (read with
git show, not run): the violation IDs forcode-fence-language,source-h1, andabsolute-internal-linkembedline N. I compared the bare opening fences inintegrations/alerts/webhooks.mdxat this PR's head (394, 400, 501, matching the baseline) with #847's head (398, 404, 505). - llms.txt claims:
curlagainst production./llms.txtis 200, a curated overview linking to/docs/llms.txt./docs/llms.txtis 200; it lists Constructs, CLI Reference, Guides, and Comparisons pages inline and links_llms/docs.txt,_llms/api-reference.txt,_llms/learn.txt./docs/llms-full.txtis 200 (about 3.4 MB)./docs/_llms/docs.txtis 200./docs/detect/synthetic-monitoring/browser-checks/overview.mdis 200 withx-robots-tag: noindex, follow. Mintlify's hidden-pages doc confirmshiddenandsearchableon groups. - Descriptions checked against page bodies: subscriber-notifications (email, RSS, Atom, Slack: OK), network-diagnostic-tools (OK), data-storage (retention table per plan: OK), allowlisting-traffic (OK), managing-false-positives (OK), secrets ("encrypted at rest and in flight": OK), heartbeat overview (backups: OK), playwright-reporter (uploads screenshots, videos, traces: OK), monitoring-an-openapi-spec (UI import: OK), api-checks/troubleshooting (OK), dns-monitors/configuration (OK), operators-metrics (Prometheus scrape: OK), cli/authentication (OK), setup-scripts-for-apis (OK), plus the four flagged inline. I read the remaining
learn/descriptions for accuracy against their titles and intros only. - Could not verify: whether
noindex: truepages (as opposed tohidden) become searchable withsearchable(the second half of the/docs/llms.txttable row); which preview check failed.
| fence = re.match(r"^\s*```(.*)$", line) | ||
| if fence: | ||
| if not in_fence and not fence.group(1).strip(): | ||
| add(violations, "code-fence-language", rel, f"line {line_number}") |
There was a problem hiding this comment.
Blocker · Violation IDs for code-fence-language, source-h1, and absolute-internal-link (lines 124, 131, 137) include the line number. So an existing, baselined violation counts as new as soon as anyone adds or removes a line above it, and CI fails on a PR that introduced nothing.
Concrete case: the baseline has code-fence-language|integrations/alerts/webhooks.mdx|line 394, 400, and 501. #847 adds lines above them, and on its head those same bare fences sit at 398, 404, and 505. Once this merges, #847 fails this check with three "new" violations. 99 of the 137 baseline entries are keyed by line, so this will happen on routine edits.
To fix it, drop the line number from the ID. Either compare counts per (rule, path) against the baseline and fail only when a count goes up, or key on content: the link destination for absolute-internal-link, the heading text for source-h1. Keep the line number in the printed message only.
| BASELINE = ROOT / ".github" / "content-quality-baseline.json" | ||
| EXCLUDED_DIRS = {".git", "node_modules", "api-reference-old", "skills"} | ||
| EDITORIAL_EXCLUDED_PREFIXES = ("api-reference/", "snippets/") | ||
| INTERNAL_DOCS_HOSTS = ("checklyhq.com/docs", "www.checklyhq.com/docs", "docs.checklyhq.com") |
There was a problem hiding this comment.
Important · The absolute-internal-link rule flags any URL under checklyhq.com/docs, including the .txt and .md resources that ai/llms-txt.mdx and index.mdx link to on purpose. This PR's own ai/llms-txt.mdx rewrite adds four of these (baseline lines 13, 14, 15, 37) and then puts them in the baseline. That goes against the script's own docstring (line 7: "Do not add a new baseline entry without reviewing the underlying page"). Either exempt non-page resources (paths ending in .txt, .md, .json) or make those links relative, then drop the entries from the baseline.
| | Resource | Use it for | | ||
| | --- | --- | | ||
| | [`/llms.txt`](https://www.checklyhq.com/llms.txt) | A curated, site-wide overview of Checkly products, differentiators, documentation, guides, and AI resources. It links to the documentation index but does not list every documentation page. | | ||
| | [`/docs/llms.txt`](https://www.checklyhq.com/docs/llms.txt) | The generated documentation index. It links to a section index for each part of the site navigation, such as Docs, API Reference, and Learn. Hidden and noindex pages are excluded unless they are configured as searchable. | |
There was a problem hiding this comment.
Important · The live /docs/llms.txt does not link a section index for every part of the navigation. It lists the Constructs, CLI Reference, Guides, and Comparisons pages directly, and only Docs, API Reference, and Learn get a /_llms/ index.
| | [`/docs/llms.txt`](https://www.checklyhq.com/docs/llms.txt) | The generated documentation index. It links to a section index for each part of the site navigation, such as Docs, API Reference, and Learn. Hidden and noindex pages are excluded unless they are configured as searchable. | | |
| | [`/docs/llms.txt`](https://www.checklyhq.com/docs/llms.txt) | The generated documentation index. It lists some sections' pages directly and links a section index for the larger ones, such as Docs, API Reference, and Learn. Hidden and noindex pages are excluded unless they are configured as searchable. | |
| title: Performance & Web Vitals | ||
| sidebarTitle: Web Vitals | ||
| description: 'Web Vitals for Playwright Check Suites' | ||
| description: 'Collect Web Vitals and custom performance measurements from Playwright Check Suite runs in Checkly.' |
There was a problem hiding this comment.
Important · The page has nothing on custom performance measurements; it covers the Web Vitals Checkly collects automatically and what caused a slow LCP or INP.
| description: 'Collect Web Vitals and custom performance measurements from Playwright Check Suite runs in Checkly.' | |
| description: 'Checkly collects Web Vitals for every page a Playwright Check Suite visits and shows what caused a slow LCP or INP.' |
| --- | ||
| title: 'Browser check timeouts' | ||
| description: 'Browser check timeouts.' | ||
| description: 'Configure Playwright test and Checkly execution timeouts so browser checks fail at the intended threshold.' |
There was a problem hiding this comment.
Important · The page's own table says the Checkly execution timeout (240 s) cannot be changed, so "Configure … Checkly execution timeouts" promises something the product doesn't do.
| description: 'Configure Playwright test and Checkly execution timeouts so browser checks fail at the intended threshold.' | |
| description: 'Understand the fixed 240-second Browser check limit and the Playwright timeouts you can change, and fix common timeout errors.' |
| title: 'Dynamic Secret Scrubbing' | ||
| sidebarTitle: 'Secret Scrubbing' | ||
| description: 'Scrub sensitive data from check results' | ||
| description: 'Configure Checkly to detect and redact sensitive values from check results, logs, and diagnostic assets.' |
There was a problem hiding this comment.
Important · The description overstates the page in three ways. The page says values are scrubbed from logs and traces (and report trees), and that screenshots and videos are not scrubbed (line 37), so "diagnostic assets" is wrong. Scrubbing works only for Browser and Multistep checks (line 17). And there is nothing to configure in Checkly: you set CHECKLY_SECRET_* environment variables.
| description: 'Configure Checkly to detect and redact sensitive values from check results, logs, and diagnostic assets.' | |
| description: 'Scrub secrets fetched at runtime from the logs and traces of Browser and Multistep checks by storing them in CHECKLY_SECRET_ variables.' |
| displayTitle: klog for structured logging | ||
| sidebarTitle: klog | ||
| description: Meet klog, the Kubernetes-specific logging library designed to provide structured and reliable logging across Kubernetes components. In this post, we’ll dive into klog’s purpose, key features, and how it’s transforming logging in Kubernetes. | ||
| description: Learn how Kubernetes components use klog and configure its structured logging, verbosity, and output options. |
There was a problem hiding this comment.
Wording · The page doesn't cover verbosity or output options. It covers InfoS/ErrorS structured logging and JSON output.
| description: Learn how Kubernetes components use klog and configure its structured logging, verbosity, and output options. | |
| description: Learn how Kubernetes components use klog for structured logging with InfoS and ErrorS and JSON output. |
| <Accordion title="Before you begin"> | ||
|
|
||
| - [Changing your email or password in Checkly](https://checklyhq.com/docs/admin/changing-your-email-password.md): Learn how to change your email address or password in your Checkly account | ||
| - [Creating an API key in Checkly](https://checklyhq.com/docs/admin/creating-api-key.md): Learn how to create and manage user and service API keys for the Checkly API and CLI | ||
| - [Adding team members to your Checkly account](https://checklyhq.com/docs/admin/team-management/adding-team-members.md): Learn how to invite team members to join your Checkly account and manage team collaboration | ||
| - [Using Microsoft Entra ID for Single Sign-on in Checkly](https://checklyhq.com/docs/admin/team-management/microsoft-azure-ad.md): This page illustrates the standard procedure to follow in order to get started with Microsoft Entra ID SSO (formerly Azure AD) on Checkly. | ||
| - [Multi-Factor Authentication in Checkly](https://checklyhq.com/docs/admin/team-management/multi-factor-authentication.md): Learn how to set up and manage multi-factor authentication for enhanced account security | ||
| - [Admin Overview](https://checklyhq.com/docs/admin/team-management/overview.md): Comprehensive guide to managing your Checkly account, team members, security, and integrations | ||
| - [SAML for Single Sign-on in Checkly](https://checklyhq.com/docs/admin/team-management/saml.md): Learn how to configure SAML SSO integration with Checkly for your organization | ||
| - [Single Sign-on in Checkly](https://checklyhq.com/docs/admin/team-management/single-sign-on.md): Overview of Single Sign-On (SSO) options available in Checkly for enterprise security | ||
| - [Team management in Checkly](https://checklyhq.com/docs/admin/team-management.md): Manage your team and collaborate effectively in Checkly | ||
| ``` | ||
| Use an HTTP client such as `curl`. These public documentation resources do not require authentication. | ||
|
|
||
| </Accordion> |
There was a problem hiding this comment.
Wording · An accordion around one sentence hides the only prerequisite (editing rules: no accordion for prerequisites by default). A plain sentence is enough.
| <Accordion title="Before you begin"> | |
| - [Changing your email or password in Checkly](https://checklyhq.com/docs/admin/changing-your-email-password.md): Learn how to change your email address or password in your Checkly account | |
| - [Creating an API key in Checkly](https://checklyhq.com/docs/admin/creating-api-key.md): Learn how to create and manage user and service API keys for the Checkly API and CLI | |
| - [Adding team members to your Checkly account](https://checklyhq.com/docs/admin/team-management/adding-team-members.md): Learn how to invite team members to join your Checkly account and manage team collaboration | |
| - [Using Microsoft Entra ID for Single Sign-on in Checkly](https://checklyhq.com/docs/admin/team-management/microsoft-azure-ad.md): This page illustrates the standard procedure to follow in order to get started with Microsoft Entra ID SSO (formerly Azure AD) on Checkly. | |
| - [Multi-Factor Authentication in Checkly](https://checklyhq.com/docs/admin/team-management/multi-factor-authentication.md): Learn how to set up and manage multi-factor authentication for enhanced account security | |
| - [Admin Overview](https://checklyhq.com/docs/admin/team-management/overview.md): Comprehensive guide to managing your Checkly account, team members, security, and integrations | |
| - [SAML for Single Sign-on in Checkly](https://checklyhq.com/docs/admin/team-management/saml.md): Learn how to configure SAML SSO integration with Checkly for your organization | |
| - [Single Sign-on in Checkly](https://checklyhq.com/docs/admin/team-management/single-sign-on.md): Overview of Single Sign-On (SSO) options available in Checkly for enterprise security | |
| - [Team management in Checkly](https://checklyhq.com/docs/admin/team-management.md): Manage your team and collaborate effectively in Checkly | |
| ``` | |
| Use an HTTP client such as `curl`. These public documentation resources do not require authentication. | |
| </Accordion> | |
| These resources are public and need no authentication. |
| Use the `llms.txt` file to crawl and index the entire Checkly documentation. Every link in the file points to [the `.md` version of the page](/ai/markdown-access#md-endpoints), so you can fetch each URL directly to get the markdown content. | ||
| Fetch the documentation index first, follow a section index, then retrieve individual pages from it: | ||
|
|
||
| ```bash |
There was a problem hiding this comment.
Wording · Shell blocks get the Terminal title.
| ```bash | |
| ```bash Terminal |
Brings over the parts of an AEO/SEO pass from early September that were never merged. The original work sat uncommitted on an old local checkout, 1,020 commits behind
main. Changes were ported field by field: a change only came over ifmainhadn't touched that field since.What's in it
1. Content quality check (CI)
.github/scripts/check_content_quality.pychecks frontmatter (missingtitle/sidebarTitle/description), descriptions over 160 characters, duplicate titles and descriptions, H1s in the page source, absolute internal links, code fences without a language, and pages missing from the navigation that aren't markedhiddenornoindex..github/content-quality-baseline.jsonrecords the 137 existing violations, regenerated against currentmainplus this PR. They don't fail CI. Only new violations do.static-docs-checks.ymlruns it on every PR. Stdlib only, no install needed.2.
ai/llms-txt.mdxrefreshThe page said
/llms.txtlists every docs page, which is no longer true. It now covers/llms.txt,/docs/llms.txt, the/docs/_llms/<section>.txtsection indexes,llms-full.txt, and per-page.mdendpoints. I checked each claim against the live site today (all endpoints return 200; the header isX-Robots-Tag: noindex, follow).3.
docs.jsonAdds a hidden,
searchablegroup holdingchangelog/changelog. The changelog isn't in the navigation, so right now it's left out of search and llms.txt.4. Page metadata (63 pages)
sidebarTitles added.hidden: true: comparison stubs,concepts/overview,integrations/alerts/overview, and the heartbeat configuration page. None of them are in the navigation, so this doesn't change rendering. It marks them as intentionally hidden.Not included
mainhas since rewritten the same field. I keptmain's version.maindeleted or moved (orphan cleanup DOC-93 and others, snippet removal, guides restructure).npx mint a11yCI step. It fails onmaintoday: brand color contrast (#0075FFis 4.21:1 against the light background;#031834against the dark background), plus 5 missing alt attributes on video and img elements inplaywright-support.mdx,visual-regressions.mdx, andindex.mdx. Adding it as a blocking step would fail every PR. It should come back once the alt text is fixed and the color question has a decision.Checks run locally
python3 .github/scripts/check_content_quality.py: pass (137 baseline, 0 new)python3 .github/scripts/check_frontmatter.py: pass (673 files)mint validate --no-build: no new issues. It does exit with a failure because of 2 warnings that already exist onmain(reactimports insnippets/copy-prompt-button.jsxandsnippets/setup-flow.jsx).🤖 Generated with Claude Code
https://claude.ai/code/session_018yu1ndW3a5Ap8jMTZhiWf6