Skip to content

docs: AEO metadata pass, llms.txt page refresh, content quality CI check - #861

Open
Dgiordano33 wants to merge 3 commits into
mainfrom
Dgiordano33/docs-aeo-metadata
Open

Dgiordano33 wants to merge 3 commits into
mainfrom
Dgiordano33/docs-aeo-metadata

Conversation

@Dgiordano33

@Dgiordano33 Dgiordano33 commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

Brings over the parts of an AEO/SEO pass from early September that were never merged. The original work sat uncommitted on an old local checkout, 1,020 commits behind main. Changes were ported field by field: a change only came over if main hadn't touched that field since.

What's in it

1. Content quality check (CI)

  • .github/scripts/check_content_quality.py checks frontmatter (missing title/sidebarTitle/description), descriptions over 160 characters, duplicate titles and descriptions, H1s in the page source, absolute internal links, code fences without a language, and pages missing from the navigation that aren't marked hidden or noindex.
  • .github/content-quality-baseline.json records the 137 existing violations, regenerated against current main plus this PR. They don't fail CI. Only new violations do.
  • A new step in static-docs-checks.yml runs it on every PR. Stdlib only, no install needed.

2. ai/llms-txt.mdx refresh
The page said /llms.txt lists every docs page, which is no longer true. It now covers /llms.txt, /docs/llms.txt, the /docs/_llms/<section>.txt section indexes, llms-full.txt, and per-page .md endpoints. I checked each claim against the live site today (all endpoints return 200; the header is X-Robots-Tag: noindex, follow).

3. docs.json
Adds a hidden, searchable group holding changelog/changelog. The changelog isn't in the navigation, so right now it's left out of search and llms.txt.

4. Page metadata (63 pages)

  • 47 descriptions rewritten. Most were vague ("Learn about…") or first-paragraph text that ran past 160 characters. I checked the specific claims against each page body. Five that overstated a page (webhooks, regions, maintenance windows, setup scripts, private-location traffic) were corrected so they only claim what the page says.
  • 12 missing sidebarTitles added.
  • 8 orphan pages marked hidden: true: comparison stubs, concepts/overview, integrations/alerts/overview, and the heartbeat configuration page. None of them are in the navigation, so this doesn't change rendering. It marks them as intentionally hidden.

Not included

  • 17 description changes where main has since rewritten the same field. I kept main's version.
  • 13 pages that main deleted or moved (orphan cleanup DOC-93 and others, snippet removal, guides restructure).
  • ~46 trailing-newline-only changes.
  • The npx mint a11y CI step. It fails on main today: brand color contrast (#0075FF is 4.21:1 against the light background; #031834 against the dark background), plus 5 missing alt attributes on video and img elements in playwright-support.mdx, visual-regressions.mdx, and index.mdx. Adding it as a blocking step would fail every PR. It should come back once the alt text is fixed and the color question has a decision.

Checks run locally

  • python3 .github/scripts/check_content_quality.py: pass (137 baseline, 0 new)
  • python3 .github/scripts/check_frontmatter.py: pass (673 files)
  • mint validate --no-build: no new issues. It does exit with a failure because of 2 warnings that already exist on main (react imports in snippets/copy-prompt-button.jsx and snippets/setup-flow.jsx).

🤖 Generated with Claude Code

https://claude.ai/code/session_018yu1ndW3a5Ap8jMTZhiWf6

Dgiordano33 and others added 3 commits November 18, 2026 19:55
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018yu1ndW3a5Ap8jMTZhiWf6
…rk orphan pages hidden

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018yu1ndW3a5Ap8jMTZhiWf6
@mintlify

mintlify Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
checkly-422f444a 🟢 Ready View Preview Nov 19, 2026, 12:56 AM

💡 Tip: Enable Automations to automatically generate PRs for you.

@hlenke hlenke left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs changes · Regenerate sitemap.xml so static-docs-checks passes, and key the content-quality baseline on something other than line numbers, because as written it fails unrelated PRs (for example #847) the moment they add a line above an existing violation.

The metadata pass is a clear improvement: most of the rewritten descriptions are tighter and match their pages. Three of them still promise more than the page says (inline below).

Merge readiness

  • CI: static-docs-checks fails (blocker). The generate-sitemap step reports sitemap.xml is out of date: adding changelog/changelog to the navigation (the hidden, searchable group) makes the generator emit https://www.checklyhq.com/docs/changelog/changelog/. Run npm run generate-sitemap and commit the result. The new content-quality step itself passed (137 accepted baseline violations, 0 new).
  • CI: preview-checks fails. 1 of the 5 Checkly checks against the preview failed. The same job also fails on #857, so it is probably not caused by this PR, but the log does not say which check failed; it's worth a look in the test session before merging.
  • Mergeable, and main has not moved since the merge base (84c88b6e).
  • Overlap. #774 (sentence case for titles) edits the title line right next to the description lines this PR changes on about a dozen of the same files (cli/importing, cli/installation, communicate/alerts/channels, communicate/status-pages/subscriber-notifications, the comparisons/*/overview pages, concepts/*), so whichever lands second will likely need a rebase. #830 (draft) touches the private-location pages, platform/dynamic-secret-scrubbing.mdx and platform/secrets.mdx. #849 also edits docs.json. #847 edits integrations/alerts/webhooks.mdx, which has three line-keyed baseline entries; see the blocker on the script.

Not on a specific line

  • Sibling now contradicts the refreshed page. ai/overview.mdx:77 still says "an llms.txt index lists them all". This PR's own description says /llms.txt no longer lists every page, and /docs/llms.txt links section indexes for some sections. Update that sentence in the same PR.
  • hidden: true is more than a marker. The PR description says it "doesn't change rendering", which is true, but Mintlify adds a noindex meta tag to hidden: true pages (Mintlify: hidden pages). Pages that are only missing from the navigation have no noindex. For these eight orphans that is probably what you want (none have inbound links from other pages), so this is just worth stating in the description.
  • Optional: comparisons/overview.mdx keeps the description 'Display inline code and code blocks', which looks copied from a component page. You're already touching that frontmatter, so you could fix it here.

Inline comments: 1 blocker, 5 important, 3 wording.

What I checked
  • Read the full diff from the merge base 84c88b6e4c49c45591d9063abd3f94711ce50261 to 1e615f2e: the script, workflow step, docs.json, ai/llms-txt.mdx, and all 65 frontmatter edits. I read the baseline JSON entries that are keyed by line.
  • gh pr checks 861; gh run view --log-failed for runs 44432904247 (sitemap step) and 44432904223 (npx checkly test against the preview: 1 failed, 4 passed); compared with gh pr checks 857.
  • gh pr view 861 --json mergeable,mergeStateStatus; git log 84c88b6e..origin/main (empty); gh pr list --json number,files for overlap.
  • pr_impact.py: no removed pages or anchors; communicate/alerts/channels and detect/assertions are hubs, but only their descriptions changed.
  • Script logic (read with git show, not run): the violation IDs for code-fence-language, source-h1, and absolute-internal-link embed line N. I compared the bare opening fences in integrations/alerts/webhooks.mdx at this PR's head (394, 400, 501, matching the baseline) with #847's head (398, 404, 505).
  • llms.txt claims: curl against production. /llms.txt is 200, a curated overview linking to /docs/llms.txt. /docs/llms.txt is 200; it lists Constructs, CLI Reference, Guides, and Comparisons pages inline and links _llms/docs.txt, _llms/api-reference.txt, _llms/learn.txt. /docs/llms-full.txt is 200 (about 3.4 MB). /docs/_llms/docs.txt is 200. /docs/detect/synthetic-monitoring/browser-checks/overview.md is 200 with x-robots-tag: noindex, follow. Mintlify's hidden-pages doc confirms hidden and searchable on groups.
  • Descriptions checked against page bodies: subscriber-notifications (email, RSS, Atom, Slack: OK), network-diagnostic-tools (OK), data-storage (retention table per plan: OK), allowlisting-traffic (OK), managing-false-positives (OK), secrets ("encrypted at rest and in flight": OK), heartbeat overview (backups: OK), playwright-reporter (uploads screenshots, videos, traces: OK), monitoring-an-openapi-spec (UI import: OK), api-checks/troubleshooting (OK), dns-monitors/configuration (OK), operators-metrics (Prometheus scrape: OK), cli/authentication (OK), setup-scripts-for-apis (OK), plus the four flagged inline. I read the remaining learn/ descriptions for accuracy against their titles and intros only.
  • Could not verify: whether noindex: true pages (as opposed to hidden) become searchable with searchable (the second half of the /docs/llms.txt table row); which preview check failed.

fence = re.match(r"^\s*```(.*)$", line)
if fence:
if not in_fence and not fence.group(1).strip():
add(violations, "code-fence-language", rel, f"line {line_number}")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocker · Violation IDs for code-fence-language, source-h1, and absolute-internal-link (lines 124, 131, 137) include the line number. So an existing, baselined violation counts as new as soon as anyone adds or removes a line above it, and CI fails on a PR that introduced nothing.

Concrete case: the baseline has code-fence-language|integrations/alerts/webhooks.mdx|line 394, 400, and 501. #847 adds lines above them, and on its head those same bare fences sit at 398, 404, and 505. Once this merges, #847 fails this check with three "new" violations. 99 of the 137 baseline entries are keyed by line, so this will happen on routine edits.

To fix it, drop the line number from the ID. Either compare counts per (rule, path) against the baseline and fail only when a count goes up, or key on content: the link destination for absolute-internal-link, the heading text for source-h1. Keep the line number in the printed message only.

BASELINE = ROOT / ".github" / "content-quality-baseline.json"
EXCLUDED_DIRS = {".git", "node_modules", "api-reference-old", "skills"}
EDITORIAL_EXCLUDED_PREFIXES = ("api-reference/", "snippets/")
INTERNAL_DOCS_HOSTS = ("checklyhq.com/docs", "www.checklyhq.com/docs", "docs.checklyhq.com")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important · The absolute-internal-link rule flags any URL under checklyhq.com/docs, including the .txt and .md resources that ai/llms-txt.mdx and index.mdx link to on purpose. This PR's own ai/llms-txt.mdx rewrite adds four of these (baseline lines 13, 14, 15, 37) and then puts them in the baseline. That goes against the script's own docstring (line 7: "Do not add a new baseline entry without reviewing the underlying page"). Either exempt non-page resources (paths ending in .txt, .md, .json) or make those links relative, then drop the entries from the baseline.

Comment thread ai/llms-txt.mdx
| Resource | Use it for |
| --- | --- |
| [`/llms.txt`](https://www.checklyhq.com/llms.txt) | A curated, site-wide overview of Checkly products, differentiators, documentation, guides, and AI resources. It links to the documentation index but does not list every documentation page. |
| [`/docs/llms.txt`](https://www.checklyhq.com/docs/llms.txt) | The generated documentation index. It links to a section index for each part of the site navigation, such as Docs, API Reference, and Learn. Hidden and noindex pages are excluded unless they are configured as searchable. |

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important · The live /docs/llms.txt does not link a section index for every part of the navigation. It lists the Constructs, CLI Reference, Guides, and Comparisons pages directly, and only Docs, API Reference, and Learn get a /_llms/ index.

Suggested change
| [`/docs/llms.txt`](https://www.checklyhq.com/docs/llms.txt) | The generated documentation index. It links to a section index for each part of the site navigation, such as Docs, API Reference, and Learn. Hidden and noindex pages are excluded unless they are configured as searchable. |
| [`/docs/llms.txt`](https://www.checklyhq.com/docs/llms.txt) | The generated documentation index. It lists some sections' pages directly and links a section index for the larger ones, such as Docs, API Reference, and Learn. Hidden and noindex pages are excluded unless they are configured as searchable. |

title: Performance & Web Vitals
sidebarTitle: Web Vitals
description: 'Web Vitals for Playwright Check Suites'
description: 'Collect Web Vitals and custom performance measurements from Playwright Check Suite runs in Checkly.'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important · The page has nothing on custom performance measurements; it covers the Web Vitals Checkly collects automatically and what caused a slow LCP or INP.

Suggested change
description: 'Collect Web Vitals and custom performance measurements from Playwright Check Suite runs in Checkly.'
description: 'Checkly collects Web Vitals for every page a Playwright Check Suite visits and shows what caused a slow LCP or INP.'

---
title: 'Browser check timeouts'
description: 'Browser check timeouts.'
description: 'Configure Playwright test and Checkly execution timeouts so browser checks fail at the intended threshold.'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important · The page's own table says the Checkly execution timeout (240 s) cannot be changed, so "Configure … Checkly execution timeouts" promises something the product doesn't do.

Suggested change
description: 'Configure Playwright test and Checkly execution timeouts so browser checks fail at the intended threshold.'
description: 'Understand the fixed 240-second Browser check limit and the Playwright timeouts you can change, and fix common timeout errors.'

title: 'Dynamic Secret Scrubbing'
sidebarTitle: 'Secret Scrubbing'
description: 'Scrub sensitive data from check results'
description: 'Configure Checkly to detect and redact sensitive values from check results, logs, and diagnostic assets.'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important · The description overstates the page in three ways. The page says values are scrubbed from logs and traces (and report trees), and that screenshots and videos are not scrubbed (line 37), so "diagnostic assets" is wrong. Scrubbing works only for Browser and Multistep checks (line 17). And there is nothing to configure in Checkly: you set CHECKLY_SECRET_* environment variables.

Suggested change
description: 'Configure Checkly to detect and redact sensitive values from check results, logs, and diagnostic assets.'
description: 'Scrub secrets fetched at runtime from the logs and traces of Browser and Multistep checks by storing them in CHECKLY_SECRET_ variables.'

Comment thread learn/kubernetes/klog.mdx
displayTitle: klog for structured logging
sidebarTitle: klog
description: Meet klog, the Kubernetes-specific logging library designed to provide structured and reliable logging across Kubernetes components. In this post, we’ll dive into klog’s purpose, key features, and how it’s transforming logging in Kubernetes.
description: Learn how Kubernetes components use klog and configure its structured logging, verbosity, and output options.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wording · The page doesn't cover verbosity or output options. It covers InfoS/ErrorS structured logging and JSON output.

Suggested change
description: Learn how Kubernetes components use klog and configure its structured logging, verbosity, and output options.
description: Learn how Kubernetes components use klog for structured logging with InfoS and ErrorS and JSON output.

Comment thread ai/llms-txt.mdx
Comment on lines +18 to +22
<Accordion title="Before you begin">

- [Changing your email or password in Checkly](https://checklyhq.com/docs/admin/changing-your-email-password.md): Learn how to change your email address or password in your Checkly account
- [Creating an API key in Checkly](https://checklyhq.com/docs/admin/creating-api-key.md): Learn how to create and manage user and service API keys for the Checkly API and CLI
- [Adding team members to your Checkly account](https://checklyhq.com/docs/admin/team-management/adding-team-members.md): Learn how to invite team members to join your Checkly account and manage team collaboration
- [Using Microsoft Entra ID for Single Sign-on in Checkly](https://checklyhq.com/docs/admin/team-management/microsoft-azure-ad.md): This page illustrates the standard procedure to follow in order to get started with Microsoft Entra ID SSO (formerly Azure AD) on Checkly.
- [Multi-Factor Authentication in Checkly](https://checklyhq.com/docs/admin/team-management/multi-factor-authentication.md): Learn how to set up and manage multi-factor authentication for enhanced account security
- [Admin Overview](https://checklyhq.com/docs/admin/team-management/overview.md): Comprehensive guide to managing your Checkly account, team members, security, and integrations
- [SAML for Single Sign-on in Checkly](https://checklyhq.com/docs/admin/team-management/saml.md): Learn how to configure SAML SSO integration with Checkly for your organization
- [Single Sign-on in Checkly](https://checklyhq.com/docs/admin/team-management/single-sign-on.md): Overview of Single Sign-On (SSO) options available in Checkly for enterprise security
- [Team management in Checkly](https://checklyhq.com/docs/admin/team-management.md): Manage your team and collaborate effectively in Checkly
```
Use an HTTP client such as `curl`. These public documentation resources do not require authentication.

</Accordion>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wording · An accordion around one sentence hides the only prerequisite (editing rules: no accordion for prerequisites by default). A plain sentence is enough.

Suggested change
<Accordion title="Before you begin">
- [Changing your email or password in Checkly](https://checklyhq.com/docs/admin/changing-your-email-password.md): Learn how to change your email address or password in your Checkly account
- [Creating an API key in Checkly](https://checklyhq.com/docs/admin/creating-api-key.md): Learn how to create and manage user and service API keys for the Checkly API and CLI
- [Adding team members to your Checkly account](https://checklyhq.com/docs/admin/team-management/adding-team-members.md): Learn how to invite team members to join your Checkly account and manage team collaboration
- [Using Microsoft Entra ID for Single Sign-on in Checkly](https://checklyhq.com/docs/admin/team-management/microsoft-azure-ad.md): This page illustrates the standard procedure to follow in order to get started with Microsoft Entra ID SSO (formerly Azure AD) on Checkly.
- [Multi-Factor Authentication in Checkly](https://checklyhq.com/docs/admin/team-management/multi-factor-authentication.md): Learn how to set up and manage multi-factor authentication for enhanced account security
- [Admin Overview](https://checklyhq.com/docs/admin/team-management/overview.md): Comprehensive guide to managing your Checkly account, team members, security, and integrations
- [SAML for Single Sign-on in Checkly](https://checklyhq.com/docs/admin/team-management/saml.md): Learn how to configure SAML SSO integration with Checkly for your organization
- [Single Sign-on in Checkly](https://checklyhq.com/docs/admin/team-management/single-sign-on.md): Overview of Single Sign-On (SSO) options available in Checkly for enterprise security
- [Team management in Checkly](https://checklyhq.com/docs/admin/team-management.md): Manage your team and collaborate effectively in Checkly
```
Use an HTTP client such as `curl`. These public documentation resources do not require authentication.
</Accordion>
These resources are public and need no authentication.

Comment thread ai/llms-txt.mdx
Use the `llms.txt` file to crawl and index the entire Checkly documentation. Every link in the file points to [the `.md` version of the page](/ai/markdown-access#md-endpoints), so you can fetch each URL directly to get the markdown content.
Fetch the documentation index first, follow a section index, then retrieve individual pages from it:

```bash

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wording · Shell blocks get the Terminal title.

Suggested change
```bash
```bash Terminal

This branch was successfully deployed

1 active deployment
staging — 1e615f2e Deployed Nov 19, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants