Skip to content

ci: let rstack-ecosystem-ci run the E2E suites - #126

Merged
fi3ework merged 4 commits into
mainfrom
ci/eco-e2e
Nov 5, 2026
Merged

fi3ework merged 4 commits into
mainfrom
ci/eco-e2e

Conversation

@fi3ework

@fi3ework fi3ework commented Nov 5, 2026

Copy link
Copy Markdown
Member

Summary

rstack-ecosystem-ci runs only test:unit and lint for this repo today. Its root override — local builds of @rstest/core / @rslint/core injected into the consumer checkout — cannot reach the E2E fixtures: every fixture is its own pnpm workspace root and installs published packages from npm. This PR adds the harness support the eco-ci suites need to run the E2E slices against an unreleased toolchain.

 packages/vscode/e2e/
 ├── run.mjs                    # re-exec under `xvfb-run -a` on Linux, only for slices that launch VS Code
+├── vscodeTestCache.ts         # VSCODE_TEST_CACHE_PATH shared by every @vscode/test-electron entry
 ├── setupFixtures.mjs          # RSTACK_E2E_TOOLCHAIN_OVERRIDES: pack once, copy into each fixture
 ├── runTest.ts
 ├── rstest/runTest.ts
 └── lint/runTest.ts
  • RSTACK_E2E_TOOLCHAIN_OVERRIDES="name=<absolute package dir>,…" packs each package once into the first fixture's .toolchain/, copies the tarball into every fixture that consumes it, and writes a fixture-relative file: override into that fixture's pnpm-workspace.yaml. The relative spec is required: the dependency-recovery suites copy a fixture to tmpdir and reinstall with --frozen-lockfile. A later run with a different override set replaces the block an earlier run wrote.
  • Setup fails loudly when an overridden package still resolves from the registry (a local build and a published release can share a version number, which proves nothing) and when no fixture in the run used any override.
  • Rslint needs two entries, @rslint/core and the host @rslint/native-<tuple>: core resolves its binary through require.resolve('@rslint/native-<tuple>/bin'), so overriding the core alone leaves the published binary in place.
  • VSCODE_TEST_CACHE_PATH moves the VS Code download out of .vscode-test/. rstack-ecosystem-ci runs git clean -fdxq in the checkout before each suite, which would wipe it.
  • The Linux xvfb re-exec is gated on a selected slice that launches VS Code, so a plain-Node slice (smoke) starts no X server. This repo's own CI stays Windows + macOS; the Linux path runs only in rstack-ecosystem-ci.

Evidence

Before: eco-ci ran test:unit + lint only, so the E2E fixtures installed the published toolchain and never saw the unreleased build.

After (local, macOS, marker-patched tarballs that print a unique line when loaded):

  • pnpm lint → pass (204 files); pnpm test:unit → 384 passed.
  • VSCODE_CLI=1 node e2e/run.mjs rstest with the marker @rstest/core → exit 0; all 4 fixtures reported resolved through .toolchain (no registry version in pnpm-lock.yaml); 23 [PROBE-RSTEST-CORE] 0.12.5-probe.1 loaded lines from the test worker.
  • VSCODE_CLI=1 node e2e/run.mjs smoke lint with marker @rslint/core + @rslint/native-darwin-arm64 → exit 0; 12 fixture resolutions (6 fixtures × 2 packages); all 21 lint suites ran; [PROBE-RSLINT-CORE] 0.10.0-probe.1 loaded.
  • VSCODE_TEST_CACHE_PATH=<scratch dir> downloaded VS Code (339.67 MB) into that directory and the filtered lint suite passed; .vscode-test/ was not touched.

Merge Danger

Door: two-way

Reverting restores the previous harness behavior; the extension itself does not change at runtime.

Blast Radius: ecosystem-ci

This repo's CI is unchanged (Windows + macOS). The Linux/xvfb path is only exercised by rstack-ecosystem-ci, whose suite change follows separately.

Supersedes the closed #66; related to #40. The companion rstack-ecosystem-ci change will follow separately.

On Linux without DISPLAY, e2e/run.mjs re-execs itself under `xvfb-run -a`,
so `pnpm test:e2e` works unchanged on rstack-ecosystem-ci's ubuntu-latest
runners (#40). This repo's CI stays Windows + macOS.
RSTACK_E2E_TOOLCHAIN_OVERRIDES="name=<package dir>" packs each package into
<fixture>/.toolchain/ and adds a file: override to every fixture's pnpm-workspace.yaml,
so rstack-ecosystem-ci can run the suites against unreleased builds. Setup fails if the
lockfile still holds a registry version; the yaml is not restored (disposable checkouts).
The fixture pin in the setupFixtures.mjs header was two releases stale.
Also record that rslint needs the host native package beside the core:
core resolves its binary through require.resolve('@rslint/native-<tuple>/bin').
VSCODE_TEST_CACHE_PATH is forwarded to @vscode/test-electron by every
runTest.ts entry. rstack-ecosystem-ci runs git clean -fdxq in the checkout
before each suite, which would wipe .vscode-test/ between jobs.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-05T03:11:39.806932Z 6f15987 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@fi3ework
fi3ework merged commit 4a66ebd into main Nov 5, 2026
3 checks passed
@fi3ework
fi3ework deleted the ci/eco-e2e branch November 5, 2026 05:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant