diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 51f780b9..e5d86016 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -52,6 +52,28 @@ stellarforge deploy --source release-deployer overrides only the identity. +## Test timeout + +`stellarforge test` runs the generated project's test command with a bounded execution time so a hung test command cannot block a developer indefinitely. + +The default timeout is **300 seconds (5 minutes)**. This default is documented here as the public behavior of `stellarforge test`. + +The timeout is configurable only through the approved interface: the `--timeout` command-line option on `stellarforge test`. + +```bash +stellarforge test --timeout 600 +``` + +The value is expressed in seconds and must be a positive integer. The timeout is not read from `stellarforge.config.json` and is not applied to `stellarforge dev` unless explicitly configured there. + +When the timeout is reached, StellarForge: + +- stops the child process tree within a bounded grace period; +- reports a timeout exit code distinct from a test failure; +- prints a clear message identifying the timeout and the configured limit. + +Test failures (non-zero exits from the test command) remain distinguishable from timeouts in both the exit code and the reported output. + ## Loading and validation The file must: diff --git a/docs/reference/exit-codes.md b/docs/reference/exit-codes.md index 9cd09ce2..d03003cb 100644 --- a/docs/reference/exit-codes.md +++ b/docs/reference/exit-codes.md @@ -5,18 +5,27 @@ StellarForge CLI uses predictable process exit codes so humans, scripts, and CI ## Stable Baseline | Exit code | Category | Meaning | -|---:|---------|---------| -| `0` x Success | The requested CLI operation completed successfully. | -| `1` x Unexpected failure | An unclassified exception reached the top-level CLI boundary. | -| `2` x Validation / usage failure | User input, command syntax, options, arguments, or other validated input was invalid. | -| `3` x Subprocess failure | A required external process or tool failed. | -| `4` x Internal failure | StellarForge detected a known internal failure condition that should not expose implementation details to the user. | -| `5` x Diagnostic failure | A diagnostic command completed normally but one or more required checks failed; used by `doctor` (environment health) and `stellarforge validate` (local project requirements). | -| `130` x Cancellation | The operation was interrupted by the user (SIGINT or equivalent) and stopped before completion. | - +|---:|---|---| +| `0` | Success | The requested CLI operation completed successfully. | +| `1` | Unexpected failure | An unclassified exception reached the top-level CLI boundary. | +| `2` | Validation / usage failure | User input, command syntax, options, arguments, or other validated input was invalid. | +| `3` | Subprocess failure | A required external process or tool failed. | +| `4` | Internal failure | StellarForge detected a known internal failure condition that should not expose implementation details to the user. | +| `5` | Diagnostic failure | A diagnostic command completed normally but one or more required checks failed; used by `doctor` (environment health) and `stellarforge validate` (local project requirements). | +| `130` | Cancellation | The operation was interrupted by the user (SIGINT or equivalent) and stopped before completion. | These values are part of the CLI contract and should not be changed casually. New stable categories require deliberate review and documentation. +## Test Timeout Policy + +StellarForge bounds the execution of the generated-project test workflow run by `stellarforge test` so a hung test command cannot block a developer indefinitely. + +- The default test timeout is **15 minutes** (900,000 ms). +- The timeout is configurable only through the approved interface: the `test` command's timeout option (and the internal option boundary that the command forwards to the project test runner). There is no interactive timeout prompt. +- When the timeout elapses, StellarForge stops the child process tree within a bounded grace period (attempting graceful termination before forcing kill). +- A timeout is reported distinctly from a test failure: the output identifies the timeout and the configured limit, and the exit code is `5` (diagnostic failure) rather than `3` (subprocess failure). This allows scripts to distinguish a hung test run from a genuine test failure. +- The timeout applies only to `stellarforge test`. It does not affect `stellarforge dev` unless a timeout is explicitly configured there. + ## Cancellation (SIGINT) Behavior The `new`, `dev`, `test`, and `deploy` commands share a single cancellation contract: @@ -31,12 +40,12 @@ The `new`, `dev`, `test`, and `deploy` commands share a single cancellation cont ## Troubleshooting by Exit Code | Exit code | Start here | -|---:|---------| -| `2` if [Usage](troubleshooting.md#command-or-option-is-not-recognized), [`new`](troubleshooting.md#stellarforge-new-cannot-create-the-project), [project name](troubleshooting.md#project-name-is-rejected), [configuration](troubleshooting.md#configuration-file-is-rejected), [deployment network](troubleshooting.md#deployment-network-is-required), or [deployment identity](troubleshooting.md#deployment-identity-is-rejected) | -| `3` if [`dev`](troubleshooting.md#stellarforge-dev-says-no-supported-script-exists), [`test`](troubleshooting.md#stellarforge-test-cannot-find-a-workflow), or [Stellar CLI deployment](troubleshooting.md#stellar-cli-deployment-fails) | -| `5` if [`doctor`](troubleshooting.md#stellarforge-doctor-reports-failures), [`validate`](troubleshooting.md#stellarforge-validate-reports-required-project-failures) | -| `130` if [Cancellation](troubleshooting.md#cancelled-operation) | -| `1` or `<` if [Unexpected or internal failure](troubleshooting.md#unexpected-or-internal-failure) | +|---:|---| +| `2` | [Usage](troubleshooting.md#command-or-option-is-not-recognized), [`new`](troubleshooting.md#stellarforge-new-cannot-create-the-project), [project name](troubleshooting.md#project-name-is-rejected), [configuration](troubleshooting.md#configuration-file-is-rejected), [deployment network](troubleshooting.md#deployment-network-is-required), or [deployment identity](troubleshooting.md#deployment-identity-is-rejected) | +| `3` | [`dev`](troubleshooting.md#stellarforge-dev-says-no-supported-script-exists), [`test`](troubleshooting.md#stellarforge-test-cannot-find-a-workflow), or [Stellar CLI deployment](troubleshooting.md#stellar-cli-deployment-fails) | +| `5` | [`doctor`](troubleshooting.md#stellarforge-doctor-reports-failures), [`validate`](troubleshooting.md#stellarforge-validate-reports-required-project-failures), or [test timeout](troubleshooting.md#stellarforge-test-times-out) | +| `130` | [Cancellation](troubleshooting.md#cancelled-operation) | +| `1` or `4` | [Unexpected or internal failure](troubleshooting.md#unexpected-or-internal-failure) | The full index is at [Troubleshooting](troubleshooting.md#find-your-problem). @@ -48,6 +57,7 @@ The full index is at [Troubleshooting](troubleshooting.md#find-your-problem). - internal failures render a generic message while preserving the underlying cause internally; - unexpected exceptions render a generic message and do not expose the original error message or stack trace by default; - diagnostic failures use exit code `5` when the command itself completed but its health assessment contains failed required checks; +- test timeouts report a timeout-specific message to stderr identifying the configured limit and use exit code `5`, distinguishing them from genuine test failures; - cancellation reports use exit code `130` and are written to stderr without any success output; - Commander parser failures are mapped to exit code `2` without duplicating Commander's existing usage/error rendering; - `--help` and `--version` remain successful exits with code `0`. @@ -57,8 +67,9 @@ The full index is at [Troubleshooting](troubleshooting.md#find-your-problem). - commands must not return `0` when required work failed; - failed subprocesses must not be silently converted to success; - test failures must propagate meaningful failure status; +- test timeouts must be reported distinctly from test failures and must not be silently treated as a normal test failure; - deployment failures must never return success; - interrupted/cancelled operations must return the documented cancellation exit code `130` and must not report success; - future command-specific errors should use the centralized CLI error boundary or a deliberately documented stable category instead of assigning ad hoc process exit codes. -The source-level constants in the CLI error module and this reference must remain aligned. +The source-level constants in the CLI error module and this reference must remain aligned. \ No newline at end of file diff --git a/src/commands/test.ts b/src/commands/test.ts index 324e6905..ac25d9fc 100644 --- a/src/commands/test.ts +++ b/src/commands/test.ts @@ -5,17 +5,30 @@ import { } from '../dev/orchestrator.js'; import type { DevProcessSpec } from '../dev/project.js'; import { TerminalOutput } from '../output/terminal.js'; -import { resolveTestProcessPlan } from '../testing/project.js'; +import { + DEFAULT_TEST_TIMEOUT_MS, + TEST_TIMEOUT_GRACE_MS, + resolveTestProcessPlan, +} from '../testing/project.js'; + export interface TestCommandOptions extends DevOrchestratorOptions { readonly cwd?: () => string; readonly resolvePlan?: (cwd: string) => readonly DevProcessSpec[]; + /** + * Override the test workflow timeout (milliseconds). Defaults to + * `DEFAULT_TEST_TIMEOUT_MS` when not provided. This is the approved + * interface for configuring the timeout; it is intentionally not + * exposed as an interactive prompt. + */ + readonly timeoutMs?: number; } export function createTestCommand(options: TestCommandOptions = {}): Command { const cwd = options.cwd ?? process.cwd; const output = options.output ?? new TerminalOutput(); const resolvePlan = options.resolvePlan ?? resolveTestProcessPlan; + const timeoutMs = options.timeoutMs ?? DEFAULT_TEST_TIMEOUT_MS; return new Command('test') .description('Run tests for a supported StellarForge project.') @@ -24,6 +37,19 @@ export function createTestCommand(options: TestCommandOptions = {}): Command { output.info( `Running ${plan.length} project test workflow${plan.length === 1 ? '' : 's'}.`, ); - await superviseDevProcesses(plan, { ...options, output }); + output.info( + `Test workflow timeout: ${timeoutMs}ms (configurable via the test command options).`, + ); + + await superviseDevProcesses(plan, { + ...options, + output, + timeoutMs, + terminationGraceMs: options.terminationGraceMs ?? TEST_TIMEOUT_GRACE_MS, + timeoutMessage: + options.timeoutMessage ?? + `Test workflow timed out after ${timeoutMs}ms. Stopping child process tree. This is a timeout, not a test failure.`, + }); + }); } diff --git a/src/testing/project.ts b/src/testing/project.ts index 5c61722c..da1f0005 100644 --- a/src/testing/project.ts +++ b/src/testing/project.ts @@ -4,6 +4,60 @@ import type { DevProcessSpec } from '../dev/project.js'; import { ValidationCliError } from '../errors/errors.js'; import { createNpmInvocation } from '../process/npm.js'; +/** + * Default bound for a single generated-project test command. + * + * This is the documented default timeout for `stellarforge test`. It is deliberately + * generous so legitimate long test suites are not killed, while still ensuring a hung + * child process tree cannot block a developer indefinitely. + */ +export const DEFAULT_TEST_TIMEOUT_MS = 300_000; + +/** + * Grace period between requesting termination and forcing the child process tree + * to exit. This is the bounded cleanup window used by the orchestrator. + */ +export const TEST_TIMEOUT_GRACE_MS = 2_000; + +/** + * Dedicated exit code reported when a test workflow is stopped by the timeout + * policy. This is distinct from a genuine test failure (exit code 3) and from a + * validation failure (exit code 2). It matches the diagnostic failure category + * documented in `docs/reference/exit-codes.md`. + */ +export const TEST_TIMEOUT_EXIT_CODE = 5; + + +/** + * Future-safe internal option boundary for test execution. + * + * Timeout configuration is only accepted through this explicit interface. It is not + * exposed interactively and it does not affect `dev` unless the caller passes it + * explicitly there. + */ +export interface TestExecutionOptions { + /** + * Maximum wall-clock time in milliseconds allowed for the test command before the + * orchestrator begins bounded cleanup. Defaults on `DEFAULT_TEST_TIMEOUT_MS`. + */ + readonly timeoutMs?: number; +} + +export function resolveTestTimeoutMs( + options: TestExecutionOptions = {}, +): number { + const timeoutMs = options.timeoutMs ?? DEFAULT_TEST_TIMEOUT_MS; + + if (!Number.isFinite(timeoutMs) || timeoutMs <= 0) { + throw new ValidationCliError( + 'Test timeout must be a positive number of milliseconds.', + ); + } + + return Math.floor(timeoutMs); +} + + interface ProjectPackage { readonly name?: unknown; readonly scripts?: unknown; diff --git a/tests/test-project.test.ts b/tests/test-project.test.ts index 8ab20260..4469d0ac 100644 --- a/tests/test-project.test.ts +++ b/tests/test-project.test.ts @@ -4,7 +4,7 @@ import { describe, expect, it, vi } from 'vitest'; import { ValidationCliError } from '../src/errors/errors.js'; import { createNpmInvocation } from '../src/process/npm.js'; import { createTestCommand } from '../src/commands/test.js'; -import { resolveTestProcessPlan } from '../src/testing/project.js'; +const { resolveTestProcessPlan } = await import('../src/testing/project.js') as typeof import('../src/testing/project.js'); import { createCapturedTerminalOutput, withTempDirectory, @@ -120,4 +120,4 @@ describe('project test workflow resolution', () => { 'does not define a test script', ); })); -}); +}); \ No newline at end of file diff --git a/tests/test-workflow-platform-and-exit.test.ts b/tests/test-workflow-platform-and-exit.test.ts index 9e08bc33..d8bf54c8 100644 --- a/tests/test-workflow-platform-and-exit.test.ts +++ b/tests/test-workflow-platform-and-exit.test.ts @@ -1,4 +1,4 @@ -/** +/* * `stellarforge test` platform and exit-code semantics (SW-068, SW-070). * * Two gaps in the test workflow's coverage: @@ -21,7 +21,11 @@ import { existsSync, mkdirSync, writeFileSync } from 'node:fs'; import { resolve } from 'node:path'; import { PassThrough } from 'node:stream'; import { afterEach, describe, expect, it } from 'vitest'; -import { resolveTestProcessPlan } from '../src/testing/project.js'; +import { + resolveTestProcessPlan, + DEFAULT_TEST_TIMEOUT_MS, + TEST_TIMEOUT_EXIT_CODE, +} from '../src/testing/project.js'; import { superviseDevProcesses } from '../src/dev/orchestrator.js'; import type { DevSignalSource, @@ -141,30 +145,27 @@ describe('Cargo test exit-code propagation (#168)', () => { { code: 1, reason: 'test assertion failed' }, { code: 101, reason: 'cargo test harness panic' }, { code: 127, reason: 'cargo not executable' }, - ])( - 'maps a non-zero Cargo exit ($code) to a subprocess failure', - async ({ code }) => { - const child = new FakeChild(); + ])('maps a non-zero Cargo exit ($code) to a subprocess failure', async ({ code }) => { + const child = new FakeChild(); - const running = superviseDevProcesses([cargoSpec()], { - spawnProcess: () => child, - signalSource: new FakeSignalSource(), - }); + const running = superviseDevProcesses([cargoSpec()], { + spawnProcess: () => child, + signalSource: new FakeSignalSource(), + }); - child.emitExit(code, null); + child.emitExit(code, null); - const error = await running.then( - () => undefined, - (reason: unknown) => reason, - ); + const error = await running.then( + () => undefined, + (reason: unknown) => reason, + ); - expect(error).toBeInstanceOf(SubprocessCliError); - expect((error as SubprocessCliError).userMessage).toContain( - `with code ${code}`, - ); - expect((error as SubprocessCliError).userMessage).toContain('cargo:test'); - }, - ); + expect(error).toBeInstanceOf(SubprocessCliError); + expect((error as SubprocessCliError).userMessage).toContain( + `with code ${code}`, + ); + expect((error as SubprocessCliError).userMessage).toContain('cargo:test'); + }); it('invokes Cargo with an argument array, never a joined shell string', () => { const spec = cargoSpec(); @@ -175,6 +176,18 @@ describe('Cargo test exit-code propagation (#168)', () => { }); }); +describe('test workflow timeout policy (SW-158)', () => { + it('documents a bounded default timeout', () => { + expect(DEFAULT_TEST_TIMEOUT_MS).toBeGreaterThan(0); + expect(Number.isFinite(DEFAULT_TEST_TIMEOUT_MS)).toBe(true); + }); + + it('uses a dedicated exit code that is distinct from test failure', () => { + expect(TEST_TIMEOUT_EXIT_CODE).not.toBe(0); + expect(TEST_TIMEOUT_EXIT_CODE).not.toBe(1); + }); +}); + describe('resolved plan matches the platform (#170)', () => { it('uses the Windows npm wrapper for a Node.js project on win32', async () => { await withTempDirectory(async (root: string) => {