Add --priority for maestro, espresso and xcuitest

CHANGELOG.md

@@ -15,6 +15,8 @@ All notable changes to `@testingbot/cli` are documented here. Releases are publi
 - `--timeout <minutes>` for `testingbot maestro`, `espresso` and `xcuitest`: how long to wait for results, counted from when the tests start and shared by every `--retry` attempt. Default 60.
 - `--fail-fast` for `testingbot maestro`, `espresso` and `xcuitest`: at the first failure that will not be retried, the runs still queued or running are cancelled. The command waits for them to stop, then reports as usual and fails with exit code 2.
 
+- `--priority high|normal|low` for `testingbot maestro`, `espresso` and `xcuitest`: where the runs go in your own queue while they wait for a free parallel slot. High starts before normal, normal before low, then oldest first. It does not take slots from other accounts or raise your parallel limit. Needs the matching API change; older API versions ignore it.
+
 ### Changed
 
 - When the wait for results times out (after `--timeout`, or the default 60 minutes), the CLI now cancels the runs that are still queued or running, prints the results so far and exits with code 2 (`outcome: failed` with an `error` in `--json`). Before, it exited with code 1 and left the runs going. `testingbot status --wait` still only stops watching and leaves the runs alone.

README.md

@@ -189,6 +189,7 @@ Values passed with `-e` are stored in plain text: they are returned in the run's
 | `--retry <count>`         | Retry failed flows up to N times (0-2, default 0). Re-runs only the flows (or shards) that failed, the moment they fail, while the rest of the run continues. Cannot be combined with `--async`.                                                                         |
 | `--timeout <minutes>`     | Stop waiting for results after N minutes (default 60), counted from when the tests start and including `--retry` attempts. Unfinished runs are stopped and the command fails with exit code 2. Cannot be combined with `--async`.                                        |
 | `--fail-fast`             | Stop at the first failure that will not be retried: the runs still queued or running are cancelled, and the summary, reports and artifacts cover what ran. With `--retry`, a test only counts as failed once its retries are used up. Cannot be combined with `--async`. |
+| `--priority <level>`      | Queue priority among your own runs waiting for a free parallel slot: `high`, `normal` (default) or `low`. Higher priority starts first, then the oldest run. It does not take slots from other accounts or raise your parallel limit.                                    |
 | `--ignore-checksum-check` | Skip checksum verification and always upload the app                                                                                                                                                                                                                     |
 
 > **Note on `--retry`:** a failed flow/shard is retried immediately — as soon as it fails — without waiting for the other flows in the run to finish. Retry attempts appear live in the flow table marked with a `↻` icon. Each flow is retried independently up to N times, stopping as soon as that flow passes. Pass/fail uses the result of the **last** attempt per flow (last-attempt-wins), consistently across the CLI exit code, the TestingBot dashboard, and reports downloaded via `--report`.
@@ -536,6 +537,7 @@ testingbot espresso [appFile] [testAppFile] [options]
 | `--retry <count>`               | Run a failed test again up to N times (0-2). The last attempt decides the result, the report and the artifacts                                                                                                                                                           |
 | `--timeout <minutes>`           | Stop waiting for results after N minutes (default 60), counted from when the tests start and including `--retry` attempts. Unfinished runs are stopped and the command fails with exit code 2. Cannot be combined with `--async`.                                        |
 | `--fail-fast`                   | Stop at the first failure that will not be retried: the runs still queued or running are cancelled, and the summary, reports and artifacts cover what ran. With `--retry`, a test only counts as failed once its retries are used up. Cannot be combined with `--async`. |
+| `--priority <level>`            | Queue priority among your own runs waiting for a free parallel slot: `high`, `normal` (default) or `low`. Higher priority starts first, then the oldest run. It does not take slots from other accounts or raise your parallel limit.                                    |
 | `--download-artifacts [mode]`   | Download video, logs and screenshots as a zip after completion: `all` (default) or `failed`                                                                                                                                                                              |
 | `--artifacts-output-dir <path>` | Directory for the artifacts zip (defaults to the current directory)                                                                                                                                                                                                      |
 
@@ -645,6 +647,7 @@ testingbot xcuitest [appFile] [testAppFile] [options]
 | `--retry <count>`               | Run a failed test again up to N times (0-2). The last attempt decides the result, the report and the artifacts                                                                                                                                                           |
 | `--timeout <minutes>`           | Stop waiting for results after N minutes (default 60), counted from when the tests start and including `--retry` attempts. Unfinished runs are stopped and the command fails with exit code 2. Cannot be combined with `--async`.                                        |
 | `--fail-fast`                   | Stop at the first failure that will not be retried: the runs still queued or running are cancelled, and the summary, reports and artifacts cover what ran. With `--retry`, a test only counts as failed once its retries are used up. Cannot be combined with `--async`. |
+| `--priority <level>`            | Queue priority among your own runs waiting for a free parallel slot: `high`, `normal` (default) or `low`. Higher priority starts first, then the oldest run. It does not take slots from other accounts or raise your parallel limit.                                    |
 | `--download-artifacts [mode]`   | Download video, logs and screenshots as a zip after completion: `all` (default) or `failed`                                                                                                                                                                              |
 | `--artifacts-output-dir <path>` | Directory for the artifacts zip (defaults to the current directory)                                                                                                                                                                                                      |
 

src/cli.ts

@@ -550,6 +550,10 @@ program
     '--fail-fast',
     'Stop at the first failure that will not be retried: cancel the runs still queued or running.',
   )
+  .option(
+    '--priority <level>',
+    'Queue priority among your own runs waiting for a free parallel slot: high, normal (default) or low.',
+  )
   .option(
     '--download-artifacts [mode]',
     'Download the video, logs and screenshots after completion. Mode: all (default) or failed.',
@@ -660,6 +664,7 @@ program
         retry: args.retry,
         timeout: args.timeout,
         failFast: args.failFast,
+        priority: args.priority,
         downloadArtifacts:
           args.downloadArtifacts === true
             ? 'all'
@@ -838,6 +843,10 @@ program
     '--fail-fast',
     'Stop at the first failure that will not be retried: cancel the runs still queued or running.',
   )
+  .option(
+    '--priority <level>',
+    'Queue priority among your own runs waiting for a free parallel slot: high, normal (default) or low.',
+  )
   .option(
     '--async',
     'Start tests and exit immediately without waiting for results.',
@@ -1079,6 +1088,7 @@ program
         retry: args.retry,
         timeout: args.timeout,
         failFast: args.failFast,
+        priority: args.priority,
         cancelPrevious: args.cancelPrevious,
         debug: args.debug,
         googlePlayStore: args.googlePlay,
@@ -1189,6 +1199,10 @@ program
     '--fail-fast',
     'Stop at the first failure that will not be retried: cancel the runs still queued or running.',
   )
+  .option(
+    '--priority <level>',
+    'Queue priority among your own runs waiting for a free parallel slot: high, normal (default) or low.',
+  )
   .option(
     '--download-artifacts [mode]',
     'Download the video, logs and screenshots after completion. Mode: all (default) or failed.',
@@ -1292,6 +1306,7 @@ program
         retry: args.retry,
         timeout: args.timeout,
         failFast: args.failFast,
+        priority: args.priority,
         downloadArtifacts:
           args.downloadArtifacts === true
             ? 'all'

src/models/espresso_options.ts

@@ -2,7 +2,9 @@ import { randomUUID } from 'node:crypto';
 import TestingBotError from './testingbot_error';
 import {
   MAX_RETRY,
+  parsePriority,
   validateTimeout,
+  type Priority,
   type ArtifactDownloadMode,
   type RunMetadata,
 } from './maestro_options';
@@ -92,6 +94,7 @@ export default class EspressoOptions {
   private _retry: number;
   private _timeout: number | undefined;
   private _failFast: boolean;
+  private _priority?: Priority;
   private _downloadArtifacts?: ArtifactDownloadMode;
   private _artifactsOutputDir?: string;
   // Metadata
@@ -132,6 +135,7 @@ export default class EspressoOptions {
       retry?: number;
       timeout?: number;
       failFast?: boolean;
+      priority?: string;
       downloadArtifacts?: ArtifactDownloadMode;
       artifactsOutputDir?: string;
       metadata?: RunMetadata;
@@ -192,6 +196,7 @@ export default class EspressoOptions {
     this._timeout = options?.timeout;
     validateTimeout(this._timeout);
     this._failFast = options?.failFast ?? false;
+    this._priority = parsePriority(options?.priority);
     this._downloadArtifacts = options?.downloadArtifacts;
     this._artifactsOutputDir = options?.artifactsOutputDir;
     this._metadata = options?.metadata;
@@ -332,6 +337,10 @@ export default class EspressoOptions {
     return this._failFast;
   }
 
+  public get priority(): Priority | undefined {
+    return this._priority;
+  }
+
   public get downloadArtifacts(): ArtifactDownloadMode | undefined {
     return this._downloadArtifacts;
   }

src/models/maestro_options.ts

@@ -93,6 +93,27 @@ export function validateTimeout(timeout: number | undefined): void {
   }
 }
 
+/**
+ * --priority: where these runs go in your own queue while they wait for a free
+ * parallel slot. high starts before normal, normal before low, then oldest
+ * first. It does not take slots from other accounts or raise your limits.
+ */
+export const PRIORITIES = ['high', 'normal', 'low'] as const;
+export type Priority = (typeof PRIORITIES)[number];
+
+export function parsePriority(
+  priority: string | undefined,
+): Priority | undefined {
+  if (priority === undefined) return undefined;
+  const level = priority.trim().toLowerCase();
+  if (!(PRIORITIES as readonly string[]).includes(level)) {
+    throw new Error(
+      `Invalid --priority value (${priority}). Use one of: ${PRIORITIES.join(', ')}.`,
+    );
+  }
+  return level as Priority;
+}
+
 export default class MaestroOptions {
   private static isIpaFile(app?: string): boolean {
     return app?.toLowerCase().endsWith('.ipa') ?? false;
@@ -151,6 +172,7 @@ export default class MaestroOptions {
   private _retry: number;
   private _timeout: number | undefined;
   private _failFast: boolean;
+  private _priority?: Priority;
   private _cancelPrevious: boolean;
   private _debug: boolean;
   private _configFile?: string;
@@ -195,6 +217,7 @@ export default class MaestroOptions {
       retry?: number;
       timeout?: number;
       failFast?: boolean;
+      priority?: string;
       cancelPrevious?: boolean;
       debug?: boolean;
       configFile?: string;
@@ -277,6 +300,7 @@ export default class MaestroOptions {
     this._timeout = options?.timeout;
     validateTimeout(this._timeout);
     this._failFast = options?.failFast ?? false;
+    this._priority = parsePriority(options?.priority);
     this._debug = options?.debug ?? false;
     this._configFile = options?.configFile;
     this._groups = options?.groups;
@@ -441,6 +465,10 @@ export default class MaestroOptions {
     return this._failFast;
   }
 
+  public get priority(): Priority | undefined {
+    return this._priority;
+  }
+
   public get cancelPrevious(): boolean {
     return this._cancelPrevious;
   }

src/models/xcuitest_options.ts

@@ -2,7 +2,9 @@ import { randomUUID } from 'node:crypto';
 import TestingBotError from './testingbot_error';
 import {
   MAX_RETRY,
+  parsePriority,
   validateTimeout,
+  type Priority,
   type ArtifactDownloadMode,
   type RunMetadata,
 } from './maestro_options';
@@ -79,6 +81,7 @@ export default class XCUITestOptions {
   private _retry: number;
   private _timeout: number | undefined;
   private _failFast: boolean;
+  private _priority?: Priority;
   private _downloadArtifacts?: ArtifactDownloadMode;
   private _artifactsOutputDir?: string;
   // Metadata
@@ -112,6 +115,7 @@ export default class XCUITestOptions {
       retry?: number;
       timeout?: number;
       failFast?: boolean;
+      priority?: string;
       downloadArtifacts?: ArtifactDownloadMode;
       artifactsOutputDir?: string;
       metadata?: RunMetadata;
@@ -165,6 +169,7 @@ export default class XCUITestOptions {
     this._timeout = options?.timeout;
     validateTimeout(this._timeout);
     this._failFast = options?.failFast ?? false;
+    this._priority = parsePriority(options?.priority);
     this._downloadArtifacts = options?.downloadArtifacts;
     this._artifactsOutputDir = options?.artifactsOutputDir;
     this._metadata = options?.metadata;
@@ -277,6 +282,10 @@ export default class XCUITestOptions {
     return this._failFast;
   }
 
+  public get priority(): Priority | undefined {
+    return this._priority;
+  }
+
   public get downloadArtifacts(): ArtifactDownloadMode | undefined {
     return this._downloadArtifacts;
   }

src/providers/espresso.ts

@@ -169,6 +169,7 @@ export default class Espresso extends BaseProvider<EspressoOptions> {
           capabilities: [capabilities],
           ...(espressoOptions && { espressoOptions }),
           ...(metadata && { metadata }),
+          ...(this.options.priority && { priority: this.options.priority }),
         },
       });
 
@@ -314,6 +315,7 @@ export default class Espresso extends BaseProvider<EspressoOptions> {
           capabilities: [capabilities],
           ...(espressoOptions && { espressoOptions }),
           ...(metadata && { metadata }),
+          ...(this.options.priority && { priority: this.options.priority }),
         },
         {
           headers: {

src/providers/maestro.ts

@@ -496,6 +496,7 @@ export default class Maestro extends BaseProvider<MaestroOptions> {
             shardSplit: this.options.shardSplit,
           }),
           ...(metadata && { metadata }),
+          ...(this.options.priority && { priority: this.options.priority }),
           ...(this.options.cancelPrevious && { cancelPrevious: true }),
           ...(otherApps.length > 0 && {
             otherApps: previewOtherAppUrls,
@@ -2486,6 +2487,7 @@ export default class Maestro extends BaseProvider<MaestroOptions> {
             shardSplit: this.options.shardSplit,
           }),
           ...(metadata && { metadata }),
+          ...(this.options.priority && { priority: this.options.priority }),
           ...(this.options.cancelPrevious && { cancelPrevious: true }),
           ...(this.otherAppUrls.length > 0 && {
             otherApps: this.otherAppUrls,

src/providers/xcuitest.ts

@@ -165,6 +165,7 @@ export default class XCUITest extends BaseProvider<XCUITestOptions> {
           capabilities: [capabilities],
           ...(xcuitestOptions && { options: xcuitestOptions }),
           ...(metadata && { metadata }),
+          ...(this.options.priority && { priority: this.options.priority }),
         },
       });
 
@@ -310,6 +311,7 @@ export default class XCUITest extends BaseProvider<XCUITestOptions> {
           capabilities: [capabilities],
           ...(xcuitestOptions && { options: xcuitestOptions }),
           ...(metadata && { metadata }),
+          ...(this.options.priority && { priority: this.options.priority }),
         },
         {
           headers: {

tests/models/maestro_options.test.ts

@@ -843,4 +843,27 @@ describe('MaestroOptions', () => {
       expect(caps).not.toHaveProperty('metadata');
     });
   });
+
+  describe('priority', () => {
+    it('accepts high, normal and low in any case', () => {
+      expect(
+        new MaestroOptions('app.apk', 'flows', 'Pixel 6', { priority: ' LOW ' })
+          .priority,
+      ).toBe('low');
+      expect(new MaestroOptions('app.apk', 'flows', 'Pixel 6').priority).toBe(
+        undefined,
+      );
+    });
+
+    it('rejects anything else', () => {
+      expect(
+        () =>
+          new MaestroOptions('app.apk', 'flows', 'Pixel 6', {
+            priority: 'urgent',
+          }),
+      ).toThrow(
+        'Invalid --priority value (urgent). Use one of: high, normal, low.',
+      );
+    });
+  });
 });

tests/providers/espresso.test.ts

@@ -577,6 +577,26 @@ describe('Espresso', () => {
       );
     });
 
+    it('sends --priority with the run', async () => {
+      const prioritized = new Espresso(
+        mockCredentials,
+        new EspressoOptions(
+          'path/to/app.apk',
+          'path/to/testApp.apk',
+          'Pixel 6',
+          {
+            priority: 'low',
+          },
+        ),
+      );
+      prioritized['appId'] = 1234;
+      axios.post = jest.fn().mockResolvedValueOnce({ data: { success: true } });
+
+      await prioritized['runTests']();
+
+      expect((axios.post as jest.Mock).mock.calls[0][1].priority).toBe('low');
+    });
+
     it('should not include metadata when not provided', async () => {
       espresso['appId'] = 1234;
 

tests/providers/maestro.test.ts

@@ -993,6 +993,40 @@ describe('Maestro', () => {
       });
     });
 
+    describe('--priority', () => {
+      it('sends the priority with the run', async () => {
+        const provider = new Maestro(
+          mockCredentials,
+          new MaestroOptions('path/to/app.apk', 'path/to/flows', 'Pixel 6', {
+            priority: 'High',
+          }),
+        );
+        provider['appId'] = 1234;
+        axios.post = jest
+          .fn()
+          .mockResolvedValueOnce({ data: { success: true }, headers: {} });
+
+        await provider['runTests']();
+
+        expect((axios.post as jest.Mock).mock.calls[0][1].priority).toBe(
+          'high',
+        );
+      });
+
+      it('is not sent unless asked', async () => {
+        maestro['appId'] = 1234;
+        axios.post = jest
+          .fn()
+          .mockResolvedValueOnce({ data: { success: true }, headers: {} });
+
+        await maestro['runTests']();
+
+        expect((axios.post as jest.Mock).mock.calls[0][1]).not.toHaveProperty(
+          'priority',
+        );
+      });
+    });
+
     it('should not include maestroOptions when none are set', async () => {
       maestro['appId'] = 1234;