Run Outqube from CI

The CLI is the shortest way to run Outqube from CI. With --wait it starts a run, waits for it, prints the failure reports and sets the exit code. It reads the commit, branch and pull request from GitHub Actions, GitLab CI and Azure Pipelines, so the run shows up with the right labels.

You need an API key with the trigger scope stored as the secret OUTQUBE_API_KEY. See Authentication.

GitHub Actions: nightly suiteCopy Anchor Link

This workflow runs the nightly suite every night at 02:00 UTC and on demand, writes a JUnit report and keeps it as an artifact.

name: Outqube nightly
on:
  schedule:
    - cron: "0 2 * * *"
  workflow_dispatch:

jobs:
  outqube:
    runs-on: ubuntu-latest
    steps:
      - name: Run the nightly suite
        run: npx outqube run --suite nightly --url ${{ vars.STAGING_URL }} --wait --timeout 45m --cancel-on-timeout --junit out.xml
        env:
          OUTQUBE_API_KEY: ${{ secrets.OUTQUBE_API_KEY }}
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: outqube-junit
          path: out.xml

The job fails when the exit code is not 0:

To fail only on failed tests and ignore run errors, pass --fail-on failed. The default, any, also fails on errors.

GitLab CICopy Anchor Link

outqube:
  image: node:22
  script: npx outqube run --suite smoke --url "$STAGING_URL" --wait --junit out.xml
  artifacts:
    when: always
    reports:
      junit: out.xml

OUTQUBE_API_KEY is a masked CI/CD variable. Add a merge_request_event rule to run in merge request pipelines.

Any schedulerCopy Anchor Link

0 2 * * * OUTQUBE_API_KEY=<your-key> npx outqube run --tag nightly --wait --fail-on failed --cancel-on-timeout --json > /var/log/outqube.json

For a schedule that Outqube runs itself, use the app's schedules instead.

Read the output in a scriptCopy Anchor Link

With --json, run --wait prints { run, failures: [{ test_run_id, name, outcome, summary, report_markdown }] }. An error is one line on stderr: {"error":{"message","exit_code","code","request_id"}}. Treat an outcome you do not know as not passed.

Without the CLICopy Anchor Link

Call the API yourself: start the run with POST /runs, poll GET /runs/{runId} until status is completed or cancelled, then fail your job unless outcome is passed. The Quickstart has the code. Send an Idempotency-Key that names the commit and job, so a retried job does not start a second run.

Next stepsCopy Anchor Link

• CLI reference: every command and option.
• Test every pull request with the GitHub App: get a result on the pull request itself.
• Receive and verify webhooks: react when a run finishes instead of waiting.