Core concepts
What apps, tests, suites, environments, runs and results are in Outqube, and how a run moves from queued to a verdict.
The API is JSON over HTTPS at https://app.outqube.com/api/v1. Timestamps are ISO 8601 in UTC and ids are UUIDs. The OpenAPI document describes every endpoint for code generators.
Apps, tests, suites and environments
- App. The web app under test. An API key belongs to one app and sees only that app.
- Test. A goal written in plain language, such as "a new user can sign up". The agent works out the steps. A test has a number and a display id, a priority (
urgent,high,mediumorlow), tags and the devices it runs on. List them withGET /tests. - Suite. A named group of tests, for example
smoke. A run takes a suite by name or id. List them withGET /suites. - Environment. Where the tests run: a base URL and the domains the agent may visit. A run uses its suite's default environment unless you pass
environment(a name or an id). Passbase_urlto test a different URL, such as a preview deployment. It must be on the environment's host or one of its allowed domains, and not a private address.
You create and edit apps, tests, suites and environments in the dashboard.
Runs
A run is one execution of a set of tests. Start one with POST /runs and give exactly one of:
suite: an id or a name.test_ids: up to 100 test ids.tag: every test with this tag.
Each test in the run becomes a test run, one per test and device. GET /runs/{runId} returns the run with each test run, its own status and outcome, failures first.
A run has a status that moves forward and an outcome once it has finished.
| Field | Values | Meaning |
|---|---|---|
status | queued, running, completed, cancelled | Where the run is. It never goes back. |
outcome | passed, failed, error, null | Set when status is completed. failed if any test failed, else error if any test errored, was blocked or was inconclusive, else passed. null while the run is going and when it was cancelled. |
counts | total, queued, running, passed, failed, error, cancelled | Tests by their final attempt. A test that errored and is waiting for its retry counts as running. |
error means something other than a verdict on your app: a timeout, a lost worker, a test that could not finish. New values can appear in status, outcome and metadata.source over time, so a client should treat an unknown outcome as not passed.
Runs carry metadata: source (api, cli, github, schedule or ui), commit_sha, branch, pr_number and repo. You set the last four when you start a run, and the CLI and the GitHub App fill them in. They label and filter runs. They are not proof of where a run came from.
You can cancel a run with POST /runs/{runId}/cancel and start a new one linked to it with POST /runs/{runId}/rerun.
Results
For each test run you can read:
| Result | Endpoint | What it holds |
|---|---|---|
| Failure report | GET /runs/{runId}/tests/{testRunId}/report | Written once when a test fails or errors. Structured JSON and Markdown of about 2,000 tokens at most. |
| Steps | GET /runs/{runId}/tests/{testRunId}/steps | One entry per step: number, action, summary, URL and outcome (ok, error or blocked). |
| Step detail | GET /runs/{runId}/tests/{testRunId}/steps/{n} | What the agent did and why (reasoning), plus that step's console and network entries. |
| Logs | GET /runs/{runId}/tests/{testRunId}/logs | Console messages and network entries, filterable by kind. |
| Screenshots | GET /artifacts/{artifactId} | A step's screenshot. The call redirects to a signed link that works for 15 minutes. |
A failure report says what broke, the failing step with expected and actual, the steps to reproduce, the console errors and failed requests near the failure, and the commit and branch. Its classification is app_bug, test_issue or environment, with a confidence. While a report is being written the answer has status: "pending".
Evidence is deleted after the retention period, so a screenshot_artifact_id can be null for an old step.
Secrets are redacted
Secrets are redacted when data is stored. Reports, logs and steps show [REDACTED:name] instead of a test password or code. Network entries have no bodies or headers, and secrets in URLs are redacted.
Next steps
- Errors, pagination and limits: the error shape, cursors, rate limits and idempotency.
- Quickstart: start a run and read its verdict.
- API reference: every endpoint with its scope and responses.