k6 Cloud test investigator

Structured 9-step workflow for investigating a Grafana Cloud k6 test or run.

This skill is workflow-only. All API calls go through

gcx api

against the plugin proxy, using v6 for REST and v5 for metrics. For the underlying mechanics (gcx auth, path conventions, endpoint discovery, log queries, script editing, threshold semantics, gotchas), see the

k6-manage

skill — every step below references it.

The content unique to this skill is:

the ordered investigation flow (Steps 1-9 below)
the date-alignment check ("last 7 days" ≠ "last 7 runs")
the 3-layer pass/fail determination (
```
result
```
vs
```
status
```
vs per-check
```
checks
```
query)
a worked example with realistic numbers (
```
references/worked-example.md
```
)

Core principles

Read before write. Always GET the script before any PUT.
```
gcx k6 load-tests update-script
```
is a write that replaces the live script with whatever file you pass — running it "just to see the URL it hits" has cost users their production scripts. If you need to learn URLs, run any non-mutating command with
```
-vvv --log-http-payload
```
instead.
Paginate when enumerating runs. The
```
/test_runs
```
endpoint caps at 1000 rows and
```
gcx k6 runs list --limit 0
```
does not auto-follow
```
@nextLink
```
. Use the
```
gcx api
```
loop documented in
```
k6-manage
```
§3.
Verify date framing. When the user says "last 7 days", "this week", "recent runs" — confirm the most-recent run's
```
created
```
actually falls in that window. Surface the gap if not.
check()
doesn't fail runs; only
thresholds
do. And thresholds with zero observations are reported as ✓ pass. See "Threshold semantics" below for the full deep dive; the per-check
```
checks
```
metric query in Step 5 catches both cases.

Prerequisites

gcx

installed and authenticated against the user's stack. See

k6-manage

§1. Verify with:

bash

gcx --context <stack> config check    # expect "✔ Connectivity: online"

Investigation workflow

Step 1: Identify the test and target run(s)

From the user's URL:

```
/a/k6-app/tests/<id>
```
→ load test (parent of many runs)
```
/a/k6-app/runs/<id>
```
→ a specific run

To go run → test: fetch the run via

gcx api

, see

k6-manage

§2 for the path-shape rules:

bash

gcx --context <stack> api /api/plugins/k6-app/resources/cloud/cloud/v6/test_runs/<run_id>

and read

.test_id

from the response. To list runs for a test, see Step 3.

Step 2: Fetch test metadata and script

bash

gcx --context <stack> k6 load-tests get <test_id> -o json

For the script, follow the GET half of the safe-edit recipe in

k6-manage

§5 — save a backup if you'll be editing later.

Two script endpoints exist, and the difference matters for investigation.

k6-manage

§5 documents both: the current load-test script and the per-run snapshot that was actually executed. They drift apart whenever the script is edited after a run. Whenever the question involves "what changed", "why did this run fail", or you're examining a run more than a few days old, also fetch the run-bundled snapshot(s) via

k6-manage

§5's run-script endpoint and diff against the current load-test script (or against another run's snapshot). The current load-test script is the wrong artifact to reason about a past run.

Step 3: List runs WITH pagination

Use the

gcx api

@nextLink

loop pattern documented in

k6-manage

§3 against

/cloud/v6/load_tests/<test_id>/test_runs

. After collecting

all_runs

python

print(f"Total: {len(all_runs)}")
runs_sorted = sorted(all_runs, key=lambda r: r['created'], reverse=True)
for r in runs_sorted[:10]:
    print(f"  {r['created']:30s} id={r['id']:>8} status={r['status']:<10} result={r.get('result','?')}")

Report to user: total run count, date range, latest run date. If "latest run" is more than a day old, call it out — they may believe the schedule is firing when it isn't.

Step 4: Verify date alignment with user's intent

If the user asked for "last 7 days" / "this week" / "recent": filter by date range, not by row count — "7 most recent runs" could span a day or a year depending on how often the test runs.

python

last7 = [r for r in all_runs if r['created'] >= '<today_minus_7_days_iso>']

len(last7) == 0

: surface this to the user immediately. Don't proceed with stale data.

Step 5: Determine pass/fail status

For each run examine three independent layers:

Layer	Field	Meaning
Run-level outcome	`result` ( `passed` / `failed` / `error` / `aborted` )	Whether thresholds breached
Run-level status	`status` ( `completed` / `aborted` )	Whether the run finished orderly
In-script checks	v5 `checks` metric, aggregated by the `check` label	Per-check success rate

For a run that the user thinks is "failing" but reports

result: passed

: check the third layer. Common pattern: every iteration's

check()

returns false but the run still "passes" because no threshold is defined on

checks

. See "Threshold semantics" below for the full deep dive (zero-observation trap,

abortOnFail

cloud delay, operator support).

Query the per-check breakdown via v5 (see

k6-manage/references/metrics.md

§7 for

query_aggregate_k6

shape):

bash

gcx --context <stack> api \
  "/api/plugins/k6-app/resources/cloud/cloud/v5/test_runs/<run_id>/query_aggregate_k6(query='ratio by (check)',metric='checks')"

Each result entry in the response carries a

check

label with the check name and a single value: the success ratio (0.0–1.0). For raw success/fail counts, also query

increase_nz by (check)

(successes) and

increase_z by (check)

(failures).

Step 6: Fetch metrics for a run

Use the v5 metrics endpoints documented in

k6-manage/references/metrics.md

. Typical workflow:

bash

# 6a. List metrics available for the run (metrics.md §1)
gcx --context <stack> api /api/plugins/k6-app/resources/cloud/cloud/v5/test_runs/<run_id>/metrics

# 6b. List labels for a metric to know what's available to filter/group by (metrics.md §4)
gcx --context <stack> api \
  "/api/plugins/k6-app/resources/cloud/cloud/v5/test_runs/<run_id>/labels?match[]=http_req_duration"

# 6c. Time-series — pick a query method that matches the metric's type (metrics.md §6)
gcx --context <stack> api \
  "/api/plugins/k6-app/resources/cloud/cloud/v5/test_runs/<run_id>/query_range_k6(query='histogram_quantile(0.95) by (name,status)',metric='http_req_duration',step=10)"

# 6d. Scalar aggregate over the whole run (metrics.md §7)
gcx --context <stack> api \
  "/api/plugins/k6-app/resources/cloud/cloud/v5/test_runs/<run_id>/query_aggregate_k6(query='increase',metric='http_reqs')"

Picking the right query method per metric type matters — see the "Query methods" tables in

k6-manage/references/metrics.md

. For browser tests, the URL/status breakdown comes from

labels

label/{name}/values

(metrics.md §4/§5), not from a separate tags endpoint.

Step 7: Fetch logs for a run

Follow the Loki recipe in

k6-manage

§4 (

gcx api

against

/api/plugins/k6-app/resources/logs/...

{test_run_id="<id>"}

selector, mandatory

X-K6TestRun-Id

header, scope

start

end

run.created

run.ended

). Save the response to

/tmp/run_<id>_logs.json

and summarise from the file as described there.

Step 8: (If asked) edit the script safely

Use the full safe-edit recipe in

k6-manage

§5 (GET → backup → edit →

k6 inspect

→ 1-iter smoke → PUT with

Content-Type: application/octet-stream

→ sha256 verify).

When the edit involves thresholds, also read "Threshold semantics" below for the zero-observation trap, the

abortOnFail

cloud delay, and operator support (

==

!=

work despite docs).

Step 9: Report

Pick a template based on the question shape.

For single-run investigations ("what's going on with run X"):

Test: <name> (id <test_id>)
Run:  <run_id> @ <created> → <ended>, load_zone=<zone>, result=<result>

Logs (<n> streams, <m> lines):
  - <stream summary>

Metrics (key indicators):
  - <metric>: <method> = <value> (n=<count>)
  ...

Per-check breakdown:
  - <check name>: <success_rate> (succ=<n>, fail=<n>)
  ...

Diagnosis: <one-paragraph>

For multi-run comparisons ("what's different between passing and failing runs", "did this change cause the failures"):

Question: <one-line restatement of what the user is trying to attribute>

Run timeline (relevant window):

| Run ID | Created (UTC) | Result | exec_duration | processing_duration | k6 build | error code | key check ratios |
|---|---|---|---|---|---|---|---|
| ... | ... | passed | 60s | 195s | <build> | — | 1.0/1.0/1.0 |
| ... | ... | failed | 27s | 193s | <build> | — | 0.0/n=0/n=0 |
| ... | ... | error  | 51s | 3601s aborted | <build> | 8016 | 1.0/1.0/1.0 |

Differences that matter:
  - <field>: <value-in-passing> vs <value-in-failing> — <interpretation>
  ...

Script diff (if relevant): <bundled-script diff between a representative passing and failing run, summarised>

Diagnosis: <one-paragraph attributing the change to test-side, SUT-side, or platform-side, with the supporting evidence>

The table makes side-by-side anomalies obvious (e.g. a column that's identical across passing and failing rules out that dimension as the cause; a column that flips between them is your candidate).

If the user asked for raw data dumps, also save them to

/tmp/k6inv/run<id>/

and tell them the paths.

Threshold semantics

Thresholds, not

check()

calls, are what determine a run's

result

. The behaviour has several non-obvious corners that mislead first-time investigators, so they're collected here. Step 5 ("Determine pass/fail status") and Step 8 ("edit the script safely") both depend on this section.

What

result

means

`result` value	Meaning
`passed`	All thresholds passed (or none defined)
`failed`	At least one threshold failed
`error`	Either the script crashed before finishing (e.g. browser wouldn't launch) or k6 Cloud aborted the run platform-side. To tell which, check `status_history[*].extra.code` on the run — a non-null code indicates a platform abort (e.g. `8016` = "Test run max lifetime exceeded" during `processing_metrics` ). Platform aborts are not your code's fault even though they surface as `error` .
`aborted`	User or system aborted the run

check()

calls do not affect

result

directly. They emit observations into the built-in

checks

rate metric, which a threshold may then evaluate. If no threshold references

checks

, failing checks are invisible to the run-level

result

The "zero observations = pass" trap

k6 reports a threshold as ✓ pass when the underlying metric has zero samples — even if the threshold expression would otherwise evaluate to false:

checks{check:response is 200}
✓ 'rate==1.0' rate=0.00%       ← ✓ pass!?

This bites when the script throws before reaching the relevant

check()

call. Force a check observation in a

catch

block to make the threshold see something:

javascript

try {
  const r = await page.goto(URL);
  check(r, { "response is 200": x => x.status() === 200 });
  // ... rest of iteration body
  check(true, { "script completed without exception": () => true });
} catch (e) {
  console.error(e);
  check(null, { "script completed without exception": () => false });
} finally {
  await page.close();
}

Then add

'checks{check:script completed without exception}': ['rate==1.0']

to thresholds. This catches both navigation failures AND later-stage exceptions.

abortOnFail

cloud delay

When k6 runs in the cloud, thresholds are evaluated every 60 seconds. The
abortOnFail
feature may be delayed by up to 60 seconds.

For runs shorter than 60 s,

abortOnFail

may not fire before iterations complete naturally. The threshold is still evaluated at end-of-run and

result

flips to

failed

— abort just doesn't save execution time.

Operators

Docs show

<=

>=

. ==
and
!=
also work even though they aren't in the docs. For

rate

metrics (value 0.00–1.00),

rate==1.0

means "every observation was non-zero" and

rate==0

means "all observations were zero."

Workflow-specific gotchas

For the canonical gotcha list (auth expiry, doubled

cloud/cloud/

, 415 on script PUT, Loki missing

X-K6TestRun-Id

, script PUT not bumping

updated

gcx k6 runs list --limit 0

not following

@nextLink

), see

k6-manage

§7. The entries below are specific to this investigation workflow:

Symptom	Cause	Fix
"Latest run was 6 months ago" but schedule says daily	You didn't paginate `/test_runs`	Use the `@nextLink` loop in `k6-manage` §3 (Step 3)
"Last 7 days" report contains only old runs	Filtered by row count, not date	Re-filter by `created >= <iso_date>` (Step 4)
Threshold reports ✓ pass but checks fail	Zero check observations; iteration aborted before `check()` ran	See "Threshold semantics" above
`result: error` but the script logs/metrics look fine	Likely a platform abort (e.g. `processing_metrics` exceeded the 1h cap, `code 8016` ). The execution itself completed normally.	Check `status_history[*].extra.code` on the run. Non-null platform code → not your code's fault. See "Threshold semantics"
Investigating a past run by reading the current load-test script	Script may have been edited since the run executed — what you're reading isn't what ran	GET the run's bundled script via the per-run endpoint ( `k6-manage` §5) and diff against the current load-test script before drawing conclusions
Just overwrote the user's script	Invoked `update-script` to "see the URL"	Restore from the backup taken in Step 2. To learn URLs without writing, use `-vvv --log-http-payload` on any non-mutating command.
CLI `--iterations 1` breaks browser scenarios	Overrides scenario block entirely	Edit `iterations:` in-file with sed instead

Reference

```
references/worked-example.md
```
— a synthetic investigation with realistic findings, useful as a template for the Step 9 report.

k6-cloud-investigate-test

NPX Install

Tags

SKILL.md Content

k6 Cloud test investigator

Core principles

Prerequisites

Investigation workflow

Step 1: Identify the test and target run(s)

Step 2: Fetch test metadata and script

Step 3: List runs WITH pagination

Step 4: Verify date alignment with user's intent

Step 5: Determine pass/fail status

Step 6: Fetch metrics for a run

Step 7: Fetch logs for a run

Step 8: (If asked) edit the script safely

Step 9: Report

Threshold semantics

What
`result`
means

The "zero observations = pass" trap

`abortOnFail`
cloud delay

Operators

Workflow-specific gotchas

Reference