k6-trend-analysis
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
Chinesek6 Trend Analysis
k6趋势分析
Analyze metric trends across multiple runs of a Grafana Cloud k6 test to catch
degradation early -- before thresholds breach and alerts fire. A P95 at 380ms
against a 500ms threshold is "green" today, but if it was 250ms a month ago,
something is quietly degrading; this skill surfaces that drift and recommends action.
分析Grafana Cloud k6测试多次运行的指标趋势,以便在阈值突破和警报触发之前尽早发现性能退化。当前P95延迟为380ms,阈值为500ms,虽然处于“正常”状态,但如果一个月前是250ms,说明存在隐性退化;此技能会发现这种漂移并给出行动建议。
What this skill does NOT do
此技能不具备的功能
- Deep-dive into a single run's failure: load
k6-cloud-investigate-test - Edit scripts or apply threshold changes: load
k6-test-maintenance - Create new test scripts: use the appropriate test creation workflow
- Query service-side metrics directly: this skill hands off to
when observability correlation is needed
debug-with-grafana
- 深入分析单次运行的失败情况:加载
k6-cloud-investigate-test - 编辑脚本或应用阈值变更:加载
k6-test-maintenance - 创建新测试脚本:使用相应的测试创建工作流
- 直接查询服务端指标:当需要可观测性关联时,此技能会转交至
debug-with-grafana
Dependencies
依赖项
This skill delegates all GCk6 API mechanics to . Read it before
executing any API call -- it covers auth, path construction (the doubled
prefix), pagination with , the spill envelope, and
metric query syntax. Do not duplicate that knowledge here.
k6-managecloud/cloud/@nextLinkTools used: (via k6-manage patterns).
gcx此技能将所有GCk6 API操作委托给****。在执行任何API调用之前请阅读该文档——它涵盖了认证、路径构造(双重前缀)、使用的分页、溢出信封以及指标查询语法。请勿在此处重复这些内容。
k6-managecloud/cloud/@nextLink使用的工具:(遵循k6-manage模式)。
gcxWorkflow
工作流程
Follow these steps in order. Present findings at the end -- do not apply changes.
按以下顺序执行步骤。在最后呈现结果——不要直接应用变更。
Step 1: Identify the test
步骤1:确定测试
The user provides one of:
- A GCk6 URL (extract the load test ID from the path)
- A test ID directly
- A test name (search via or the v6 API)
gcx k6 tests list
Confirm the test exists by fetching its metadata. Record the , ,
, and timestamp. You will need the load test ID (not a
run ID) for the multi-run metric endpoints.
idnameproject_idcreated用户会提供以下信息之一:
- GCk6 URL(从路径中提取负载测试ID)
- 直接提供测试ID
- 测试名称(通过或v6 API搜索)
gcx k6 tests list
通过获取测试元数据确认测试存在。记录、、和时间戳。多运行指标端点需要负载测试ID(而非运行ID)。
idnameproject_idcreatedStep 2: Determine the analysis window
步骤2:确定分析窗口
Default to last 30 days. Adjust if:
- The user requests a specific window
- The test has very few runs (<5 in 30 days) -- widen the window and note this
- The test runs very frequently (hundreds of runs in 30 days) -- consider sampling or narrowing. Ask the user if the volume is extreme (>200 runs)
State the window explicitly: "Analyzing runs from {start_date} to {end_date}."
默认使用最近30天。在以下情况调整窗口:
- 用户请求特定窗口
- 测试运行次数极少(30天内少于5次)——扩大窗口并注明这一点
- 测试运行非常频繁(30天内数百次)——考虑抽样或缩小窗口。如果数量极多(>200次),请询问用户
明确说明窗口:“分析{start_date}至{end_date}的运行情况。”
Step 3: Fetch runs in the window
步骤3:获取窗口内的运行记录
List all runs for the test using the v6 API with
pagination (see k6-manage Section 3 for the loop pattern). Filter
to runs within the analysis window.
$orderby=created desc@nextLinkFor each run, record:
- ,
id,createdended - (passed/failed/timed_out)
result - (created/queued/initializing/running/finished/aborted)
status - (if present -- users sometimes annotate runs)
note
Discard runs that did not reach status (aborted, timed_out, etc.)
unless the user specifically asks about them -- incomplete runs produce
unreliable metric aggregates.
finishedCount the runs. If fewer than 3 usable runs exist, inform the user that trend
analysis is not meaningful with this sample size and suggest widening the window
or waiting for more runs.
使用v6 API并按分页(查看k6-manage第3节的循环模式)列出测试的所有运行记录。筛选出分析窗口内的运行。
$orderby=created desc@nextLink为每个运行记录:
- 、
id、createdended - (通过/失败/超时)
result - (已创建/排队/初始化/运行中/已完成/已中止)
status - (如果存在——用户有时会为运行添加注释)
note
除非用户特别要求,否则丢弃未达到状态的运行(已中止、超时等)——不完整的运行会产生不可靠的指标聚合结果。
finished统计运行次数。如果可用运行次数少于3次,告知用户此样本量下趋势分析无意义,并建议扩大窗口或等待更多运行记录。
Step 4: Fetch all metrics and their types
步骤4:获取所有指标及其类型
Before querying values, discover what metrics the test emits. Use the multi-run
metric listing endpoint (k6-manage references/metrics.md Section 2):
GET /cloud/v5/load_tests/{loadTestId}/metrics(test_run_ids=[{id1},{id2},...])Pass a representative subset of run IDs (the first and last few) to catch
metrics that may have been added or removed over the window. Record each
metric's and (counter, gauge, trend, rate).
nametypeGroup metrics by type -- the query method must match the metric type (see
metrics.md "Query methods"). Using the wrong method returns empty results.
在查询指标值之前,先发现测试会输出哪些指标。使用多运行指标列表端点(k6-manage参考文档/metrics.md第2节):
GET /cloud/v5/load_tests/{loadTestId}/metrics(test_run_ids=[{id1},{id2},...])传入有代表性的部分运行ID(前几个和后几个),以捕捉窗口内可能新增或移除的指标。记录每个指标的和(计数器、仪表、趋势、比率)。
nametype按类型分组指标——查询方法必须与指标类型匹配(查看metrics.md的“查询方法”)。使用错误的方法会返回空结果。
Step 5: Fetch per-run aggregate values
步骤5:获取每运行的聚合值
For each metric, query its aggregate value across all runs in the window using
the multi-run aggregate endpoint (metrics.md Section 8):
GET /cloud/v5/load_tests/{loadTestId}/query_aggregate_k6(
query='<method>',
metric='<metric_name>',
test_run_ids=[{id1},{id2},...]
)Concrete form (proxy prefix per k6-manage §2; avoids the spill envelope):
gcx-o jsonbash
LT=<load_test_id>; IDS="123,124,125"
gcx --context <ctx> api "/api/plugins/k6-app/resources/cloud/cloud/v5/load_tests/$LT/query_aggregate_k6(query='histogram_quantile(0.95)',metric='http_req_duration',test_run_ids=[$IDS])" -o jsonChoose the aggregate method based on metric type:
| Metric type | Primary method | What it captures |
|---|---|---|
| trend | | P95 latency -- the most common SLO target |
| trend | | Median -- shows typical behavior |
| trend | | Mean -- sensitive to outliers |
| counter | | Total count per run |
| rate | | Success/failure ratio |
| gauge | | Peak value per run |
For trend-type metrics (latencies), query multiple quantiles (P50, P90, P95,
P99) to see if degradation is uniform or concentrated in the tail.
Always break down by request grouping for multi-target tests. A single
aggregate p95 across a whole test obscures regressions confined to one
endpoint or one page -- a 3x slowdown on one URL can be invisible at the
test-level p95 if the test hits many URLs. Default to grouped queries:
| Metric pattern | Default grouping | Why |
|---|---|---|
| | Each navigation emits its own web vital; per-URL trends pin regressions to a specific page. |
| | Different endpoints have different baselines; mixing them hides per-endpoint drift. |
| | Browser and protocol scenarios have very different durations; mixing them is meaningless. |
| Custom trends with tags | | Whatever the user tagged on is presumably what they care about. |
Discover available labels first via the labels endpoint (metrics.md §4) and
label-values endpoint (metrics.md §5) before constructing the grouped query.
Don't assume labels from reading the script -- a tag rename or refactor can
silently shift the label space. Example:
undefined对于每个指标,使用多运行聚合端点(metrics.md第8节)查询窗口内所有运行的聚合值:
GET /cloud/v5/load_tests/{loadTestId}/query_aggregate_k6(
query='<method>',
metric='<metric_name>',
test_run_ids=[{id1},{id2},...]
)具体的格式(遵循k6-manage第2节的代理前缀;避免溢出信封):
gcx-o jsonbash
LT=<load_test_id>; IDS="123,124,125"
gcx --context <ctx> api "/api/plugins/k6-app/resources/cloud/cloud/v5/load_tests/$LT/query_aggregate_k6(query='histogram_quantile(0.95)',metric='http_req_duration',test_run_ids=[$IDS])" -o json根据指标类型选择聚合方法:
| 指标类型 | 主要方法 | 捕获内容 |
|---|---|---|
| 趋势 | | P95延迟——最常见的SLO目标 |
| 趋势 | | 中位数——展示典型行为 |
| 趋势 | | 平均值——对异常值敏感 |
| 计数器 | | 每次运行的总计数 |
| 比率 | | 成功/失败比率 |
| 仪表 | | 每次运行的峰值 |
对于趋势型指标(延迟),查询多个分位数(P50、P90、P95、P99),以了解退化是均匀的还是集中在尾部。
**对于多目标测试,始终按请求分组拆分。**整个测试的单一聚合P95会掩盖局限于某个端点或页面的回归——如果测试访问多个URL,某个URL的3倍变慢在测试级P95中可能不可见。默认使用分组查询:
| 指标模式 | 默认分组 | 原因 |
|---|---|---|
| | 每次导航都会输出自己的Web Vital;按URL的趋势可将回归定位到特定页面。 |
| 如果请求标记了 | 不同端点有不同的基线;混合它们会隐藏端点级的漂移。 |
| | 浏览器和协议场景的持续时间差异很大;混合它们没有意义。 |
| 带标签的自定义趋势 | | 用户标记的内容大概是他们关心的。 |
在构建分组查询之前,先通过标签端点(metrics.md第4节)和标签值端点(metrics.md第5节)发现可用的标签。不要假设通过阅读脚本就能知道标签——标签重命名或重构可能会悄然改变标签空间。示例:
undefinedList labels for a specific metric on a representative run
列出特定指标在代表性运行中的标签
GET /cloud/v5/test_runs/{id}/labels?match[]=browser_web_vital_lcp
GET /cloud/v5/test_runs/{id}/labels?match[]=browser_web_vital_lcp
Then enumerate values for a useful label
枚举有用标签的值
GET /cloud/v5/test_runs/{id}/label/url/values
Only fall back to the bare aggregate (no `by`) for metrics where grouping
adds no information -- e.g., `vus`, `load_generator_cpu_percent`,
single-target tests where every request hits the same URL.
The response includes `test_run_id` as a label -- use this to map each value
back to its run timestamp from Step 3. Grouped queries return one series per
(group, run) combination; flatten into a tidy "rid x group" table for the
trend computation in Step 7.
If there are too many run IDs to fit in a single URL (hundreds), batch the
queries into groups of 50 run IDs and merge the results.GET /cloud/v5/test_runs/{id}/label/url/values
仅当分组无法增加信息时才回退到裸聚合(无`by`)——例如`vus`、`load_generator_cpu_percent`、所有请求都命中同一URL的单目标测试。
响应包含`test_run_id`作为标签——使用它将每个值映射回步骤3中的运行时间戳。分组查询会返回每个(组,运行)组合的一个序列;将其扁平化为整洁的“运行ID x 组”表格,用于步骤7的趋势计算。
如果运行ID太多无法放入单个URL(数百个),将查询分批为每组50个运行ID并合并结果。Step 6: Extract thresholds
步骤6:提取阈值
Thresholds define what "passing" means. Fetch the test's current script (via
k6-manage Section 5) and parse the
block. Record each threshold's:
export const options = { thresholds: {...} }- Metric name and selector (e.g., )
http_req_duration{name:homepage} - Condition (e.g., )
p(95)<500 - Whether is set
abortOnFail
Also check the most recent run's threshold results from the run data to see
which thresholds are currently passing vs. failing.
Not all metrics will have thresholds -- that's fine. Metrics without thresholds
still get trend analysis; they just won't have headroom calculations.
阈值定义了“通过”的标准。获取测试的当前脚本(通过k6-manage第5节)并解析块。记录每个阈值的:
export const options = { thresholds: {...} }- 指标名称和选择器(例如)
http_req_duration{name:homepage} - 条件(例如)
p(95)<500 - 是否设置
abortOnFail
同时查看最近一次运行的阈值结果,了解哪些阈值当前是通过还是失败的。
并非所有指标都有阈值——这没关系。没有阈值的指标仍会进行趋势分析,只是不会计算余量。
Step 7: Compute trends
步骤7:计算趋势
Detect inflection points and rule out script changes deterministically.
Before drawing conclusions about a regression, look for discontinuities in
the metric values -- sudden jumps or drops that align across multiple
metrics on the same date. When you spot one, do not guess whether the
script changed. The run-bundled script endpoint gives a deterministic
answer in seconds:
-
sha256-diff the bundled scripts at the boundary. Fetch the snapshot from the last "before" run and the first "after" run via(k6-manage §5, "Two distinct script endpoints" -- use the run-scoped endpoint, not the load-test one, since the latter only shows the current version). Compare with
GET /cloud/v6/test_runs/{id}/script.shasum -a 256bashgcx --context <ctx> api /api/plugins/k6-app/resources/cloud/cloud/v6/test_runs/<before_id>/script > /tmp/before.bin gcx --context <ctx> api /api/plugins/k6-app/resources/cloud/cloud/v6/test_runs/<after_id>/script > /tmp/after.bin shasum -a 256 /tmp/before.bin /tmp/after.binFor thoroughness on multi-run inflections, hash every run across the transition -- if N consecutive runs share one hash and N more share another, you have a clean before/after boundary. If hashes change mid-stream, the test was edited multiple times. -
If the sha256 differs, the test script changed -- the inflection may be a test-side artifact, not a service regression. Split the analysis into distinct eras at the boundary and compute trends within each era separately. Comparing metrics across script changes produces misleading trends -- a P95 drop from 3,000ms to 150ms is not an "improvement" if the script simply stopped hitting a slow endpoint. State the eras explicitly in the report and focus recommendations on the most recent era.
-
If the sha256 matches, the script is byte-identical and the regression is external to the test (service-side, infrastructure, or load-zone). This is high-confidence information -- carry it into Step 9 (service-side correlation) instead of leaving "did the test change?" as an open question.
This sha256 diff is a 5-second deterministic check that rules out a huge
class of causes. Run it at every detected inflection, not just when the
user asks.
For each metric, build a time-ordered series of (run_timestamp, value) pairs.
Then compute:
Basic statistics:
- Mean, standard deviation, min, max across all runs
- Current value (most recent run)
- Baseline value (oldest run in window, or mean of first 3 runs for stability)
Trend direction:
Split the runs into two halves (first half and second half of the time window).
Compare the mean of each half:
- Degrading: second-half mean is worse by >10% (higher for latency/errors, lower for success rates)
- Improving: second-half mean is better by >10%
- Stable: change is within 10%
- Volatile: standard deviation exceeds 25% of the mean, regardless of direction
The 10% and 25% thresholds are starting points. If the user's test has very
tight tolerances or very noisy metrics, adjust and explain the reasoning.
Rate of change:
Express as percentage change per week: . This normalizes across different window sizes.
((recent_mean - baseline_mean) / baseline_mean) * 100 / weeks_in_windowAnomaly detection:
Flag any run where the metric value is more than 2 standard deviations from the
overall mean. These are potential inflection points worth investigating
individually.
**检测拐点并确定性地排除脚本变更。**在得出回归结论之前,查看指标值中的不连续点——同一日期多个指标出现的突然上升或下降。当发现这种情况时,不要猜测脚本是否变更。运行绑定的脚本端点可在几秒内给出确定性答案:
-
**对边界处的绑定脚本进行sha256差异对比。**通过(k6-manage第5节,“两个不同的脚本端点”——使用运行范围的端点,而非负载测试端点,因为后者仅显示当前版本)获取最后一个“变更前”运行和第一个“变更后”运行的快照。使用
GET /cloud/v6/test_runs/{id}/script进行对比。shasum -a 256bashgcx --context <ctx> api /api/plugins/k6-app/resources/cloud/cloud/v6/test_runs/<before_id>/script > /tmp/before.bin gcx --context <ctx> api /api/plugins/k6-app/resources/cloud/cloud/v6/test_runs/<after_id>/script > /tmp/after.bin shasum -a 256 /tmp/before.bin /tmp/after.bin为了全面分析多运行拐点,对过渡期间的每个运行进行哈希——如果N个连续运行共享一个哈希,接下来N个运行共享另一个哈希,那么你就得到了清晰的变更前后边界。如果哈希在中途变化,说明测试被多次编辑。 -
如果sha256不同,说明测试脚本已变更——拐点可能是测试端的产物,而非服务回归。在边界处将分析拆分为不同阶段,并分别计算每个阶段的趋势。跨脚本变更比较指标会产生误导性趋势——如果脚本只是停止访问慢端点,P95从3000ms降至150ms并非“改进”。在报告中明确说明阶段,并将建议重点放在最近的阶段。
-
如果sha256匹配,说明脚本字节完全相同,回归是测试外部的(服务端、基础设施或负载区域)。这是高可信度信息——将其带入步骤9(服务端关联),而非留下“测试是否变更?”的疑问。
这个sha256差异对比是一个5秒的确定性检查,可以排除大量原因。在每个检测到的拐点都运行此检查,而不仅仅是用户询问时。
对于每个指标,构建按时间排序的(运行时间戳,值)对序列。然后计算:
基本统计数据:
- 所有运行的平均值、标准差、最小值、最大值
- 当前值(最近一次运行)
- 基线值(窗口内最早的运行,或前3次运行的平均值以保证稳定性)
趋势方向:
将运行分为两半(时间窗口的前半部分和后半部分)。比较每部分的平均值:
- 退化:后半部分平均值变差超过10%(延迟/错误更高,成功率更低)
- 改进:后半部分平均值变好超过10%
- 稳定:变化在10%以内
- 波动:标准差超过平均值的25%,无论方向如何
10%和25%的阈值是起点。如果用户的测试有非常严格的容差或非常嘈杂的指标,请调整并解释理由。
变化率:
以每周变化百分比表示:。这可以对不同窗口大小进行归一化。
((近期平均值 - 基线平均值) / 基线平均值) * 100 / 窗口内的周数异常检测:
标记任何指标值超出总体平均值2个标准差的运行。这些是值得单独调查的潜在拐点。
Step 8: Headroom analysis
步骤8:余量分析
For metrics that have thresholds defined (from Step 6), compute headroom:
headroom_pct = ((threshold_value - current_value) / threshold_value) * 100Classify headroom:
- Comfortable (>50%): well within limits
- Adequate (20-50%): healthy but worth monitoring
- Thin (<20%): at risk of breaching if trends continue
- Breached (<0%): already failing
For degrading metrics with thin headroom, estimate when the threshold will be
breached if the current rate of change continues:
weeks_until_breach = headroom_absolute / rate_of_change_per_weekThis is a rough projection, not a prediction -- present it as "at the current
rate of degradation, this metric could breach in approximately N weeks."
对于有阈值定义的指标(来自步骤6),计算余量:
余量百分比 = ((阈值值 - 当前值) / 阈值值) * 100对余量进行分类:
- 充足 (>50%):远在限制范围内
- 健康 (20-50%):健康但值得监控
- 紧张 (<20%):如果趋势持续,有突破风险
- 已突破 (<0%):已失败
对于余量紧张的退化指标,估计如果当前变化率持续,阈值何时会被突破:
突破剩余周数 = 绝对余量 / 每周变化率这是粗略的预测,而非精确预言——表述为“按照当前退化速度,此指标可能在约N周后突破阈值。”
Step 9: Service-side correlation (when warranted)
步骤9:服务端关联(必要时)
If Step 7 reveals degradation, offer (don't auto-run) to correlate with
service-side data to separate service degradation (fix the service),
test-environment changes (load-zone latency, LG exhaustion), and
script changes (a slower edit). Present findings and ask first.
Hand off to (via ) to query the service's Prometheus
metrics and Loki logs for the same window; look for service error-rate changes,
upstream latency, resource pressure (CPU/memory/pools), and deploys coinciding
with inflection points.
debug-with-grafanagcxWhen the service has no observability (third-party or another team's
service -- no Prometheus job, Loki stream, or probe), fall back to client-side
signal, which still localises the regression:
- Browser tests: compare Tempo iteration traces (k6-manage §7,
+
/api/v1/tempo/api/search) for a before/after run. The span-name rollup, slowest spans, and per-URL navigation durations pin the regression to a URL, locator action, or asset; web vitals are/traces/{id}span attributes -- read them directly.web_vital.* - Protocol tests: compare and
http_req_duration by (name, status); a regression on one named request points at that endpoint. Splithttp_req_failed by (name, status)(server time) vshttp_req_waiting(transfer) to separate slow processing from slow download.http_req_receiving - Payload corroboration (both): query /
data_received. Latency up + payload up -> server content changes; latency up + payload stable -> server processing changes; intermittent payload -> flaky cache or A/B test.browser_data_received
Prefer server-side correlation when available -- it answers "why" directly; the
fallbacks answer "where" and "what kind of change", and make a good ticket for
the owning team.
如果步骤7发现退化,主动提出(不要自动运行)与服务端数据关联,以区分服务退化(修复服务)、测试环境变更(负载区域延迟、LG耗尽)和脚本变更(较慢的编辑)。先呈现结果并询问用户。
转交至(通过)查询服务的Prometheus指标和Loki日志,查找与拐点同时发生的服务错误率变化、上游延迟、资源压力(CPU/内存/池)和部署。
debug-with-grafanagcx当服务无可观测性时(第三方或其他团队的服务——无Prometheus任务、Loki流或探针),回退到客户端信号,这仍能定位回归:
- 浏览器测试:比较变更前后运行的Tempo迭代跟踪(k6-manage第7节,+
/api/v1/tempo/api/search)。跨度名称汇总、最慢跨度和按URL的导航持续时间可将回归定位到URL、定位器操作或资源;Web Vital是/traces/{id}跨度属性——直接读取即可。web_vital.* - 协议测试:比较和
http_req_duration by (name, status);某个命名请求的回归指向该端点。拆分http_req_failed by (name, status)(服务器时间)与http_req_waiting(传输时间),以区分慢处理和慢下载。http_req_receiving - 负载验证(两者):查询/
data_received。延迟上升+负载上升 -> 服务器内容变更;延迟上升+负载稳定 -> 服务器处理变更;负载间歇性变化 -> 缓存不稳定或A/B测试。browser_data_received
如果可用,优先使用服务端关联——它直接回答“为什么”;回退方法回答“在哪里”和“哪种类型的变更”,并可为负责团队生成良好的工单。
Step 10: Present the report
步骤10:呈现报告
Always present findings as a structured report. Never apply changes directly.
始终以结构化报告呈现结果。永远不要直接应用变更。
Report template
报告模板
Use this structure. Omit sections that don't apply (e.g., skip "Anomalies" if
none were detected).
markdown
undefined使用此结构。省略不适用的部分(例如,如果未检测到异常,跳过“异常”部分)。
markdown
undefinedTrend Analysis: {test_name}
趋势分析:{test_name}
Test ID: {test_id}
Analysis window: {start_date} to {end_date} ({N} runs analyzed)
Overall health: {Healthy | Watch | Degrading | Critical}
测试ID:{test_id}
分析窗口:{start_date}至{end_date}(分析了{N}次运行)
整体健康状况:{健康 | 关注 | 退化 | 严重}
Run Summary
运行摘要
| Period | Runs | Passed | Failed | Pass Rate |
|---|---|---|---|---|
| First half | N | N | N | N% |
| Second half | N | N | N | N% |
| 时段 | 运行次数 | 通过次数 | 失败次数 | 通过率 |
|---|---|---|---|---|
| 前半段 | N | N | N | N% |
| 后半段 | N | N | N | N% |
Metric Trends
指标趋势
| Metric | Type | Current | Baseline | Change | Trend | Threshold | Headroom |
|---|---|---|---|---|---|---|---|
| http_req_duration (P95) | trend | 380ms | 250ms | +52% | Degrading | 500ms | 24% |
| http_req_failed | rate | 0.8% | 0.3% | +167% | Degrading | 1% | 20% |
| http_reqs | counter | 15,230 | 15,100 | +0.9% | Stable | - | - |
| 指标 | 类型 | 当前值 | 基线值 | 变化 | 趋势 | 阈值 | 余量 |
|---|---|---|---|---|---|---|---|
| http_req_duration (P95) | 趋势 | 380ms | 250ms | +52% | 退化 | 500ms | 24% |
| http_req_failed | 比率 | 0.8% | 0.3% | +167% | 退化 | 1% | 20% |
| http_reqs | 计数器 | 15,230 | 15,100 | +0.9% | 稳定 | - | - |
Flagged Issues
标记的问题
1. {metric_name}: {classification}
1. {metric_name}:{分类}
- Current: {value} | Baseline: {value} | Change: {pct}%
- Threshold: {threshold} | Headroom: {pct}%
- Rate of change: {pct}% per week
- Projected breach: ~{N} weeks at current rate
- Anomalous runs: {run_ids with dates, if any}
- 当前值:{value} | 基线值:{value} | 变化:{pct}%
- 阈值:{threshold} | 余量:{pct}%
- 变化率:每周{pct}%
- 预计突破时间:按当前速度约{N}周
- 异常运行:{带日期的运行ID,如果有}
Threshold Recommendations
阈值建议
| Metric | Current Threshold | Recommended | Rationale |
|---|---|---|---|
| http_req_duration | p(95)<500 | p(95)<420 | Current P95 is 380ms; tightening to 420ms gives 10% headroom from current performance while surfacing further degradation early |
| 指标 | 当前阈值 | 建议阈值 | 理由 |
|---|---|---|---|
| http_req_duration | p(95)<500 | p(95)<420 | 当前P95为380ms;收紧至420ms可从当前性能获得10%的余量,同时尽早发现进一步退化 |
Suggested Next Steps
建议的下一步行动
- Investigate service side: P95 latency has increased 52% -- consider
loading to check service health
debug-with-grafana - Deep-dive run {run_id}: anomalous P95 spike on {date} -- consider
loading for this run
k6-cloud-investigate-test - Tighten thresholds: 2 metrics have >30% headroom that could be
tightened -- consider loading to apply changes
k6-test-maintenance
undefined- 调查服务端:P95延迟已增加52%——考虑加载检查服务健康状况
debug-with-grafana - 深入分析运行{run_id}:{date}出现异常P95峰值——考虑加载分析此运行
k6-cloud-investigate-test - 收紧阈值:有2个指标的余量超过30%可收紧——考虑加载应用变更
k6-test-maintenance
undefinedOverall health classification
整体健康状况分类
Derive the overall health from the worst-case metric:
- Healthy: all metrics stable or improving, headroom comfortable or adequate
- Watch: at least one metric degrading but headroom still adequate
- Degrading: at least one metric degrading with thin headroom
- Critical: at least one metric has breached its threshold, or multiple metrics are degrading with thin headroom
从最差情况的指标推导整体健康状况:
- 健康:所有指标稳定或改进,余量充足或健康
- 关注:至少一个指标退化但余量仍健康
- 退化:至少一个指标退化且余量紧张
- 严重:至少一个指标已突破阈值,或多个指标退化且余量紧张
Threshold recommendations
阈值建议
When recommending threshold changes, follow these principles:
- Only tighten, never loosen unless the user asks. Loosening thresholds masks problems.
- Target 10-20% headroom above the recent P95 of the metric. Enough room for normal variance but tight enough to catch real degradation.
- Use the second-half mean as the baseline, not the single most recent run (which could be an outlier).
- Respect the user's intent: if thresholds are currently very loose (>100% headroom), they may be intentionally permissive. Mention the opportunity to tighten but don't push hard -- the user knows their context.
- Consider grouped thresholds: if the test uses tag-based thresholds
(e.g., ), recommend per-endpoint thresholds where the trends differ between endpoints.
http_req_duration{name:homepage}
建议阈值变更时遵循以下原则:
- 仅收紧,不放宽,除非用户要求。放宽阈值会掩盖问题。
- 目标为指标近期P95以上10-20%的余量。足够应对正常波动,但又足够严格以捕捉真实退化。
- 使用后半段平均值作为基线,而非单次最近运行(可能是异常值)。
- 尊重用户意图:如果当前阈值非常宽松(>100%余量),可能是用户故意设置的。提及收紧的机会但不要强行推荐——用户了解其上下文。
- 考虑分组阈值:如果测试使用基于标签的阈值(例如),建议对端点间趋势不同的情况使用端点级阈值。
http_req_duration{name:homepage}
Gotchas
注意事项
| Issue | Detail |
|---|---|
| Zero-observation thresholds | A threshold with zero observations passes by default in k6. If a metric appears to pass but has no data, flag it -- the threshold is not actually being evaluated. |
| Metric type changes across runs | If a metric's type changed between runs (e.g., script refactor), the multi-run aggregate endpoint uses the latest type. Earlier runs queried with the wrong method return empty. Flag this if detected. |
| Incomplete runs skew trends | Aborted or timed-out runs typically have shorter durations and fewer iterations, producing unrepresentative metric values. Exclude them by default. |
| LG resource metrics | |
| Rate metric direction | For |
| 问题 | 详情 |
|---|---|
| 零观测阈值 | k6中,零观测的阈值默认通过。如果某个指标看似通过但无数据,标记它——阈值实际上未被评估。 |
| 跨运行的指标类型变更 | 如果指标类型在运行间变更(例如脚本重构),多运行聚合端点会使用最新类型。使用错误方法查询早期运行会返回空结果。如果检测到这种情况,标记它。 |
| 不完整运行影响趋势 | 已中止或超时的运行通常持续时间更短、迭代次数更少,产生不具代表性的指标值。默认排除它们。 |
| LG资源指标 | |
| 比率指标方向 | 对于 |