gcx help-tree profiles -o text

undefined

CLI support does not imply backend support. `data-range` returns tenant-wide
`dataIngested`, `oldestProfileTime`, and `newestProfileTime` (Unix
milliseconds; zero means unknown). These bounds do not prove that the selected
service or lens has continuous coverage. Do not assume a fixed retention
period.

Apply these outcomes before proceeding. Every exit stops the affected profile
queries and produces the Section 5 report; other usable signals may continue
through gcx's bundled `debug-with-grafana` skill
(`gcx agent skills get debug-with-grafana`) with the missing-evidence question
made explicit.

- **EXIT A — profiling unavailable:** no target datasource, missing access, or
  a required CLI capability is unavailable. Record the exact limitation; use
  `gcx login` and `gcx config check` for missing setup or access, or a
  compatible gcx build for a CLI mismatch.
- **EXIT B — no ingestion reported:** a successful stats response says
  `dataIngested: false`. Report the instrumentation/data gap,
  not service health.
- **EXIT C — incident not covered:** the incident has no overlap with known
  queryable bounds. Report both ranges; do not replace the incident with today.
  For partial overlap, analyze only that overlap, record the coverage gap, and
  limit conclusions to the observed interval.
- **EXIT D — no usable incident profiles:** after checking the selector, window,
  and lens against discovered facts, incident samples remain absent or zero. A
  missing baseline alone is handled in Section 2, not by this exit. Report the
  unresolved coverage/instrumentation gap, not proof of zero cost.

**Continue with scoped checks** when stats are absent, unsupported, or have
unknown bounds. Missing stats alone must not block usable type/profile queries.
These queries still need matching samples before they can support attribution.

For query errors, retain the operation and diagnostic. Correct a demonstrated
selector/flag mistake; use a documented fallback for unsupported capabilities.
Do not retry authentication failures or vary selectors blindly. A failed query
is not an empty successful result.

gcx profiles labels -d <pyro-uid> \
  --from <incident-from> --to <incident-to> -o json
gcx profiles labels -d <pyro-uid> --label service_name \
  --from <incident-from> --to <incident-to> -o json

Use `profiles series` to discover actual label combinations for a service
without requiring a profile type. See
[scoped series discovery](references/advanced-analysis.md#scoped-series-discovery)
for label projection. Tenant-wide label inventories alone do not prove labels
coexist on the suspect.

Repeat the chosen grouping for the baseline. Leaderboards are capped candidates,
not a fleet census; missing groups may have fallen outside the limit. A single
group establishes no peer comparison. Similar totals do not establish uniform
per-instance behavior, and the largest group may simply serve more traffic or
contain more replicas. Use request counts or replica exposure from existing
metrics when needed; if unavailable, state the confounder. Do not use a fixed
ratio threshold to declare a slice anomalous.

Use time-series values to check persistence/onset; `--top` collapses the window.
Keep units and aggregation semantics explicit, especially for live-heap or
count snapshots. Do not sum snapshot values into "bytes allocated" or infer
request-normalized cost from raw profile totals.

undefined

Take function rankings and shares from the table. It aggregates each function
and lists the top 20 by SELF. Its PERCENTAGE column is SELF divided by the
profile total, not the function TOTAL share. The table omits everything past
the top 20, so to total the share of a set of functions, such as all GC
functions, sum SELF over the full profile from `-o json` or a pprof export.
Names longer than 60 characters are cut and end in `...`; take the full name
from the matching DOT node before reporting or interpreting it.

Use DOT for caller/callee context, source locations, and full function names.
Its nodes can be single source lines (`file.go:line`), so one function may
appear as several nodes. A label splits the function name across lines,
starting with the last element of its package path, then reads
`self (%) of cumulative (%)`, or a single value when they are equal. Do not
rank functions or read self shares from DOT nodes. gcx falls back
to table output with a warning when the backend explicitly rejects DOT as
unsupported or returns a flame graph instead. Other errors still fail the query.
Inspect the returned format and retain stderr. JSON encodes flame-graph wire
data, not the human function table; agent output may instead reference a spill
file that must be read.

DOT is pruned by default; dotted edges can skip intermediate frames. When
caller context is missing, repeat the same query with `-o dot --max-nodes 250`
in both windows. This raises the node limit; it does not guarantee an unpruned
graph. Neither format can rule out a missing path. For ambiguous attribution or
numerical differences, read
[pprof comparison](references/advanced-analysis.md#pprof-comparison) and export
both profiles. Do not assume an exported profile repairs collection gaps.

Interpret the evidence with these constraints:

- Compare absolute cost and share in each window, with units and a denominator.
  Equal shares can hide doubled cost; larger shares can reflect other work
  disappearing. Percentages are not request latency or proof of regression.
- Self/flat means work at that frame; total/cumulative includes descendants.
  Use caller/callee context to locate a meaningful application path. Ancestor
  totals overlap: do not add them or assign runtime work to a guessed caller.
- Runtime symbols suggest mechanisms to test. `mallocgc` suggests allocation
  work; GC workers may run separately from allocation sites; `nanotime` alone
  does not prove busy-waiting. CPU samples in `futex`/locks do not measure time
  asleep. Confirm waiting with a supported wait lens and other evidence.
- In Go, heap stacks identify allocation sites, not the object retaining
  references. Mutex profiles attribute contention to the holder/unlock path;
  block profiles to the waiter. Verify the actual runtime semantics before
  transferring this interpretation to another language or collector.

When a matching checkout/build is available, inspect the implicated function to
validate the mechanism. Read profile locations and verified deployment revision
metadata; do not assume a Build ID is a Git SHA or that `service_repository`
always exists. Report file:line only when symbolization and revision support it;
otherwise retain function-level attribution and say what is missing. Do not
edit source as part of this investigation.

Conclusion:     <established cause | suspected cause | cost location |
                 inconclusive> — <answer to the original question>
Exit:           <A | B | C | D | none> — <reason, if applicable>
Scope:          <context, datasource UID, service>
Lens:           <exact profile type and units>
Incident:       <UTC from/to; exact selector; observed coverage>
Baseline:       <UTC from/to; exact selector; or unavailable with reason>
Control:        <why comparable; workload/collection differences or gaps>
Slice:          <affected label/value; evidence of scope or unknown>
Location:       <application stack; file:line and revision only if verified>
Cost:           <incident vs baseline absolute values, units, aggregation>
Share:          <incident vs baseline; self/cumulative and denominator>
Normalization:  <method and measured denominator | not applied, with reason>
Mechanism:      <supported explanation or explicitly labeled hypothesis>
Evidence:       <profile artifacts/links and corroborating symptom evidence>
Counterevidence: <alternatives checked and results | not checked, with reason>
Confidence:     <high | medium | low> — <reason tied to available evidence>
Limitations:    <coverage, sampling, baseline, symbolization or access gaps>
Next action:    <smallest missing evidence or handoff question | none>