integration-testing

When to use

• authoring or debugging pipeline test fixtures (

  .log

  /

  .json

  inputs,

  *-expected.json

  output, config files)
• setting up or debugging system tests (

  _dev/test/system/

  ,

  _dev/deploy/

  , mock APIs, 0-hits failures)
• writing script tests for failure paths, API error handling, or package upgrades

When not to use

• ingest pipeline processor design and architecture (

  ingest-pipelines

  )
• CEL program development (

  cel-programs

  )
• broad elastic-package command selection and stack lifecycle (

  elastic-package-cli

  )

Reference files — load the one that matches your test type

references/pipeline-testing.md

references/system-testing.md

references/script-testing.md

System test input-specific references

system-testing.md

cel

references/system-testing-cel.md

tcp

udp

references/system-testing-tcp-udp.md

http_endpoint

references/system-testing-http-endpoint.md

logfile

filestream

references/system-testing-logfile.md

kafka

gcp-pubsub

references/system-testing-kafka-pubsub.md

aws-s3

gcs

azure-blob-storage

azure-eventhub

references/system-testing-cloud-skip.md

References

• references/pipeline-testing.md

  — directory layout, naming conventions, fixture formats, config options, expected output format and review, core workflow, fixture scenario coverage, data anonymization, troubleshooting

• references/system-testing.md

  — generic system test reference: required layout, config fields, core commands, teardown failures,

  sample_event.json

  verification, general debugging

• references/system-testing-cel.md

  — CEL mock API wiring, 0-hits debugging for CEL, variable-capture patterns

• references/system-testing-tcp-udp.md

  — TCP/UDP log sender pattern with

  elastic/stream

  , signal coordination, port alignment

• references/system-testing-http-endpoint.md

  — webhook/HTTP endpoint testing with

  STREAM_PROTOCOL=webhook

  , auth headers

• references/system-testing-logfile.md

  — Alpine container +

  SERVICE_LOGS_DIR

  pattern for logfile/filestream inputs

• references/system-testing-kafka-pubsub.md

  — Kafka broker + stream producer, Pub/Sub emulator patterns

• references/system-testing-cloud-skip.md

  — when and why to skip system tests for cloud storage inputs (aws-s3, gcs, azure-blob-storage, azure-eventhub)

• references/script-testing.md

  — txtar format, env smoke test, system-level skeleton, mock service docker-compose and config.yml, upgrade test pattern, pitfalls, full o365 examples

• references/builder-setup-subagent-guidance.md

  — subagent operating manual for wiring data collection (docker-compose, sample logs, agent stream template, system test config, manifest var cleanup) for non-CEL data streams. The orchestrator dispatches subagents by passing this file's path in the task prompt; the subagent reads it itself in its own fresh context. Do NOT embed/paste its contents into the task prompt.

• references/builder-system-test-subagent-guidance.md

  — subagent operating manual for running

  elastic-package test system --generate

  after pipeline work completes (any testable input). Same dispatch rule as above: orchestrators pass the path, the subagent reads the file itself.