Scripting adb For CI — Bash Idioms, Exit Codes, Fan-out
This skill is the connective tissue that turns the other ADB skills into a CI pipeline: exit-code propagation, retries, timeouts, parallel device fan-out, port forwarding, idempotent setup, capture-on-failure, and Test Orchestrator wiring. Each section cross-links to the skill that owns the underlying primitive.
When to use this skill
- A CI job invokes and "passes" even when tests fail — it gates on , but always exits ; the real status lives in
INSTRUMENTATION_STATUS_CODE
lines on stdout (and is needed just to get that stdout at all).
- The team has multiple physical devices on a single agent and wants to run sharded tests in parallel.
- and are confused — most teams have at least one bug from the opposite argument order.
- Transient / failures are not retried; the job fails on the first hiccup.
- A script lacks cleanup so leaked port forwards or modified survive into the next run.
- The team wires Test Orchestrator with
androidTestImplementation
— the build silently fails to install the orchestrator APK.
When NOT to use this skill
- The single primitive (logcat, screencap, pull, push, settings, input, am instrument) is not yet understood. Read the relevant sibling skill first.
- The CI itself (Gradle plugin, Firebase Test Lab cloud config). Use FTL's own docs for that layer.
- Compose / instrumentation test mechanics. Use
../../../compose/synchronization/synchronizing-with-idle/SKILL.md
and friends.
Prerequisites
- Bash (zsh works for most of this; macOS users should for if relying on ).
- Platform Tools 24+ on the host. On older toolchains exit codes are unreliable.
- Devices on API 24+ for reliable shell exit-code propagation. Pre-Nougat use a sentinel in stdout and grep on the host.
Exit codes — what adb propagates
| Operation | Behavior |
|---|
| / | on success, non-zero on transport or filesystem failure. Always check. |
| / | on success. Modern adb returns non-zero on , but old versions exit even on package-manager failure — verify by piping stderr. |
| (and , , etc.) | Block until the state is reached, then exit . Useful boot gate: `adb wait-for-device shell getprop sys.boot_completed |
| On platform-tools ≥ 24 and device adbd ≥ Android 7.0 (Nougat / API 24), the device-side command's exit code is propagated. Older devices always returned regardless. |
| Transport ID, NOT a timeout. Most common scripting confusion. |
| Always exits , pass or fail — AOSP ends with an unconditional and AndroidJUnitRunner calls finish(Activity.RESULT_OK, …)
regardless of test outcome. makes the shell wait for the runner so its stdout is complete; without the shell returns immediately with nothing usable. Either way, gate CI on parsed INSTRUMENTATION_STATUS_CODE
lines (below), never on . |
exit-code propagation can be
disabled with
:
shell [-e ESCAPE] [-n] [-Tt] [-x] [COMMAND...]
-x: disable remote exit codes and stdout/stderr separation
You almost never want
in CI. The default is what you want.
AndroidJUnitRunner status codes
emits per-test
INSTRUMENTATION_STATUS_CODE: <int>
followed by a final
INSTRUMENTATION_CODE: <int>
:
| Code | Symbolic name | Meaning |
|---|
| REPORT_VALUE_RESULT_START
| Test started |
| | Test passed |
| REPORT_VALUE_RESULT_ERROR
| Unexpected error (process death, runner failure) |
| REPORT_VALUE_RESULT_FAILURE
| Assertion failed |
| REPORT_VALUE_RESULT_IGNORED
(AndroidJUnitRunner extension) | d / filtered out |
| REPORT_VALUE_RESULT_ASSUMPTION_FAILURE
(AndroidJUnitRunner extension) | JUnit returned false |
Don't treat or as failures — they are skips; only
and
are failures. The trailing
line is unrelated: it's an
value (
=
= no runner-level crash;
=
= runner died) and, like the shell
(always
), it does
not reflect test pass/fail. Grep stdout for per-test
INSTRUMENTATION_STATUS_CODE: -1
/
(see the WRONG/RIGHT pattern below).
Port forwarding — vs
The argument order is opposite. This is the most common scripting bug.
adb forward [--no-rebind] LOCAL REMOTE # host → device. LOCAL REMOTE
adb reverse [--no-rebind] REMOTE LOCAL # device → host. REMOTE LOCAL
| Direction | Use case | Recipe |
|---|
| Host needs to drive a device-side service. | adb forward tcp:9222 localabstract:chrome_devtools_remote
|
| Device needs to reach a local-only host service (CI runner, dev laptop). | adb reverse tcp:8080 tcp:8080
(Metro/local backend) |
| Token | Where |
|---|
| Either side. on LOCAL = pick any free port; chosen port is printed to stdout. |
| Linux abstract Unix domain socket. Used by Chrome DevTools / Stetho / Flipper. |
| Reserved Unix socket namespace. |
| Filesystem-backed Unix socket. |
| REMOTE only. Java Debug Wire Protocol for the given device-side pid. |
| REMOTE only. virtio-vsock (emulator). |
| Character device passthrough. |
| LOCAL only — listen on a pre-opened fd. |
Cleanup is the same surface:
,
,
. Always remove what was added in a
so re-runs on the same agent don't fail with "cannot bind: address already in use".
Timeouts — wrap with , not
is unrelated to timeout. The host has no per-command adb timeout flag. Wrap with the GNU/BSD
utility:
bash
timeout 60s adb -s emulator-5554 shell am instrument -w -r ...
echo $?
# 124 = timeout reached (process killed); 137 = had to escalate to SIGKILL.
# 0 just means am instrument exited (it always does) — see the timeout pattern below for pass/fail parsing.
For background captures, set a max duration so a stuck job doesn't fill the disk:
bash
( timeout 600s adb logcat -v threadtime > artifacts/logcat.txt ) &
LOGCAT_PID=$!
trap 'kill $LOGCAT_PID 2>/dev/null' EXIT
Retries on transient errors
The usual offenders:
error: device 'XYZ' not found
,
,
,
. A
+ reconnect resolves most.
bash
retry() {
local n=0 max=3 delay=2
until "$@"; do
n=$((n+1))
[ "$n" -ge "$max" ] && return 1
echo "[retry] $* failed (attempt $n); restarting adb"
adb kill-server; adb start-server
sleep "$delay"
done
}
retry adb install -r app.apk
retry adb shell am instrument -w -r ...
For
retries specifically, also
the test pkg first — otherwise transient state leaks across attempts. Note
only re-runs on
transport errors (the
host binary exits non-zero); a genuine test failure leaves
exiting
, so it won't be retried — detect those by parsing the output, not by exit code.
Parallel device fan-out
bash
adb devices \
| tail -n +2 \
| awk '$2=="device"{print $1}' \
| xargs -I {} -P 4 adb -s {} <command>
Key fragments:
- skips the header.
- filters out , , , etc.
- runs 4 in parallel; tune to physical USB-hub count.
- targets a serial. is the env-var equivalent inside the spawned shells.
For tests with shared device-state (logcat clear, app install), serialise the prep then parallelise the run:
bash
SERIALS=$(adb devices | awk '$2=="device"{print $1}')
echo "$SERIALS" | xargs -I {} -P 0 adb -s {} install -r app-debug.apk
echo "$SERIALS" | xargs -I {} -P 0 adb -s {} install -r app-debug-androidTest.apk
# Distribute shards across devices (one shard per device)
i=0
for S in $SERIALS; do
adb -s "$S" logcat -c
adb -s "$S" shell am instrument -w -r \
-e numShards $(echo "$SERIALS" | wc -l) -e shardIndex "$i" \
com.example.test/androidx.test.runner.AndroidJUnitRunner \
> "results-$S.txt" &
i=$((i+1))
done
wait
# am instrument always exits 0 — derive pass/fail from the captured output:
if grep -qE '^INSTRUMENTATION_STATUS_CODE: -[12]$' results-*.txt; then
echo "test failures detected"; exit 1
fi
(
runs as many concurrently as there are inputs.)
Sharding via /
"If you need to parallelize the execution of your tests ... use the
option to specify the number of separate shards to create and the
option to specify which shard to run." — AndroidJUnitRunner docs.
Distribution is hash-bucketed by test name — deterministic across runs given the same test set. Adding/removing tests reshuffles buckets.
Per-device artefact paths
Always namespace by serial:
bash
artifact_dir() { echo "artifacts/${1//[:_]/-}"; }
for S in $(adb devices | awk '$2=="device"{print $1}'); do
D=$(artifact_dir "$S"); mkdir -p "$D"
adb -s "$S" logcat -d -v threadtime > "$D/logcat.txt"
adb -s "$S" shell screencap -p /sdcard/last.png
adb -s "$S" pull /sdcard/last.png "$D/last.png"
done
Idempotent setup + cleanup with
Never assume clean state from a previous run. Bake cleanup into setup, and use
to enforce teardown.
Minimal setup preamble
bash
adb wait-for-device
adb shell input keyevent 82 # wake; no-op if already on
adb shell pm clear com.example.app # wipe app data + cache
adb shell pm clear com.example.app.test # wipe test process state
adb shell am force-stop com.example.app
# Hermetic animations
adb shell settings put global window_animation_scale 0
adb shell settings put global transition_animation_scale 0
adb shell settings put global animator_duration_scale 0
(See
../../control/injecting-input-and-state/SKILL.md
for the full
vs
reasoning.)
cleanup
bash
cleanup() {
adb shell am force-stop com.example.app || true
adb shell pm clear com.example.app || true
adb uninstall com.example.app || true
adb forward --remove tcp:6100 || true
adb reverse --remove-all || true
adb shell settings put global window_animation_scale 1 || true
adb shell settings put global transition_animation_scale 1 || true
adb shell settings put global animator_duration_scale 1 || true
}
trap cleanup EXIT
Each step is
— partial-state cleanup must never mask the original failure exit code.
Capture-on-failure
bash
set +e
./gradlew connectedDebugAndroidTest
RC=$?
set -e
if [ "$RC" -ne 0 ]; then
mkdir -p artifacts
adb shell screencap -p /sdcard/fail.png
adb pull /sdcard/fail.png artifacts/fail.png
adb logcat -d -v threadtime > artifacts/logcat.txt
adb logcat -d -b crash -v threadtime > artifacts/crash.txt
adb logcat -d -b events -v descriptive > artifacts/events.txt
adb shell dumpsys activity > artifacts/dumpsys-activity.txt
fi
exit $RC
(See
../../capture/capturing-screenshots-and-screenrecord/SKILL.md
and
../../observability/extracting-logs-with-logcat/SKILL.md
for the underlying primitives.)
Test Orchestrator wiring
"Android Test Orchestrator collects JUnit tests at the beginning of your test suite run, but it then executes each test separately, in its own instance of
." — developer.android.com.
Each
runs in its own
invocation, so process state, statics, and
singletons are reset between tests.
Gradle wiring — , NOT androidTestImplementation
gradle
android {
defaultConfig {
testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
testInstrumentationRunnerArguments clearPackageData: 'true'
}
testOptions {
execution 'ANDROIDX_TEST_ORCHESTRATOR'
}
}
dependencies {
androidTestImplementation 'androidx.test:runner:1.7.0'
androidTestUtil 'androidx.test:orchestrator:1.6.1' // NOT androidTestImplementation
}
makes the Orchestrator run
between tests — the strongest isolation knob.
Direct ADB invocation (non-Gradle)
bash
adb install --force-queryable -r path/to/orchestrator-1.6.1.apk
adb install --force-queryable -r path/to/test-services-1.6.0.apk
adb shell 'CLASSPATH=$(pm path androidx.test.services) app_process / \
androidx.test.services.shellexecutor.ShellMain am instrument -w -e \
targetInstrumentation com.example.test/androidx.test.runner.AndroidJUnitRunner \
androidx.test.orchestrator/.AndroidTestOrchestrator'
is needed on API 30+ so the Orchestrator can
into the target despite package-visibility rules.
Tradeoffs
- Pros: each test starts from a known-good ; one crash doesn't kill the rest of the run; between tests is real, not "I cleaned up in ".
- Cons: per-test process spin-up adds ~1–2 s. A 500-test suite goes from ~5 minutes to ~15+ minutes. Reserve for integration suites where state leakage actually bites.
Patterns
Pattern: WRONG — gating CI on 's exit code
bash
# WRONG
adb shell am instrument -w -r com.example.test/androidx.test.runner.AndroidJUnitRunner
RC=$?
[ "$RC" -eq 0 ] && echo "tests passed" || echo "tests failed (rc=$RC)"
# WRONG because: am instrument calls System.exit(0) (AOSP Instrument.java) and AndroidJUnitRunner's
# finish() reports RESULT_OK regardless. $? is 0 even when tests fail — with OR without -w.
bash
# RIGHT — keep -w (so stdout is complete) and -r (raw, parseable), then parse the stream
adb shell am instrument -w -r com.example.test/androidx.test.runner.AndroidJUnitRunner \
| tee instrument.log
# -1 = error, -2 = failure; -3 (ignored) and -4 (assumption failure) are skips, not failures.
if grep -qE '^INSTRUMENTATION_STATUS_CODE: -[12]$' instrument.log; then
echo "tests failed"; exit 1
fi
echo "tests passed"
Pattern: WRONG — confusing and argument order
bash
# WRONG — copy/pasted the forward syntax for reverse
adb reverse tcp:8080 tcp:8080 # ambiguous (works because both are tcp:8080)
adb reverse tcp:9000 tcp:8080 # this means: device port 9000 → host port 8080
# Developer expected: host service on 8080 reachable as 9000 on device.
# WRONG because: reverse argument order is REMOTE LOCAL. The device tries to dial 8080 on
# the host, but the dev meant to expose the local 8080 to the device on port 9000.
bash
# RIGHT — adb reverse REMOTE LOCAL
# Expose host service on 8080 to the device as 9000:
adb reverse tcp:9000 tcp:8080
# In the app code: open a connection to localhost:9000 — it terminates at the host's :8080.
# Or, simpler, identical port:
adb reverse tcp:8080 tcp:8080 # device 8080 ⇆ host 8080
Pattern: WRONG — androidTestImplementation
for the orchestrator APK
gradle
// WRONG
dependencies {
androidTestImplementation 'androidx.test:orchestrator:1.6.1'
}
// WRONG because: the orchestrator is a SEPARATE APK, not a library. androidTestImplementation
// merges it into the test APK's classpath, which AGP rejects (or silently ignores). The
// correct configuration is androidTestUtil, which AGP installs as an additional APK.
gradle
// RIGHT
dependencies {
androidTestImplementation 'androidx.test:runner:1.7.0'
androidTestUtil 'androidx.test:orchestrator:1.6.1'
}
Pattern: timeout an instrumentation invocation
bash
timeout 600s adb shell am instrument -w -r \
com.example.test/androidx.test.runner.AndroidJUnitRunner | tee instrument.log
RC=${PIPESTATUS[0]} # exit of `timeout`, not `tee`
case "$RC" in
124) echo "timed out at 600s"; exit 124 ;;
137) echo "timed out and had to be SIGKILLed"; exit 137 ;;
esac
# RC is 0 whether tests passed or failed (am instrument always exits 0); pass/fail comes from the stream:
if grep -qE '^INSTRUMENTATION_STATUS_CODE: -[12]$' instrument.log; then
echo "failed"; exit 1
fi
echo "passed"
Pattern: retry-with-server-bounce
bash
retry adb -s "$SERIAL" install -r app-debug.apk
retry adb -s "$SERIAL" shell am instrument -w -r \
-e clearPackageData true \
com.example.test/androidx.test.runner.AndroidJUnitRunner
The
helper bounces the adb server on transient failure (see "Retries on transient errors" above).
Pattern: parallel sharded fan-out with per-device artefacts
bash
SERIALS=( $(adb devices | awk '$2=="device"{print $1}') )
N=${#SERIALS[@]}
mkdir -p artifacts
pids=()
for i in "${!SERIALS[@]}"; do
S="${SERIALS[$i]}"
D="artifacts/${S//[:_]/-}"
mkdir -p "$D"
(
adb -s "$S" logcat -c
adb -s "$S" shell pm clear com.example.app
adb -s "$S" shell am instrument -w -r \
-e numShards "$N" -e shardIndex "$i" \
com.example.test/androidx.test.runner.AndroidJUnitRunner \
> "$D/results.txt"
# am instrument exits 0 regardless — pass/fail is in the captured stream.
if grep -qE '^INSTRUMENTATION_STATUS_CODE: -[12]$' "$D/results.txt"; then
adb -s "$S" shell screencap -p /sdcard/fail.png
adb -s "$S" pull /sdcard/fail.png "$D/fail.png"
adb -s "$S" logcat -d -v threadtime > "$D/logcat.txt"
exit 1
fi
) &
pids+=($!)
done
rc=0
for p in "${pids[@]}"; do wait "$p" || rc=1; done # fail the job if any shard failed
exit $rc
Mandatory rules
- MUST pass to so the shell waits for the runner and emits raw, parseable output. Without the shell returns before any output is produced.
- MUST NOT gate CI on from — it calls regardless of test outcome (with or without ). Detect failures by grepping stdout for
INSTRUMENTATION_STATUS_CODE: -1
(error) and (failure).
- MUST distinguish IGNORED and ASSUMPTION_FAILURE from real failures when parsing
INSTRUMENTATION_STATUS_CODE
. and are the failure codes.
- MUST remember that is (host first) and is (device first). The argument orders are opposite.
- MUST wrap long-running adb commands with (or on macOS). has no built-in timeout flag — is transport ID, not timeout.
- MUST use
androidTestUtil("androidx.test:orchestrator:1.6.1")
, not androidTestImplementation
. The orchestrator is a separate APK, not a library.
- MUST retry transient adb errors (, , ) with a bounded helper that calls
adb kill-server; adb start-server
between attempts.
- MUST install a that removes port forwards (, ), restores animation scales, and s the app under test. Each step so cleanup never masks the test exit code.
- MUST (not just ) before each test run. See
../../control/injecting-input-and-state/SKILL.md
.
- MUST NOT rely on exit-code propagation on devices below API 24. Use a sentinel in stdout for older targets, or pin minSdk for the test infra.
- MUST NOT confuse with . They share zero semantics.
- MUST NOT parallelise without serialising the install / setup phase — concurrent to the same device race.
- PREFERRED: Test Orchestrator + for hermetic isolation; pair with
useTestStorageService: 'true'
for routed artefacts (see ../../transfer/extracting-test-artifacts/SKILL.md
).
- PREFERRED: namespace artefacts by serial (
artifacts/${SERIAL//[:_]/-}/
) so in CI captures everything cleanly.
Verification
References
- developer.android.com/tools/adb — / syntax, transport, server.
- developer.android.com/studio/test/command-line — , / , AndroidJUnitRunner arguments, output dirs.
- developer.android.com/training/testing/instrumented-tests — Test Orchestrator overview.
- developer.android.com/reference/android/app/Instrumentation — constants (, , , ).
- firebase.google.com/docs/test-lab/android/command-line — Firebase Test Lab CI exit codes ( pass, test failure, indeterminate, incompatible, infra error).
- Research note
tasks/research/A3-adb-observability-automation.md
— full bash idioms, exit-code propagation matrix, port-forward token grammar, parallel fan-out, retries, capture-on-failure, Test Orchestrator wiring.
- Research note
tasks/research/A2-adb-shell-commands.md
— arguments, status codes, AndroidJUnitRunner / extensions.
- Sibling skill:
../../architecture/understanding-adb-architecture/SKILL.md
— server / daemon / .
- Sibling skill:
../../devices/connecting-to-devices/SKILL.md
— device states, .
- Sibling skill:
../../devices/connecting-over-wifi/SKILL.md
— / for headless CI.
- Sibling skill:
../../apps/installing-and-managing-apps/SKILL.md
— / / .
- Sibling skill:
../../tests/running-instrumented-tests-via-adb/SKILL.md
— deep dive.
- Sibling skill:
../../control/injecting-input-and-state/SKILL.md
— hermetic preamble (animations to 0, , ).
- Sibling skill:
../../capture/capturing-screenshots-and-screenrecord/SKILL.md
— / capture-on-failure.
- Sibling skill:
../../observability/extracting-logs-with-logcat/SKILL.md
— capture-on-failure.
- Sibling skill:
../../transfer/extracting-test-artifacts/SKILL.md
— / / / .
- Cross-set:
../../../instrumentation/runner/running-instrumented-tests-with-androidjunit4/SKILL.md
— runner internals.
- Cross-set:
../../../instrumentation/scenarios/launching-activities-with-activityscenario/SKILL.md
— Activity scenarios under instrumentation.
- Cross-set:
../../../fundamentals/strategies/applying-testing-strategies/SKILL.md
— when to invest in CI orchestration vs simpler tooling.