Loading...
Loading...
Use this skill to size an Android test suite using Google's small / medium / big scope vocabulary and the qualitative pyramid. Explains why most apps should hold "many small tests and relatively few big tests", how scope (small/medium/big) is orthogonal to execution location (local vs instrumented), and where the older 70/20/10 numeric ratio actually comes from. Use when the user asks "how many unit vs UI tests should I write", "what's the testing pyramid", "all my tests are instrumented and CI is slow", "test pyramid 70 20 10", "Android small medium large tests", "Robolectric counts as which layer", or mentions Unit / Component / Feature / Application / Release Candidate framing from the strategies page.
npx skill4agent add skydoves/android-testing-skills understanding-the-testing-pyramid/training/testing/fundamentals/strategies../choosing-what-to-test/SKILL.md../../strategies/applying-testing-strategies/SKILL.md/strategies../choosing-what-to-test/SKILL.mdtestImplementationandroidTestImplementation../../strategies/organizing-test-source-sets/SKILL.mdFakeMockStubSpy../../doubles/picking-test-doubles/SKILL.md../../../compose/synchronization/synchronizing-with-idle/SKILL.md../../../jvm-tests/coroutines/testing-coroutines-with-runtest/SKILL.mdsrc/test/src/androidTest/../../strategies/organizing-test-source-sets/SKILL.md/training/testing/fundamentals"Tests are also classified by their scope, or how much of the code they cover. There are three categories of test scope:
- Unit tests or small tests only verify a very small portion of the app, such as a method or class.
- End-to-end tests or big tests verify larger parts of the app at the same time, such as a whole screen or user flow.
- Medium tests are in between and check the integration between two or more units." —
developer.android.com/training/testing/fundamentals
| Scope | What it covers | Typical runner | Typical execution |
|---|---|---|---|
| Small | One method or class | JUnit4 + Mockito/MockK + fakes | |
| Medium | 2+ classes integrating, possibly Android framework via Robolectric | JUnit4 + Robolectric or in-process Android | |
| Big | A whole screen, user flow, or release-build smoke | AndroidJUnitRunner + Espresso/Compose-test/UiAutomator | |
/training/testing/fundamentals/strategies| Level | Scope | Network |
|---|---|---|
| Unit | Single method or class with minimal dependencies | None |
| Component | Module or component | None |
| Feature | Multiple components / modules | "supports mocked network access" |
| Application | Whole app, debuggable binary | n/a |
| Release Candidate | Whole release build, minified | n/a |
"Not all unit tests are local, and not all end-to-end tests run on a device. For example:
- Big local test: You can use an Android simulator that runs locally, such as Robolectric.
- Small instrumented test: You can verify that your code works well with a framework feature, such as a SQLite database." —
developer.android.com/training/testing/fundamentals
LOCAL (JVM) INSTRUMENTED (device)
SMALL pure JUnit + fakes unit test of SQLite via real DB
MEDIUM Robolectric component component test on real Android
BIG Robolectric flow Espresso / Compose UI flowsrc/test/../../strategies/organizing-test-source-sets/SKILL.md../../../jvm-tests/robolectric/using-robolectric-correctly/SKILL.md/strategies"Most apps should have many small tests and relatively few big tests, forming a pyramid shape." —developer.android.com/training/testing/fundamentals/strategies
"Key Point: In general, you should try to add tests as soon as possible in the development cycle. That typically means starting with small tests." —developer.android.com/training/testing/fundamentals/strategies
/training/testing/fundamentals/training/testing/fundamentals/strategiestasks/research/R8-android-fundamentals.mddeveloper.android.com| Layer | Wall-clock per test | Failure signal | Maintenance cost |
|---|---|---|---|
| Small | 1-50 ms | Pinpoints a class/method | Low |
| Medium (Robolectric) | 50-500 ms | Pinpoints a component | Medium |
| Big (instrumented) | 5-30 s | Says "the screen is broken" | High (flake, emulator state, animation) |
/strategies/training/testing/instrumented-tests"We recommend using instrumented tests only in cases where you must test against the behavior of a real device." —developer.android.com/training/testing/instrumented-tests
ViewModelrunBlocking { dataStore.data.first() }../../../compose/audit/auditing-compose-test-suite/SKILL.mddeveloper.android.com/training/testing/fundamentals../../doubles/picking-test-doubles/SKILL.md../../../adb/observability/extracting-logs-with-logcat/SKILL.md<!-- WRONG -->
> Per Google's developer documentation, the testing pyramid recommends
> a 70/20/10 split between unit, integration, and end-to-end tests.
> Source: developer.android.com/training/testing/fundamentals/strategies<!-- RIGHT -->
> Google's Android docs commit to a qualitative pyramid only:
> "Most apps should have many small tests and relatively few big tests"
> (developer.android.com/training/testing/fundamentals/strategies).
> The familiar 70/20/10 split comes from Software Engineering at Google,
> ch. 11 (https://abseil.io/resources/swe-book), not the Android docs.<!-- WRONG -->
> Robolectric tests are instrumented tests because they exercise the
> Android framework. Put them in src/androidTest/./training/testing/fundamentalssrc/test/<!-- RIGHT -->
> Robolectric is a local (JVM) Android simulator. By scope it can be small,
> medium, or big depending on what the test exercises. Place it in src/test/.
> Use src/androidTest/ only for tests that require a real device.<!-- WRONG -->
> Our suite has 200 small tests, 40 medium tests, 8 feature tests, and
> 3 release-candidate tests, which is a healthy pyramid.<!-- RIGHT — pick Framing A and stick with it -->
> Our suite has 200 small tests, 40 medium (Robolectric component) tests,
> and 11 big (instrumented Compose-test + Espresso) tests. Per Google's
> small/medium/big framing on /training/testing/fundamentals, the ratio
> is roughly 80/16/4, in line with the pyramid recommendation.Question → Scope
----------------------------------------------------------
"Does it depend on the Android framework?"
No → Small (src/test/)
Yes, simulated via Robolectric → Medium (src/test/)
Yes, must be a real device → Big (src/androidTest/)
"Does it exercise more than one class?"
No → Small
Yes, but no I/O / no UI → Medium
Yes, with UI rendering or system services → Big
"Does it cross a network or process boundary?"
No → Small or Medium
Yes (mocked) → Medium (Feature in Framing B)
Yes (real) → Big (Application/RC in Framing B)/strategiesdeveloper.android.com@SmallTest@MediumTest@LargeTestandroidx.test.filters@SmallTest@MediumTest@LargeTestsrc/androidTest/src/test/developer.android.comdeveloper.android.com/training/testing/fundamentalsdeveloper.android.com/training/testing/fundamentals/strategiesdeveloper.android.com/training/testing/instrumented-teststasks/research/R8-android-fundamentals.mdandroidx.test.filters@SmallTest@MediumTest@LargeTest../choosing-what-to-test/SKILL.md../../doubles/picking-test-doubles/SKILL.md../../strategies/applying-testing-strategies/SKILL.md../../strategies/organizing-test-source-sets/SKILL.md../../../jvm-tests/robolectric/using-robolectric-correctly/SKILL.md../../../jvm-tests/runner/configuring-junit4-on-android/SKILL.md../../../instrumentation/runner/running-instrumented-tests-with-androidjunit4/SKILL.md../../../compose/audit/auditing-compose-test-suite/SKILL.md