understanding-the-testing-pyramid

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Understanding the Testing Pyramid — Pick the 3-Layer Framing and Stick to It

理解测试金字塔——选定 3 层框架并坚持使用

Android test suites slow CI and rot when teams ship one big instrumented test per behavior and skip unit coverage of the underlying logic. Google's
/training/testing/fundamentals
page frames test sizing with three scopes — small, medium, big — and the
/strategies
page sketches a qualitative pyramid on top of that. This skill encodes the 3-layer framing, names the alternative 5-layer framing so authors do not blend them, and is honest about where the famous numeric ratios come from. Subsequent skills (
../choosing-what-to-test/SKILL.md
,
../../strategies/applying-testing-strategies/SKILL.md
) assume this vocabulary.
Android 测试套件会拖慢 CI;当团队为每种行为只发布一个大型 instrumented test,并跳过底层逻辑的 unit coverage 时,测试套件还会逐渐腐化。Google 的
/training/testing/fundamentals
页面用三个 scope——small、medium、big——来划分测试规模,
/strategies
页面则在此基础上勾勒了一个定性金字塔。此 skill 固化了 3 层框架,同时为替代的 5 层框架命名,以避免作者将两者混用,并如实说明著名数字比例的来源。后续 skills(
../choosing-what-to-test/SKILL.md
、
../../strategies/applying-testing-strategies/SKILL.md
)都假定使用这套术语。

When to use this skill

何时使用此 skill

  • The user asks "how many unit vs UI tests should I write" or quotes a 70/20/10 / 80/15/5 ratio and asks if it is in Google's docs.
  • The user calls every test "an integration test" / "an end-to-end test" and the agent needs a vocabulary to sort them.
  • The user reports "CI is slow" / "the suite takes 40 minutes" and wants to know whether to delete instrumented tests.
  • The user asks whether Robolectric is "a unit test" or "an integration test" — it is a medium-local test in Google's framing.
  • The user is starting a new module and asks where to begin (answer: small tests first, per
    /strategies
    ).
  • 用户询问“我应该编写多少 unit 测试与 UI 测试”,或引用 70/20/10 / 80/15/5 比例并询问它是否来自 Google 文档。
  • 用户把每个 test 都称作“integration test”或“end-to-end test”,而 agent 需要一套术语来对它们进行归类。
  • 用户报告“CI 很慢”或“测试套件需要 40 分钟”,并想知道是否应该删除 instrumented tests。
  • 用户询问 Robolectric 是“unit test”还是“integration test”——在 Google 的框架中,它是一个 medium-local test。
  • 用户开始开发一个新模块,并询问应从哪里入手(答案:按照
    /strategies
    的建议,先从 small tests 开始)。

When NOT to use this skill

何时不应使用此 skill

  • The user wants to know what to put in each test (which screens, which ViewModels) — use
    ../choosing-what-to-test/SKILL.md
    .
  • The user wants to wire
    testImplementation
    vs
    androidTestImplementation
    Gradle configurations — use
    ../../strategies/organizing-test-source-sets/SKILL.md
    .
  • The user wants to pick between
    Fake
    /
    Mock
    /
    Stub
    /
    Spy
    — use
    ../../doubles/picking-test-doubles/SKILL.md
    .
  • The user is debugging a single flaky test and wants to fix it — start with
    ../../../compose/synchronization/synchronizing-with-idle/SKILL.md
    or
    ../../../jvm-tests/coroutines/testing-coroutines-with-runtest/SKILL.md
    .
  • 用户想知道每个 test 中应该包含什么内容,即测试哪些 screens、哪些 ViewModels——请使用
    ../choosing-what-to-test/SKILL.md
    。
  • 用户想配置
    testImplementation
    与
    androidTestImplementation
    Gradle configurations——请使用
    ../../strategies/organizing-test-source-sets/SKILL.md
    。
  • 用户想在
    Fake
    /
    Mock
    /
    Stub
    /
    Spy
    之间做选择——请使用
    ../../doubles/picking-test-doubles/SKILL.md
    。
  • 用户正在调试单个 flaky test 并希望修复它——请先使用
    ../../../compose/synchronization/synchronizing-with-idle/SKILL.md
    或
    ../../../jvm-tests/coroutines/testing-coroutines-with-runtest/SKILL.md
    。

Prerequisites

前置条件

  • A module with at least one test class so the discussion is concrete.
  • Familiarity with the
    src/test/
    (local, JVM) and
    src/androidTest/
    (instrumented) split. If unfamiliar, read
    ../../strategies/organizing-test-source-sets/SKILL.md
    first.
  • An understanding that "scope" (small/medium/big) and "execution location" (local/instrumented) are two independent axes per Google.
  • 一个至少包含一个 test class 的 module,以便讨论能够具体化。
  • 熟悉
    src/test/
    (local、JVM)与
    src/androidTest/
    (instrumented)的划分。如果尚不熟悉,请先阅读
    ../../strategies/organizing-test-source-sets/SKILL.md
    。
  • 理解按照 Google 的说法,“scope”(small/medium/big)与“execution location”(local/instrumented)是两个彼此独立的轴。

The two framings (pick exactly one per skill / per design doc)

两种框架(每个 skill / 每份 design doc 恰好选择一种)

Google publishes two framings on adjacent pages. They are not contradictory; they are different views.
Google 在相邻的页面上发布了两种框架。两者并不矛盾;它们是不同视角。

Framing A — Small / Medium / Big (3 layers) ← this skill recommends this one

框架 A——Small / Medium / Big(3 层) ← 此 skill 推荐使用这一种

From
/training/testing/fundamentals
("Types of tests in Android" → "By Scope"):
"Tests are also classified by their scope, or how much of the code they cover. There are three categories of test scope:
  • Unit tests or small tests only verify a very small portion of the app, such as a method or class.
  • End-to-end tests or big tests verify larger parts of the app at the same time, such as a whole screen or user flow.
  • Medium tests are in between and check the integration between two or more units." —
    developer.android.com/training/testing/fundamentals
ScopeWhat it coversTypical runnerTypical execution
SmallOne method or classJUnit4 + Mockito/MockK + fakes
src/test/
(JVM)
Medium2+ classes integrating, possibly Android framework via RobolectricJUnit4 + Robolectric or in-process Android
src/test/
(host) OR
src/androidTest/
(instrumented)
BigA whole screen, user flow, or release-build smokeAndroidJUnitRunner + Espresso/Compose-test/UiAutomator
src/androidTest/
内容来自
/training/testing/fundamentals
(“Types of tests in Android” → “By Scope”):
“测试还可以按 scope 分类,也就是按它们覆盖的代码量分类。测试 scope 分为三类:
  • Unit tests 或 small tests 只验证应用中很小的一部分,例如一个方法或一个类。
  • End-to-end tests 或 big tests 同时验证应用中更大的部分,例如整个 screen 或 user flow。
  • Medium tests 介于两者之间,用于检查两个或更多 unit 之间的集成。” —
    developer.android.com/training/testing/fundamentals
Scope覆盖内容典型 runner典型执行方式
Small一个方法或一个类JUnit4 + Mockito/MockK + fakes
src/test/
(JVM)
Medium两个或更多类进行集成,可能通过 Robolectric 使用 Android frameworkJUnit4 + Robolectric 或进程内 Android
src/test/
(host)或
src/androidTest/
(instrumented)
Big整个 screen、user flow 或 release-build smoke testAndroidJUnitRunner + Espresso/Compose-test/UiAutomator
src/androidTest/

Framing B — Unit / Component / Feature / Application / Release Candidate (5 layers)

框架 B——Unit / Component / Feature / Application / Release Candidate(5 层)

From
/training/testing/fundamentals/strategies
(the "scalable strategy" table):
LevelScopeNetwork
UnitSingle method or class with minimal dependenciesNone
ComponentModule or componentNone
FeatureMultiple components / modules"supports mocked network access"
ApplicationWhole app, debuggable binaryn/a
Release CandidateWhole release build, minifiedn/a
Most teams stay with the 3-layer Framing A because it is the simpler, more cited shape. Reach for Framing B only when designing a multi-team CI strategy where the extra granularity earns its keep. MUST NOT blend the two in a single doc — pick one and stick with it.
内容来自
/training/testing/fundamentals/strategies
(“scalable strategy”表格):
LevelScopeNetwork
Unit依赖极少,覆盖单个方法或类无
Component覆盖 module 或 component无
Feature覆盖多个 components/modules“支持模拟的网络访问”
Application覆盖整个 app,使用可调试二进制文件不适用
Release Candidate覆盖整个 release build,并经过压缩不适用
大多数团队会继续采用 3 层的框架 A,因为它更简单,也是引用更广泛的结构。只有在设计多团队 CI strategy、且更细粒度确实能带来价值时,才应采用框架 B。MUST NOT 在同一份文档中混用这两种框架——选定一种并坚持使用。

Scope is orthogonal to execution location

Scope 与执行位置相互独立

The single most important quote on the page, and the one the agent should cite when a developer says "all my tests are instrumented":
"Not all unit tests are local, and not all end-to-end tests run on a device. For example:
  • Big local test: You can use an Android simulator that runs locally, such as Robolectric.
  • Small instrumented test: You can verify that your code works well with a framework feature, such as a SQLite database." —
    developer.android.com/training/testing/fundamentals
Two axes:
                 LOCAL (JVM)              INSTRUMENTED (device)
SMALL    pure JUnit + fakes          unit test of SQLite via real DB
MEDIUM   Robolectric component      component test on real Android
BIG      Robolectric flow           Espresso / Compose UI flow
Robolectric is a local Android simulator. It is not "an instrumented test" — it runs on the JVM under
src/test/
. See
../../strategies/organizing-test-source-sets/SKILL.md
and the cross-category skill
../../../jvm-tests/robolectric/using-robolectric-correctly/SKILL.md
.
页面上最重要的引述,也是当开发者说“我的所有测试都是 instrumented”时 agent 应当引用的一段:
“并非所有 unit tests 都在本地运行,也并非所有 end-to-end tests 都在设备上运行。例如:
  • Big local test:可以使用在本地运行的 Android 模拟器,例如 Robolectric。
  • Small instrumented test:可以验证代码与某项 framework feature 的配合是否正常,例如 SQLite database。” —
    developer.android.com/training/testing/fundamentals
两个轴:
                 LOCAL (JVM)              INSTRUMENTED (device)
SMALL    pure JUnit + fakes          unit test of SQLite via real DB
MEDIUM   Robolectric component      component test on real Android
BIG      Robolectric flow           Espresso / Compose UI flow
Robolectric 是一个 local Android simulator。它不是“instrumented test”——它在 JVM 的
src/test/
下运行。请参阅
../../strategies/organizing-test-source-sets/SKILL.md
,以及跨分类 skill
../../../jvm-tests/robolectric/using-robolectric-correctly/SKILL.md
。

The qualitative pyramid

定性金字塔

The
/strategies
page only commits to a qualitative shape:
"Most apps should have many small tests and relatively few big tests, forming a pyramid shape." —
developer.android.com/training/testing/fundamentals/strategies
"Key Point: In general, you should try to add tests as soon as possible in the development cycle. That typically means starting with small tests." —
developer.android.com/training/testing/fundamentals/strategies
That is the entirety of Google's current quantitative claim: "many small, few big". No ratio.
/strategies
页面只承诺了一种定性结构:
“大多数 app 应包含许多 small tests 和相对较少的 big tests,从而形成金字塔形状。” —
developer.android.com/training/testing/fundamentals/strategies
“要点: 通常,你应尽量在开发生命周期的早期就添加测试。这通常意味着从 small tests 开始。” —
developer.android.com/training/testing/fundamentals/strategies
这就是 Google 当前关于数量比例的全部说法:“many small, few big”。没有具体的比例数字。

Where the 70/20/10 ratio actually comes from

70/20/10 比例的实际来源

The "70 percent unit, 20 percent integration, 10 percent end-to-end" rule is widely cited but is NOT present on the current
/training/testing/fundamentals
or
/training/testing/fundamentals/strategies
pages (as of 2026-05-06; verified in
tasks/research/R8-android-fundamentals.md
). The number is from Google's own engineering literature — Software Engineering at Google, chapter 11 — not the Android training pages.
MUST NOT attribute "70/20/10" to
developer.android.com
. The numeric ratio is a community heuristic popularised in Software Engineering at Google (https://abseil.io/resources/swe-book, ch. 11) — paraphrase carefully, do NOT present any specific sentence as a verbatim quote unless you confirm it line-by-line in the book. The book frames the split as a heuristic, not a law: different domains warrant different ratios; see "When to break the pyramid" below.
“70 percent unit、20 percent integration、10 percent end-to-end”规则被广泛引用,但不存在于当前的
/training/testing/fundamentals
或
/training/testing/fundamentals/strategies
页面(截至 2026-05-06;已在
tasks/research/R8-android-fundamentals.md
中核实)。这个数字来自 Google 自己的工程著作——Software Engineering at Google 第 11 章——而不是 Android training 页面。
MUST NOT 将“70/20/10”数字比例归因于
developer.android.com
。这一数字比例是 Software Engineering at Google(https://abseil.io/resources/swe-book,第 11 章)中推广的一种社区启发式规则——应谨慎转述,除非已逐行确认书中原文,否则不要将任何具体句子表述为逐字引用。该书将这种划分视为启发式规则,而非法律:不同领域适合不同比例;请参阅下文“何时有意打破金字塔”。

Cost / speed tradeoff

成本 / 速度权衡

The reason the pyramid bottom is wide:
LayerWall-clock per testFailure signalMaintenance cost
Small1-50 msPinpoints a class/methodLow
Medium (Robolectric)50-500 msPinpoints a componentMedium
Big (instrumented)5-30 sSays "the screen is broken"High (flake, emulator state, animation)
A flat or inverted pyramid (lots of big tests, few small) produces:
  • Slow CI (10x to 100x slower than a unit-heavy suite of equivalent coverage).
  • High flake rate; flake correlates with test size and external dependencies (
    /strategies
    notes "no network access" for unit and component layers).
  • Vague failure signal — "screen X broken" instead of "function Y returned the wrong value for input Z".
  • Painful refactors — UI tests assert on rendered text and break on every copy change.
/training/testing/instrumented-tests
makes the price explicit:
"We recommend using instrumented tests only in cases where you must test against the behavior of a real device." —
developer.android.com/training/testing/instrumented-tests
That is Google's "use big tests as a last resort" stance, in their own words.
金字塔底部之所以更宽,原因如下:
层级每项测试的 wall-clock失败信号维护成本
Small1-50 ms可精确定位到类或方法低
Medium(Robolectric)50-500 ms可精确定位到 component中
Big(instrumented)5-30 s只能表明“screen X 坏了”高(flake、emulator 状态、动画)
扁平或倒置的金字塔(大量 big tests、少量 small tests)会导致:
  • CI 变慢(相比覆盖范围相当、以 unit 为主的测试套件,慢 10 到 100 倍)。
  • flake rate 较高;flake 与测试规模及外部依赖项相关(
    /strategies
    指出 unit 和 component 层应“无网络访问”)。
  • 失败信号模糊——“screen X 坏了”,而不是“function Y 对输入 Z 返回了错误值”。
  • 重构痛苦——UI tests 会断言渲染后的文本,因此每次修改文案都会导致测试失败。
/training/testing/instrumented-tests
明确说明了这项代价:
“我们建议仅在必须针对真实设备行为进行测试的情况下使用 instrumented tests。” —
developer.android.com/training/testing/instrumented-tests
这就是 Google 所说的“将 big tests 作为最后手段”的立场。

When to break the pyramid intentionally

何时有意打破金字塔

The pyramid is a default, not a law. Break it when:
  1. The product IS the integration. A payments SDK that primarily wires three vendor SDKs together has more value in feature/application tests than in unit tests of glue code. The unit tests would be tautological mock verifications.
  2. The framework owns the logic. A pure CRUD screen with a DataStore-backed
    ViewModel
    and stock Material widgets has very little to unit-test. Big tests catch wiring; small tests would just be testing
    runBlocking { dataStore.data.first() }
    .
  3. Determinism is the point. A render-correctness suite (screenshot tests, animation timing) lives at the big-test layer because that is where the bug surfaces. See the cross-category skill
    ../../../compose/audit/auditing-compose-test-suite/SKILL.md
    .
  4. Cost economics flip. On Firebase Test Lab / Gradle Managed Devices the per-minute cost of a big test may be small enough that the maintenance cost dominates anyway — and unit tests still win on flake.
What does NOT justify breaking the pyramid:
  • "Unit tests are too hard to set up" — that is a signal of bad architecture, not a reason to pile on big tests. See
    developer.android.com/training/testing/fundamentals
    ("With a testable app architecture …").
  • "We can't mock X" — use a fake instead of a mock. See
    ../../doubles/picking-test-doubles/SKILL.md
    .
  • "It works on my machine, just ship more device tests" — the flake will follow the suite. Diagnose with
    ../../../adb/observability/extracting-logs-with-logcat/SKILL.md
    .
金字塔是一种默认做法,而不是法律。以下情况下可以打破它:
  1. 产品本身就是集成。 一个主要用于连接三个 vendor SDKs 的支付 SDK,其 feature/application tests 比针对胶水代码的 unit tests 更有价值。那些 unit tests 只会变成同义反复的 mock 验证。
  2. 逻辑由 framework 掌控。 对于一个使用 DataStore-backed
    ViewModel
    和标准 Material widgets 的纯 CRUD screen,几乎没有值得 unit-test 的逻辑。Big tests 能捕获连接装配问题;small tests 只会测试
    runBlocking { dataStore.data.first() }
    。
  3. 确定性正是目标。 一套用于验证渲染正确性的测试(screenshot tests、动画时序测试)属于 big-test 层,因为问题会在这一层暴露。请参阅跨分类 skill
    ../../../compose/audit/auditing-compose-test-suite/SKILL.md
    。
  4. 成本经济性发生反转。 在 Firebase Test Lab / Gradle Managed Devices 上,每分钟执行 big test 的成本可能足够低,以至于维护成本最终占据主导——而且 unit tests 在 flake 方面仍然更优。
以下理由不能成为打破金字塔的正当依据:
  • “Unit tests 太难搭建”——这是架构不佳的信号,而不是不断添加 big tests 的理由。请参阅
    developer.android.com/training/testing/fundamentals
    (“With a testable app architecture …”)。
  • “我们无法 mock X”——请改用 fake,而不是 mock。请参阅
    ../../doubles/picking-test-doubles/SKILL.md
    。
  • “在我机器上能运行,直接多写设备测试吧”——flake 会追随整个测试套件。请使用
    ../../../adb/observability/extracting-logs-with-logcat/SKILL.md
    进行诊断。

Patterns

模式

Pattern: WRONG vs RIGHT — citing the 70/20/10 ratio

模式:错误 vs 正确——引用 70/20/10 比例

markdown
<!-- WRONG -->
> Per Google's developer documentation, the testing pyramid recommends
> a 70/20/10 split between unit, integration, and end-to-end tests.
> Source: developer.android.com/training/testing/fundamentals/strategies
WRONG because: that ratio is not on the strategies page as of 2026-05-06. The page only says "many small, few big" qualitatively. Citing developer.android.com here is fabrication.
markdown
<!-- RIGHT -->
> Google's Android docs commit to a qualitative pyramid only:
> "Most apps should have many small tests and relatively few big tests"
> (developer.android.com/training/testing/fundamentals/strategies).
> The familiar 70/20/10 split comes from Software Engineering at Google,
> ch. 11 (https://abseil.io/resources/swe-book), not the Android docs.
markdown
<!-- WRONG -->
> Per Google's developer documentation, the testing pyramid recommends
> a 70/20/10 split between unit, integration, and end-to-end tests.
> Source: developer.android.com/training/testing/fundamentals/strategies
错误,因为:截至 2026-05-06,该比例并不在 strategies 页面上。页面只以定性方式说明“many small, few big”。在这里引用 developer.android.com 属于捏造。
markdown
<!-- RIGHT -->
> Google's Android docs commit to a qualitative pyramid only:
> "Most apps should have many small tests and relatively few big tests"
> (developer.android.com/training/testing/fundamentals/strategies).
> The familiar 70/20/10 split comes from Software Engineering at Google,
> ch. 11 (https://abseil.io/resources/swe-book), not the Android docs.

Pattern: WRONG vs RIGHT — labeling Robolectric

模式:错误 vs 正确——为 Robolectric 分类

markdown
<!-- WRONG -->
> Robolectric tests are instrumented tests because they exercise the
> Android framework. Put them in src/androidTest/.
WRONG because: Robolectric runs on the JVM with no Android device or emulator. Per
/training/testing/fundamentals
("Big local test: You can use an Android simulator that runs locally, such as Robolectric"), Robolectric is a local test, in
src/test/
, regardless of scope.
markdown
<!-- RIGHT -->
> Robolectric is a local (JVM) Android simulator. By scope it can be small,
> medium, or big depending on what the test exercises. Place it in src/test/.
> Use src/androidTest/ only for tests that require a real device.
markdown
<!-- WRONG -->
> Robolectric tests are instrumented tests because they exercise the
> Android framework. Put them in src/androidTest/.
错误,因为:Robolectric 在 JVM 上运行,不需要 Android 设备或 emulator。根据
/training/testing/fundamentals
(“Big local test: You can use an Android simulator that runs locally, such as Robolectric”),Robolectric 是一个 local test,位于
src/test/
,无论其 scope 如何。
markdown
<!-- RIGHT -->
> Robolectric is a local (JVM) Android simulator. By scope it can be small,
> medium, or big depending on what the test exercises. Place it in src/test/.
> Use src/androidTest/ only for tests that require a real device.

Pattern: WRONG vs RIGHT — pyramid framing in one design doc

模式:错误 vs 正确——在一份 design doc 中使用金字塔框架

markdown
<!-- WRONG -->
> Our suite has 200 small tests, 40 medium tests, 8 feature tests, and
> 3 release-candidate tests, which is a healthy pyramid.
WRONG because: this blends Framing A ("small/medium") with Framing B ("feature/release candidate"). The reader cannot tell whether "small" means Framing A's small or Framing B's unit.
markdown
<!-- RIGHT — pick Framing A and stick with it -->
> Our suite has 200 small tests, 40 medium (Robolectric component) tests,
> and 11 big (instrumented Compose-test + Espresso) tests. Per Google's
> small/medium/big framing on /training/testing/fundamentals, the ratio
> is roughly 80/16/4, in line with the pyramid recommendation.
markdown
<!-- WRONG -->
> Our suite has 200 small tests, 40 medium tests, 8 feature tests, and
> 3 release-candidate tests, which is a healthy pyramid.
错误,因为这混用了框架 A(“small/medium”)与框架 B(“feature/release candidate”)。读者无法判断“small”指的是框架 A 的 small,还是框架 B 的 unit。
markdown
<!-- RIGHT — pick Framing A and stick to it -->
> Our suite has 200 small tests, 40 medium (Robolectric component) tests,
> and 11 big (instrumented Compose-test + Espresso) tests. Per Google's
> small/medium/big framing on /training/testing/fundamentals, the ratio
> is roughly 80/16/4, in line with the pyramid recommendation.

Decision matrix — what scope should a new test be?

决策矩阵——新 test 应采用什么 scope?

Question                                         → Scope
----------------------------------------------------------
"Does it depend on the Android framework?"
  No                                             → Small (src/test/)
  Yes, simulated via Robolectric                 → Medium (src/test/)
  Yes, must be a real device                     → Big (src/androidTest/)
"Does it exercise more than one class?"
  No                                             → Small
  Yes, but no I/O / no UI                        → Medium
  Yes, with UI rendering or system services      → Big
"Does it cross a network or process boundary?"
  No                                             → Small or Medium
  Yes (mocked)                                   → Medium (Feature in Framing B)
  Yes (real)                                     → Big (Application/RC in Framing B)
Question                                         → Scope
----------------------------------------------------------
"Does it depend on the Android framework?"
  No                                             → Small (src/test/)
  Yes, simulated via Robolectric                 → Medium (src/test/)
  Yes, must be a real device                     → Big (src/androidTest/)
"Does it exercise more than one class?"
  No                                             → Small
  Yes, but no I/O / no UI                        → Medium
  Yes, with UI rendering or system services      → Big
"Does it cross a network or process boundary?"
  No                                             → Small or Medium
  Yes (mocked)                                   → Medium (Feature in Framing B)
  Yes (real)                                     → Big (Application/RC in Framing B)

Mandatory rules

强制规则

  • MUST use the 3-layer "small / medium / big" framing in default skill copy and design docs. Reach for the 5-layer Unit/Component/Feature/Application/Release Candidate framing only when explicitly invoking the
    /strategies
    page.
  • MUST NOT blend the two framings in a single document. Pick one.
  • MUST NOT attribute the "70/20/10" numeric ratio to
    developer.android.com
    . Cite Software Engineering at Google (https://abseil.io/resources/swe-book).
  • MUST NOT describe Robolectric as "an instrumented test". It is a local Android simulator that may carry any scope.
  • MUST label every new test with its scope in a comment or naming convention (
    @SmallTest
    ,
    @MediumTest
    ,
    @LargeTest
    from
    androidx.test.filters
    ) so CI can shard accordingly.
  • PREFERRED: start a new module with small tests first, per Google's "starting with small tests" guidance, and only add medium/big tests where small ones cannot reach.
  • PREFERRED: when CI is slow, delete redundant big tests before adding more small ones — the pyramid tightens by removing the top, not just by widening the base.
  • MUST 在默认 skill 文案和 design docs 中使用 3 层的“small / medium / big”框架。只有在明确调用
    /strategies
    页面时,才采用 5 层的 Unit/Component/Feature/Application/Release Candidate 框架。
  • MUST NOT 在同一份文档中混用这两种框架。选定一种并坚持使用。
  • MUST NOT 将“70/20/10”数字比例归因于
    developer.android.com
    。应引用 Software Engineering at Google(https://abseil.io/resources/swe-book)。
  • MUST NOT 将 Robolectric 描述为“instrumented test”。它是一个 local Android simulator,可以承载任何 scope。
  • MUST 通过注释或命名约定(
    androidx.test.filters
    中的
    @SmallTest
    、
    @MediumTest
    、
    @LargeTest
    )标注每个新 test 的 scope,以便 CI 按层进行分片。
  • PREFERRED: 按照 Google 关于“从 small tests 开始”的建议,新 module 应优先从 small tests 入手,仅在 small tests 无法覆盖时才添加 medium/big tests。
  • PREFERRED: 当 CI 很慢时,应先删除冗余的 big tests,再添加更多 small tests——这是通过移除金字塔顶部来收紧结构,而不只是拓宽底部。

Verification

验证

  • Every test class in the module is reachable via one of
    @SmallTest
    ,
    @MediumTest
    ,
    @LargeTest
    (or by source-set placement).
  • No test in
    src/androidTest/
    could equivalently run as a Robolectric host test in
    src/test/
    without losing fidelity. (If yes, move it.)
  • CI logs report unit-test wall time < instrumented-test wall time per layer per change.
  • No design doc cites "70/20/10" attributed to
    developer.android.com
    .
  • No design doc blends Framing A and Framing B vocabulary.
  • The team's testing strategy doc names exactly one framing in its glossary.
  • module 中的每个 test class 都可以通过
    @SmallTest
    、
    @MediumTest
    、
    @LargeTest
    中的一种访问(或可通过 source-set placement 确定)。
  • src/androidTest/
    中没有任何 test,可以在不损失测试保真度的情况下改为在
    src/test/
    中作为 Robolectric host test 运行。(如果可以,则应移动。)
  • CI logs 会报告每次变更中各层的 unit-test wall time 小于 instrumented-test wall time。
  • 没有 design doc 将“70/20/10”归因于
    developer.android.com
    。
  • 没有 design doc 混用框架 A 与框架 B 的术语。
  • 团队的 testing strategy doc 在其术语表中只命名了一种框架。

References

参考资料

  • developer.android.com/training/testing/fundamentals
    — small/medium/big scope, "Not all unit tests are local" quote, testable architecture rationale.
  • developer.android.com/training/testing/fundamentals/strategies
    — qualitative pyramid ("many small, few big"), 5-level Unit/Component/Feature/Application/Release Candidate table, "starting with small tests" guidance.
  • developer.android.com/training/testing/instrumented-tests
    — "instrumented tests only in cases where you must test against the behavior of a real device".
  • Software Engineering at Google, ch. 11 — origin of the 70/20/10 ratio. https://abseil.io/resources/swe-book
  • tasks/research/R8-android-fundamentals.md
    — verbatim quotes, version-sensitive notes, the "70/20/10 not on the page" finding.
  • androidx.test.filters
    —
    @SmallTest
    /
    @MediumTest
    /
    @LargeTest
    annotations for runtime sharding.
  • Sibling skills:
    ../choosing-what-to-test/SKILL.md
    ,
    ../../doubles/picking-test-doubles/SKILL.md
    ,
    ../../strategies/applying-testing-strategies/SKILL.md
    ,
    ../../strategies/organizing-test-source-sets/SKILL.md
    .
  • Cross-category:
    ../../../jvm-tests/robolectric/using-robolectric-correctly/SKILL.md
    ,
    ../../../jvm-tests/runner/configuring-junit4-on-android/SKILL.md
    ,
    ../../../instrumentation/runner/running-instrumented-tests-with-androidjunit4/SKILL.md
    ,
    ../../../compose/audit/auditing-compose-test-suite/SKILL.md
    .
  • developer.android.com/training/testing/fundamentals
    ——small/medium/big scope、“Not all unit tests are local”引述,以及可测试架构的原理。
  • developer.android.com/training/testing/fundamentals/strategies
    ——定性金字塔(“many small, few big”)、5 层 Unit/Component/Feature/Application/Release Candidate 表格,以及“starting with small tests”指南。
  • developer.android.com/training/testing/instrumented-tests
    ——“仅在必须针对真实设备行为进行测试时使用 instrumented tests”。
  • Software Engineering at Google,第 11 章——70/20/10 比例的来源。https://abseil.io/resources/swe-book
  • tasks/research/R8-android-fundamentals.md
    ——逐字引述、对版本敏感的信息,以及“70/20/10 不在该页面上”的核实结果。
  • androidx.test.filters
    ——用于运行时分片的
    @SmallTest
    /
    @MediumTest
    /
    @LargeTest
    注解。
  • Sibling skills:
    ../choosing-what-to-test/SKILL.md
    、
    ../../doubles/picking-test-doubles/SKILL.md
    、
    ../../strategies/applying-testing-strategies/SKILL.md
    、
    ../../strategies/organizing-test-source-sets/SKILL.md
    。
  • Cross-category:
    ../../../jvm-tests/robolectric/using-robolectric-correctly/SKILL.md
    、
    ../../../jvm-tests/runner/configuring-junit4-on-android/SKILL.md
    、
    ../../../instrumentation/runner/running-instrumented-tests-with-androidjunit4/SKILL.md
    、
    ../../../compose/audit/auditing-compose-test-suite/SKILL.md
    。