Back to Details

android-benchmark-comparison

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Android benchmark comparison

Android 基准测试对比

Core principle

核心原则

Treat a physical Android configuration comparison as a reproducible experiment: verify comparable workloads and device conditions before interpreting a ranking or choosing a default.
将实体Android设备的配置对比视为可复现实验：在解读排名或选择默认配置前，需验证工作负载与设备状态具备可比性。

Procedure

操作流程

  1. State the decision, configurations, workloads, metric definitions, and repetitions. Preserve exact build identity, configuration, raw results, and traces; then verify every intended case and iteration ran. Distinguish missing, failed, and excluded runs; do not compare only the fastest survivors.
  2. Control and record relevant device conditions, including device model and state, thermal and power mode, display brightness, background load, and network or input conditions. Keep device-specific commands and CPU masks in the project's runbook.
  3. Balance or reverse run order and repeat the comparison. Report the spread and whether the ordering holds; do not discard slow iterations after seeing the result.
  4. When rankings reverse or variability is material, investigate before making the decision. Use traces to check the measured interval and plausible causes such as CPU placement, contention, or thermal state. A fixed- performance setting does not prove CPU placement. If affinity is used, discover the device topology, verify placement during the measured trace interval, label the outcome a controlled comparison, and restore device settings afterwards.
  5. Calculate summaries from unrounded observations, then round only for presentation. Name the aggregation explicitly: the mean of per-run percentiles is not a percentile of pooled observations. Choose an aggregation that answers the stated decision; do not prescribe one statistic universally.
  6. Separate controlled-experiment evidence from normal user performance. If several conditions changed together, report the comparison as more controlled but do not attribute its whole difference to one control. Use CPU frame-duration evidence to inform a visual quality/performance decision, without claiming it measures GPU shader time.
  7. Finish with the raw-evidence location, completed-case counts, variability, trace findings, controls and restoration status, plus the bounded decision or remaining uncertainty.
  1. 明确说明决策目标、配置项、工作负载、指标定义与重复次数。保留确切的构建版本标识、配置、原始结果与trace；随后验证所有预期用例与迭代均已执行。区分缺失、失败与被排除的运行结果；不得仅对比筛选出的最快结果。
  2. 控制并记录相关设备状态，包括设备型号与状态、散热与功耗模式、屏幕亮度、后台负载，以及网络或输入条件。将设备专属命令与CPU掩码存入项目的运行手册中。
  3. 平衡或打乱运行顺序并重复对比实验。报告结果的离散程度以及排名是否稳定；不得在看到结果后舍弃较慢的迭代数据。
  4. 当排名出现反转或差异较大时，需先调查原因再做决策。使用trace检查测量区间以及可能的诱因，例如CPU调度、资源争用或散热状态。固定性能设置并不能证明CPU调度合理。如果使用了CPU亲和性设置，需先了解设备拓扑结构，在测量的trace区间内验证调度情况，将结果标记为受控对比，并在实验后恢复设备设置。
  5. 基于未舍入的原始观测值计算汇总数据，仅在展示时进行舍入。明确说明聚合方式：每次运行的百分位数的平均值，并非合并所有观测值后的百分位数。选择能够支撑决策目标的聚合方式；不要通用地规定某一种统计指标。
  6. 区分受控实验结论与普通用户场景下的性能表现。如果多个条件同时发生变化，应将对比结果标注为更高程度的受控实验，但不得将全部差异归因于某一个控制变量。使用CPU帧时长数据辅助视觉质量/性能决策，但不得声称其可以衡量GPU着色器运行时间。
  7. 最后需说明原始证据的存放位置、已完成用例数量、结果离散程度、trace调查结论、控制措施与恢复状态，以及有适用边界的决策结论或尚存的不确定性。

Boundaries

适用边界

  • A single stable benchmark run can support a narrow observation, but not a robust configuration ranking.
  • Do not turn a device-specific CPU mask, brightness value, iteration count, or summary statistic into a permanent default.
  • When traces or repeat coverage cannot resolve a reversal, keep the default unchanged or make a provisional decision with that limitation explicit.
  • 单次稳定的基准测试运行仅能支撑窄范围的观测结论，不足以作为可靠的配置排名依据。
  • 不要将设备专属的CPU掩码、亮度值、迭代次数或汇总统计方式设为永久默认值。
  • 当trace或重复实验无法解释排名反转时，保持原有默认配置不变，或做出临时决策并明确说明该局限性。