litert-google-tensor

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

LiteRT on Google Tensor: compile, benchmark, package for apps

Google Tensor 上的 LiteRT:编译、基准测试并为应用打包

Three scriptable workflows for Pixel TPUs:
  1. Compile a plain
    .tflite
    into a Google Tensor AOT-compiled
    .tflite
    with the public LiteRT Python API (
    ai_edge_litert.aot
    ), locally on Linux
    x86_64
    or on a Google Colab VM (
    --colab
    , via
    google-colab-cli
    ) when the user is on macOS/Windows or lacks a Linux box.
  2. Benchmark a compiled model on a connected Pixel with LiteRT's
    benchmark_model
    , NPU-only, optionally next to a CPU baseline, including TPU tile time (pure TPU compute time) and TPU energy per inference.
  3. Package compiled builds for an Android (Gradle) app as a Google Play AI pack with the Google Tensor NPU runtime and a Kotlin loader (or, for an app that already ships Google Tensor builds, only the files to replace), apply the plan to the project when the user agrees, and check the app on a Pixel.
Everything runs synchronously in the user's terminal; the scripts never start background processes and always clean up what they pushed to the phone or allocated on Colab. All paths below are relative to this skill's directory (
SKILL_DIR
).
PathPurpose
assets/versions.jsonLiteRT versions the scripts use (single source)
scripts/check_compiled.pyIs this
.tflite
already Tensor-compiled?
scripts/compile_model.pyAOT compile (local Linux
x86_64
or
--colab
)
scripts/benchmark_on_device.pyadb push -> benchmark -> parse -> cleanup
scripts/package_ai_pack.pyCompiled builds -> AI pack + Gradle modules for an app, or an update of one
scripts/install_and_verify.pyBuild the app, install it like Google Play, read the NPU/CPU result
assets/templates/Gradle, Kotlin,
INTEGRATION.md
and
UPDATE.md
templates (
package_ai_pack.py
)
references/compile-flags.mdAPI, compiler flags, script options,
--colab
references/benchmark-flags.mdWhat the benchmark runs, binaries, writing the report, example output
references/app-integration.mdPackaging for an app: contents, checks, delivery modes, applying the plan, checking on a Pixel
references/troubleshooting.mdSymptom -> fix tables
references/contributing.mdMaintainers: adopting a new LiteRT release
Every script supports
--help
and
--dry-run
and is Python 3 standard library only, so it runs with any
python3
, with two exceptions that import LiteRT: a local compile (
compile_model.py
without
--colab
,
--dry-run
or
--print-install
) must use the virtualenv's interpreter (
<venv>/bin/python SKILL_DIR/scripts/compile_model.py ...
, or activate the venv first), and a real
package_ai_pack.py
run (not
--dry-run
or
--check-project
) needs a Python with
ai-edge-litert
(the compile venv works; the Beta SDK is not needed).
面向 Pixel TPU 的三种可脚本化工作流:
  1. 编译:使用公共 LiteRT Python API (
    ai_edge_litert.aot
    ),在 Linux
    x86_64
    本地或 Google Colab VM(
    --colab
    ,通过
    google-colab-cli
    )上,将普通
    .tflite
    编译为经 Google Tensor AOT 编译的
    .tflite
    ;当用户使用 macOS/Windows 或没有 Linux 机器时适用。
  2. 基准测试:使用 LiteRT 的
    benchmark_model
    ,在已连接的 Pixel 上对编译后的模型进行仅 NPU 基准测试;可选择与 CPU baseline 对比,并测量 TPU tile time(纯 TPU 计算时间)和每次推理的 TPU energy。
  3. 打包:为 Android (Gradle) 应用将编译后的构建产物打包为 Google Play AI pack,其中包含 Google Tensor NPU runtime 和 Kotlin loader;如果应用已经包含 Google Tensor 构建产物,则只打包用于替换的文件。在用户同意后,将该方案应用到项目,并在 Pixel 上检查应用。
所有操作都在用户终端中同步运行;脚本绝不会启动后台进程,并且始终清理推送到手机的内容或在 Colab 上分配的资源。以下所有路径均相对于此 skill 的目录 (
SKILL_DIR
)。
PathPurpose
assets/versions.json脚本使用的 LiteRT 版本(单一来源)
scripts/check_compiled.py检查该
.tflite
是否已经过 Tensor 编译
scripts/compile_model.pyAOT 编译(Linux
x86_64
本地或
--colab
)
scripts/benchmark_on_device.pyadb push → 基准测试 → 解析 → 清理
scripts/package_ai_pack.py将编译后的构建产物打包为应用的 AI pack + Gradle modules,或更新现有包
scripts/install_and_verify.py构建应用,以类似 Google Play 的方式安装,并读取 NPU/CPU 结果
assets/templates/Gradle、Kotlin、
INTEGRATION.md
和
UPDATE.md
模板(
package_ai_pack.py
)
references/compile-flags.mdAPI、编译 flags、脚本选项、
--colab
references/benchmark-flags.md基准测试的执行内容、二进制文件、报告写入方式和示例输出
references/app-integration.md应用打包:内容、检查、交付模式、应用方案以及在 Pixel 上检查
references/troubleshooting.md症状 → 修复对照表
references/contributing.md维护者:采用新的 LiteRT release
每个脚本都支持
--help
和
--dry-run
,且仅使用 Python 3 标准库,因此可使用任意
python3
运行;有两个例外需要导入 LiteRT:本地编译(运行
compile_model.py
且不带
--colab
、
--dry-run
或
--print-install
)必须使用 virtualenv 的解释器(
<venv>/bin/python SKILL_DIR/scripts/compile_model.py ...
,或先激活 venv);以及实际运行
package_ai_pack.py
(不是
--dry-run
或
--check-project
)需要安装了
ai-edge-litert
的 Python(编译 venv 即可;不需要 Beta SDK)。

Prerequisites (tell the user up front)

前置条件(提前告知用户)

NeedCompile (local)Compile (
--colab
)
BenchmarkPackage (app)
Google Tensor SDK Beta access - sign up at https://services.google.com/fb/forms/tensor_ml_sdk_experimental_access (overview: https://developers.google.com/edge/litert/next/tensor-sdk). Google emails a download link for
litert_plugin_compiler.tar.gz
; the compiler cannot be installed without it.
requiredrequired (
--sdk-tarball
or
--sdk-url
)
--
Host OSLinux x86_64, >= 16 GB RAMany OS (
colab
CLI on PATH; runs on a remote Colab Linux
x86_64
VM)
any OS with
adb
any OS where
ai-edge-litert
installs (the compile venv's Linux
x86_64
host works)
Python environmentLocal venv with
ai-edge-litert
+
ai-edge-litert-sdk-google-tensor
at the version in
versions.json
(
compile_model.py --print-install
)
Any local
python3
+
google-colab-cli
(
compile_model.py --print-install --colab
); SDK is installed on the Colab VM automatically
any
python3
(stdlib only)
a Python with
ai-edge-litert
(the compile venv works; the SDK package is not needed)
adb
on PATH, Pixel 10/11 with USB debugging enabled
--requiredstep 7 (install and check the app)
A JDK the app's Gradle supports (
JAVA_HOME
), the Android SDK, the project's
./gradlew
---steps 6-7 (build, install); bundletool is downloaded once
libLiteRtDispatch_GoogleTensor.so
- downloaded automatically from the public LiteRT binaries bucket (same directory as
benchmark_model
), or supplied with
--dispatch-lib
--required- (the app gets it from Google Maven)
Internetonce for
pip install
required (Colab VM +
pip install
)
once to fetch
benchmark_model
+ dispatch lib (cached in
~/.cache/litert-google-tensor/
) unless supplied locally
- (the app's Gradle build downloads the LiteRT Maven artifacts)
Supported targets:
Tensor_G5
(Pixel 10 family) and
Tensor_G6
(Pixel 11 family), as listed by the public Tensor SDK docs; Pixel 9 and older are not.
NeedCompile (local)Compile (
--colab
)
BenchmarkPackage (app)
Google Tensor SDK Beta 访问权限 - 在 https://services.google.com/fb/forms/tensor_ml_sdk_experimental_access 注册(概述:https://developers.google.com/edge/litert/next/tensor-sdk)。Google 会通过电子邮件发送
litert_plugin_compiler.tar.gz
的下载链接;没有该文件就无法安装编译器。
必需必需(
--sdk-tarball
或
--sdk-url
)
--
主机 OSLinux x86_64,RAM ≥ 16 GB任意 OS(PATH 中有
colab
CLI;运行在远程 Colab Linux
x86_64
VM 上)
任意安装了
adb
的 OS
任意可安装
ai-edge-litert
的 OS(编译 venv 的 Linux
x86_64
主机可用)
Python 环境本地 venv,其中包含
ai-edge-litert
+
ai-edge-litert-sdk-google-tensor
,版本与
versions.json
一致(
compile_model.py --print-install
)
任意本地
python3
+
google-colab-cli
(
compile_model.py --print-install --colab
);SDK 会自动安装在 Colab VM 上
任意
python3
(仅标准库)
安装了
ai-edge-litert
的 Python(编译 venv 即可;不需要 SDK package)
PATH 中有
adb
,且 Pixel 10/11 已启用 USB debugging
--必需第 7 步(安装并检查应用)
应用 Gradle 支持的 JDK(
JAVA_HOME
)、Android SDK、项目的
./gradlew
---第 6–7 步(构建、安装);bundletool 只需下载一次
libLiteRtDispatch_GoogleTensor.so
- 从公共 LiteRT binaries bucket 自动下载(与
benchmark_model
位于同一目录),也可通过
--dispatch-lib
提供
--必需-(应用从 Google Maven 获取)
互联网安装
pip
时需要一次
必需(Colab VM +
pip install
)
获取
benchmark_model
+ dispatch lib 时需要一次(缓存在
~/.cache/litert-google-tensor/
中),除非已在本地提供
-(应用的 Gradle 构建会下载 LiteRT Maven artifacts)
支持的目标:公共 Tensor SDK 文档中列出的
Tensor_G5
(Pixel 10 系列)和
Tensor_G6
(Pixel 11 系列);不支持 Pixel 9 及更早机型。

Step 0 - Collect parameters

第 0 步 - 收集参数

Ask the user only for what is missing; never invent values:
ParameterHow to get it
Model pathAbsolute path to the
.tflite
. Confirm it exists.
Goalcheck only, compile only, benchmark only, compile + benchmark, or package for an app (step 5). Infer it from the request ("how fast is it on the TPU vs CPU" = compile if needed + benchmark with a CPU baseline; "put it in my app" / "deploy it" = package); do not add an unrequested device run.
Target SoCFor a compile, the user's stated device wins (Pixel 10 family =
Tensor_G5
, Pixel 11 family =
Tensor_G6
), even if a different Pixel is attached. For a benchmark, the attached phone decides:
adb shell getprop ro.soc.model
->
Tensor G5
=>
Tensor_G5
,
Tensor G6
=>
Tensor_G6
(the benchmark script reads and enforces this itself).
Output pathDefault
<output-dir>/<stem>_Google_<target>.tflite
.
compile_model.py
refuses to overwrite an existing file (exit 2): run
check_compiled.py
on the existing file and reuse it, or pass
--output-name
, or
--overwrite
after asking. When a second build is needed next to an existing one, name it
<stem>_Google_<target>.<YYYYMMDD-HHMM>.tflite
(e.g.
--output-name model_Google_Tensor_G5.20260107-0930.tflite
) or, for a flag variant,
<stem>_Google_<target>.<flag-summary>.tflite
(
...G5.half-maxsharding.tflite
); keep the
_Google_<target>
part so the file stays recognisable.
Device serialOnly if
adb devices
lists more than one device.
Dispatch library / benchmark binaryOptional. Both are downloaded automatically from the same public bucket directory and cached. Only ask if the user wants a specific build, or if the host is offline: then
--benchmark-bin <benchmark_model>
plus
--dispatch-lib <libLiteRtDispatch_GoogleTensor.so | dir | .zip | .aar>
(also env
LITERT_GOOGLE_TENSOR_DISPATCH_LIB
) use local copies and no network is touched.
CPU baselineOptional; needs the path of the original uncompiled model (ask for it if only a compiled file was given; the benchmark script rejects a compiled file here with exit 2).
Run counts / modeDefaults 5 warmup / 50 measured,
burst
performance mode.
benchmark_model
treats counts as minimums and also runs at least 0.5 s warmup / 1 s measured, so fast models execute more runs than requested. If the user states exact counts, say so and ask whether to disable the floors (
--extra-arg=--min_secs=0 --extra-arg=--warmup_min_secs=0
); "steady-state" / "sustained" requests map to
--performance-mode sustained_performance
.
TPU tile timeOn by default, nothing to ask: a second, hooked NPU pass.
--no-tile-time
turns it off; the hooked pass itself (it repeats the NPU run, so slow models take about twice as long) is skipped only with
--no-tile-time --no-power
.
TPU energyOn by default, nothing to ask: the power metric of the same hooked pass (the TPU rail's energy per inference).
--no-power
skips it. The script leaves it off, and says why, for a dispatch library whose power hook predates the fix.
Compile modeLocal Linux
x86_64
by default. Use
--colab
when the host is not Linux
x86_64
(macOS/Windows/ARM) or when the user asks to compile on Google Colab; ask for the Beta tarball path (
--sdk-tarball
/
GOOGLE_TENSOR_SDK_BETA
) or URL (
--sdk-url
/
GOOGLE_TENSOR_ML_SDK_URL
) before running.
Compiler flagsDefault to the compiler defaults; only set flags the user asks for or that troubleshooting requires (see references/compile-flags.md).
App deployment levelPackaging jobs only; nothing to ask up front. After step 5, tell the user the plan is ready (
INTEGRATION.md
, or
UPDATE.md
for an update) and ask whether you should apply it to their project (step 6). After that, offer to install the app on their Pixel and check it uses the NPU (step 7).
New integration or updatePackaging jobs only. Ask whether the app already ships Google Tensor builds (an AI pack with folders like
model#group_Google_Tensor_G5
under its
src/main/assets
). If it does, it is an update (step 5b): ask for the project root (the folder with
settings.gradle.kts
or
settings.gradle
). The pack, model file, device groups, delivery mode and app module then come from the project, so do not ask for them.
Builds to packageOne compiled build per target SoC (
Tensor_G5
,
Tensor_G6
) plus the original uncompiled model they were compiled from (the app's CPU path). Compile missing targets first (step 2); a build is per SoC, so a Pixel 11 needs a
Tensor_G6
build. For an update, build every SoC the app already ships.
Delivery modeNew integrations only (an update keeps the app's). When Google Play puts the model on the phone:
install-time
(default; installed with the app, ready at first launch),
fast-follow
(downloaded right after install) or
on-demand
(downloaded when the app asks). Explain each in one sentence (references/app-integration.md) and use the default when the user has no preference.
Pack / model names, app moduleNew integrations only. Defaults: pack name = the original's file name made valid, model name = pack name, app module
app
, Kotlin package
com.example.litert
. Ask only when the user's project differs (another app module name, their package).
Quick test (optional)New integrations only. Offer it when the user wants to try the model on their own Pixel before setting up bundletool:
--quick-test
adds a copy of each build and the steps for
./gradlew installDebug -PtensorQuickTest=<SoC>
, which puts the build and the NPU runtime in the debug APK. Local testing only; release builds are unaffected.
仅询问用户尚未提供的信息;绝不臆造值:
ParameterHow to get it
模型路径
.tflite
的绝对路径。确认文件存在。
目标仅检查、仅编译、仅基准测试、编译 + 基准测试,或为应用打包(第 5 步)。根据请求推断(“TPU 和 CPU 上各有多快”= 如有需要则编译,并使用 CPU baseline 进行基准测试;“放进我的应用”/“部署它”= 打包);不要添加用户未要求的设备运行。
目标 SoC对于编译,以用户指定的设备为准(Pixel 10 系列 =
Tensor_G5
,Pixel 11 系列 =
Tensor_G6
),即使连接的是其他 Pixel。对于基准测试,以已连接的手机为准:
adb shell getprop ro.soc.model
→
Tensor G5
⇒
Tensor_G5
,
Tensor G6
⇒
Tensor_G6
(基准测试脚本会自行读取并强制验证)。
输出路径默认值为
<output-dir>/<stem>_Google_<target>.tflite
。
compile_model.py
拒绝覆盖现有文件(退出码 2):应对现有文件运行
check_compiled.py
并复用,或者传入
--output-name
,或者在询问后传入
--overwrite
。当需要在现有文件旁创建第二个构建产物时,命名为
<stem>_Google_<target>.<YYYYMMDD-HHMM>.tflite
(例如
--output-name model_Google_Tensor_G5.20260107-0930.tflite
);如果是编译 flag 变体,则命名为
<stem>_Google_<target>.<flag-summary>.tflite
(
...G5.half-maxsharding.tflite
);保留
_Google_<target>
部分,以便文件保持可识别。
设备序列号仅当
adb devices
列出了多台设备时需要。
Dispatch library / benchmark binary可选。两者都会从同一个公共 bucket 目录自动下载并缓存。仅当用户需要特定构建版本或主机离线时才询问;此时使用
--benchmark-bin <benchmark_model>
和 `--dispatch-lib <libLiteRtDispatch_GoogleTensor.so
CPU baseline可选;需要原始未编译模型的路径(如果只提供了已编译文件,则询问该路径;基准测试脚本会拒绝在此使用已编译文件并以退出码 2 结束)。
运行次数 / 模式默认 5 次预热 / 50 次测量,模式为
burst
。
benchmark_model
将次数视为最低值,并且至少运行 0.5 秒预热 / 1 秒测量,因此快速模型执行的次数会多于请求值。如果用户指定了精确次数,请说明这一点,并询问是否禁用下限(
--extra-arg=--min_secs=0 --extra-arg=--warmup_min_secs=0
);“steady-state”/“sustained”请求映射到
--performance-mode sustained_performance
。
TPU tile time默认开启,无需询问:执行第二次带 hook 的 NPU pass。
--no-tile-time
可将其关闭;只有在同时使用
--no-tile-time --no-power
时,才会跳过该 hooked pass 本身(它会重复 NPU 运行,因此较慢的模型耗时约为两倍)。
TPU energy默认开启,无需询问:来自同一次 hooked pass 的 power metric(TPU 电源轨每次推理消耗的能量)。
--no-power
会跳过它。对于 power hook 早于修复版本的 dispatch library,脚本会将其关闭并说明原因。
编译模式默认在 Linux
x86_64
本地执行。当主机不是 Linux
x86_64
(macOS/Windows/ARM),或用户要求在 Google Colab 上编译时,使用
--colab
;运行前询问 Beta tarball 路径(
--sdk-tarball
/
GOOGLE_TENSOR_SDK_BETA
)或 URL(
--sdk-url
/
GOOGLE_TENSOR_ML_SDK_URL
)。
编译器 flags默认使用编译器默认值;仅设置用户明确要求或 troubleshooting 需要的 flags(见 references/compile-flags.md)。
应用部署级别仅适用于打包任务;无需提前询问。第 5 步后,告知用户方案已就绪(
INTEGRATION.md
,更新场景则为
UPDATE.md
),并询问是否将其应用到项目(第 6 步)。随后,询问是否愿意将应用安装到其 Pixel 并检查其是否使用 NPU(第 7 步)。
新集成或更新仅适用于打包任务。询问应用是否已经包含 Google Tensor 构建产物(其
src/main/assets
下有类似
model#group_Google_Tensor_G5
文件夹的 AI pack)。如果有,则属于更新(第 5b 步):询问项目根目录(包含
settings.gradle.kts
或
settings.gradle
的文件夹)。pack、模型文件、设备组、交付模式和 app module 均从项目中获取,因此不要询问用户。
需要打包的构建产物每个目标 SoC(
Tensor_G5
、
Tensor_G6
)需要一个已编译构建产物,以及编译所依据的原始未编译模型(应用的 CPU path)。先编译缺失的目标(第 2 步);构建产物与 SoC 一一对应,因此 Pixel 11 需要
Tensor_G6
构建产物。对于更新,构建应用已经包含的每个 SoC。
交付模式仅适用于新集成(更新时保持应用原有模式)。当 Google Play 负责将模型放到手机上时:
install-time
(默认;随应用安装,首次启动即可用)、
fast-follow
(安装后立即下载)或
on-demand
(应用请求时下载)。每种模式用一句话说明(references/app-integration.md);用户没有偏好时使用默认值。
Pack / 模型名称、app module仅适用于新集成。默认值:pack name = 将原始文件名处理为合法名称,model name = pack name,app module 为
app
,Kotlin package 为
com.example.litert
。仅当用户项目存在差异时询问(其他 app module 名称、自定义 package)。
快速测试(可选)仅适用于新集成。当用户希望在配置 bundletool 前先在自己的 Pixel 上试用模型时提供:
--quick-test
会添加每个构建产物的副本以及运行
./gradlew installDebug -PtensorQuickTest=<SoC>
的步骤,从而将构建产物和 NPU runtime 放入 debug APK。仅用于本地测试;release 构建不受影响。

Commands or a plan only (the user says not to run anything)

仅提供命令或方案(用户表示不运行任何内容)

Run nothing - not even the read-only scripts - and give the same steps as a real run, in the same order, as commands the user can paste:
  1. Start with
    check_compiled.py
    on every input model (step 1), before any install or compile command; checking only the compiled output afterwards is not enough.
  2. For installs, give
    python3 SKILL_DIR/scripts/compile_model.py --print-install [--colab]
    as the command that produces them (step 2a), then the lines it prints with the user's paths filled in (venv,
    export GOOGLE_TENSOR_SDK_BETA=<their tarball>
    before
    pip install
    ) and each version written as
    <version printed by --print-install>
    . Never copy a version number from
    versions.json
    or from memory.
  3. Give no results, not even estimates: latency, speed-up, memory and ops offloaded come only from the scripts' output. Say where they will appear (for a benchmark, the table
    benchmark_on_device.py
    prints and saves as
    report.md
    ).
  4. When the plan changes the user's app project (step 6), say that nothing in the project changes until they approve, and end by asking whether you should apply it.
不运行任何命令,甚至不运行只读脚本;仍按照真实运行的相同顺序,提供用户可以直接粘贴的命令:
  1. 首先对每个输入模型运行
    check_compiled.py
    (第 1 步),且必须早于任何安装或编译命令;仅检查编译后的输出并不足够。
  2. 对于安装操作,先给出
    python3 SKILL_DIR/scripts/compile_model.py --print-install [--colab]
    作为生成安装命令的命令(第 2a 步),然后给出它打印的行,并填入用户路径(venv、在
    pip install
    前执行
    export GOOGLE_TENSOR_SDK_BETA=<their tarball>
    ),并将每个版本写为
    <version printed by --print-install>
    。绝不从
    versions.json
    或记忆中复制版本号。
  3. 不提供任何结果,即使是估算值也不提供:延迟、加速比、内存和卸载到 CPU 的 ops 只能来自脚本输出。说明这些结果将在哪里出现(基准测试结果位于
    benchmark_on_device.py
    打印并保存为
    report.md
    的表格中)。
  4. 当方案会修改用户的应用项目时(第 6 步),说明在用户批准前不会更改项目中的任何内容,并在最后询问是否应用方案。

Step 1 - Check whether the model is already compiled

第 1 步 - 检查模型是否已经编译

bash
python3 SKILL_DIR/scripts/check_compiled.py /abs/path/model.tflite [--expect-target Tensor_G5]
Run it on every model file involved (the model to compile or benchmark, the CPU-baseline file, and an existing output file you are about to reuse). For a benchmark job read the phone first (
adb shell getprop ro.soc.model
prints
Tensor G5
or
Tensor G6
) and pass the underscore form as
--expect-target
(
Tensor G5
->
--expect-target Tensor_G5
), so one check answers both "compiled?" and "for this phone?" (a plain model exits 1 either way); the benchmark script enforces the match again anyway (exit 4).
ExitMeaningNext action
0compiled for Google Tensor (SoC printed)skip compile; report the SoC (and the
Compile info
line if a
.compile.json
sidecar exists). Continue to step 3 only if the goal includes a benchmark. No sidecar (compiled elsewhere, flags unknown): for a plain "how fast is it" job, benchmark the file as-is and say "compile flags unknown"; when the user needs to know how it was built (or wants a reproducible comparison), compile the original with the defaults under a new
--output-name
and compare SHA-256 (
sha256sum
of both files, or
output_sha256
in the new sidecar): compilation is deterministic, so an identical hash proves the existing file is a default build (references/compile-flags.md). Never
--overwrite
to find out.
1plain (uncompiled) modelgo to step 2 if compiling; this is the correct file for
--cpu-baseline
2compiled for a different SoC than
--expect-target
compiled models are SoC-specific (a
Tensor_G5
file never runs on a
Tensor_G6
TPU, and a
Tensor_G6
file never runs on a
Tensor_G5
TPU; there is no runtime fallback, and
--allow-mismatch
only produces dispatch errors). Tell the user which SoC the phone has, then offer, in this order: an existing
*_Google_<phone target>.tflite
build of the same model in the project (verify it with this check first; do not benchmark it unless the user says so), attaching the matching phone, or recompiling the original uncompiled export for the phone's target.
3not a
.tflite
/ missing
fix the path
4compiled by another toolchain: an Edge TPU custom op (
edgetpu-custom-op...
) instead of LiteRT's
DISPATCH_OP
not usable here: LiteRT cannot recompile it or run it on the Google Tensor TPU, and the CPU cannot run it (so it is no
--cpu-baseline
or
--fallback
either; the scripts refuse it). Ask for the original model it was compiled from and compile that (step 2)
The
LiteRtStamp
is authoritative (the
_Google_Tensor_G5
suffix is only a naming convention) and records the SoC but not the compiler flags: those come from the
<model>.compile.json
sidecar that
compile_model.py
writes next to its output; without it, say "flags unknown" rather than guessing.
bash
python3 SKILL_DIR/scripts/check_compiled.py /abs/path/model.tflite [--expect-target Tensor_G5]
对涉及的每个模型文件运行该命令(待编译或待基准测试的模型、CPU baseline 文件,以及准备复用的现有输出文件)。对于基准测试任务,先读取手机信息(
adb shell getprop ro.soc.model
会打印
Tensor G5
或
Tensor G6
),并使用下划线形式传入
--expect-target
(
Tensor G5
→
--expect-target Tensor_G5
),这样一次检查即可同时回答“是否已编译?”和“是否适用于此手机?”(普通模型无论哪种情况都会以退出码 1 结束);基准测试脚本还会再次强制验证是否匹配(退出码 4)。
ExitMeaningNext action
0已为 Google Tensor 编译(打印 SoC)跳过编译;报告 SoC(如果存在
.compile.json
sidecar,也报告其
Compile info
行)。仅当目标包含基准测试时继续第 3 步。没有 sidecar(在别处编译、flags 未知):对于普通的“它有多快”任务,直接按原样基准测试该文件,并说明“compile flags unknown”;当用户需要了解构建方式(或希望进行可复现比较)时,使用默认 flags 从原始模型编译到一个新的
--output-name
下,并比较 SHA-256(两个文件的
sha256sum
,或新 sidecar 中的
output_sha256
):编译是确定性的,因此哈希值相同即可证明现有文件是默认构建(references/compile-flags.md)。绝不通过
--overwrite
来查明这一点。
1普通(未编译)模型如果需要编译则进入第 2 步;这是
--cpu-baseline
应使用的正确文件
2编译目标 SoC 与
--expect-target
不同
编译后的模型与 SoC 专用(
Tensor_G5
文件无法在
Tensor_G6
TPU 上运行,
Tensor_G6
文件也无法在
Tensor_G5
TPU 上运行;不存在 runtime fallback,而
--allow-mismatch
只会产生 dispatch errors)。告知用户手机具有哪个 SoC,然后按以下顺序提供选项:项目中同一模型的现有
*_Google_<phone target>.tflite
构建产物(先通过本检查验证;除非用户要求,否则不要对其运行基准测试)、连接匹配的手机,或为手机目标重新编译原始未编译导出文件。
3不是
.tflite
/ 文件不存在
修复路径
4由其他 toolchain 编译:使用 Edge TPU custom op(
edgetpu-custom-op...
),而不是 LiteRT 的
DISPATCH_OP
此处不可用:LiteRT 无法重新编译它,也无法在 Google Tensor TPU 上运行它,CPU 同样无法运行它(因此也不能用作
--cpu-baseline
或
--fallback
);脚本会拒绝处理。请用户提供编译它时使用的原始模型,然后编译该模型(第 2 步)
LiteRtStamp
是权威依据(
_Google_Tensor_G5
后缀仅是命名约定),它会记录 SoC,但不会记录编译 flags;编译 flags 来自
compile_model.py
写在输出旁边的
<model>.compile.json
sidecar;如果没有该文件,应说明“flags unknown”,不要猜测。

Step 2 - Compile (Linux x86_64 or Google Colab via
--colab
)

第 2 步 - 编译(Linux x86_64 或通过
--colab
使用 Google Colab)

2a. One-time environment

2a. 一次性环境配置

The SDK pip package contains no compiler by itself: at install time it unpacks the Beta tarball that
GOOGLE_TENSOR_SDK_BETA
points to (same flow as the public codelab). Ask the user for the tarball path first, then print the install commands for the LiteRT version the skill pins in
versions.json
:
bash
python3 SKILL_DIR/scripts/compile_model.py --print-install [--colab]
Local Linux
x86_64
(
--print-install
): runs with any
python3
and prints comment lines (the pinned version, when and on what it was last verified, the supported Python range, and whether this host and Python qualify) followed by:
text
python3 -m venv ~/.venvs/litert-tensor && source ~/.venvs/litert-tensor/bin/activate
pip install --upgrade pip
export GOOGLE_TENSOR_SDK_BETA=/abs/path/litert_plugin_compiler.tar.gz   # from the Beta e-mail
pip install --no-cache-dir ai-edge-litert==<pinned> ai-edge-litert-sdk-google-tensor==<pinned>
Remote Google Colab (
--print-install --colab
) - for macOS, Windows, non-x86 Linux, or when compiling on Colab without a local Linux venv: installs
google-colab-cli
into
~/.venvs/colab-cli
(how
--colab
works
). Never ask the user to run a terminal command for OAuth: when
--colab
needs one-time OAuth, show its printed URL in chat, ask the user for the code, and re-run with
--colab-auth-code <CODE>
.
  • Tell the user before installing anything. Run the printed commands with the real tarball path and never type a LiteRT version yourself: the runtime and SDK packages must be one version (
    WARNING
    when they differ;
    NOTE
    when they match each other but not the pin). Use
    -nightly
    packages only when the user explicitly asks for nightlies.
  • If
    --print-install
    warns about the Python version, create the venv with a
    python3.N
    in the supported range. Install errors: references/troubleshooting.md.
If the user has no Beta access yet: say so plainly - the compiler cannot be installed until Google e-mails the tarball link (sign-up form above) - and do not install anything or allocate a Colab VM meanwhile (
--colab
checks for the tarball before
colab new
). Still useful now: run
check_compiled.py
on their files, confirm the host and phone SoC, and run
--dry-run
/
--print-install
.
SDK pip package 本身不包含编译器:安装时,它会解包
GOOGLE_TENSOR_SDK_BETA
指向的 Beta tarball(与公共 codelab 相同)。先询问用户 tarball 路径,然后打印此 skill 在
versions.json
中固定的 LiteRT 版本对应的安装命令:
bash
python3 SKILL_DIR/scripts/compile_model.py --print-install [--colab]
Linux
x86_64
本地
(
--print-install
):可使用任意
python3
运行,并打印注释行(固定版本、最近一次验证的时间和环境、受支持的 Python 版本范围,以及此主机和 Python 是否符合要求),随后打印:
text
python3 -m venv ~/.venvs/litert-tensor && source ~/.venvs/litert-tensor/bin/activate
pip install --upgrade pip
export GOOGLE_TENSOR_SDK_BETA=/abs/path/litert_plugin_compiler.tar.gz   # from the Beta e-mail
pip install --no-cache-dir ai-edge-litert==<pinned> ai-edge-litert-sdk-google-tensor==<pinned>
远程 Google Colab(
--print-install --colab
)- 适用于 macOS、Windows、非 x86 Linux,或在 Colab 上编译但本地没有 Linux venv 的情况:它会将
google-colab-cli
安装到
~/.venvs/colab-cli
(有关
--colab
工作方式的说明
)。 绝不让用户运行终端命令完成 OAuth:当
--colab
需要一次性 OAuth 时,在聊天中显示其打印的 URL,询问用户获取 code,然后使用
--colab-auth-code <CODE>
重新运行。
  • 安装任何内容前先告知用户。使用真实 tarball 路径运行打印出的命令,绝不手动输入 LiteRT 版本:runtime 和 SDK packages 必须使用同一版本(版本不同时会显示
    WARNING
    ;两者彼此一致但与固定版本不同时会显示
    NOTE
    )。仅当用户明确要求 nightly 时才使用
    -nightly
    packages。
  • 如果
    --print-install
    警告 Python 版本问题,请使用受支持范围内的
    python3.N
    创建 venv。安装错误见:references/troubleshooting.md。
**如果用户尚未获得 Beta 访问权限:**明确说明这一点 - 在 Google 通过电子邮件发送 tarball 链接之前,编译器无法安装(使用上方注册表单) - 并且不要安装任何内容,也不要同时分配 Colab VM(
--colab
会在执行
colab new
前检查 tarball)。目前仍然可以执行:对其文件运行
check_compiled.py
、确认主机和手机 SoC,以及运行
--dry-run
/
--print-install
。

2b. Compile

2b. 编译

bash
undefined
bash
undefined

Local Linux x86_64 (using the virtualenv's interpreter):

Local Linux x86_64 (using the virtualenv's interpreter):

python3 SKILL_DIR/scripts/compile_model.py /abs/path/model.tflite
--target Tensor_G5 --output-dir /abs/path/compiled
python3 SKILL_DIR/scripts/compile_model.py /abs/path/model.tflite
--target Tensor_G5 --output-dir /abs/path/compiled

Remote Google Colab VM (any OS, any local python3 + google-colab-cli):

Remote Google Colab VM (any OS, any local python3 + google-colab-cli):

python3 SKILL_DIR/scripts/compile_model.py /abs/path/model.tflite
--target Tensor_G5 --colab --sdk-tarball /abs/path/litert_plugin_compiler.tar.gz
--output-dir /abs/path/compiled
python3 SKILL_DIR/scripts/compile_model.py /abs/path/model.tflite
--target Tensor_G5 --colab --sdk-tarball /abs/path/litert_plugin_compiler.tar.gz
--output-dir /abs/path/compiled

-> /abs/path/compiled/model_Google_Tensor_G5.tflite

-> /abs/path/compiled/model_Google_Tensor_G5.tflite


The script prints the compilation report, re-verifies the Tensor stamp, writes
`<output>.compile.json` (flags, ops offloaded / total, package versions,
SHA-256s) and never overwrites an existing output (exit 2) unless
`--overwrite` is given. Ops left on the CPU in the report mean the strict NPU
benchmark will fail (step 3, `--allow-cpu-fallback`). Every compiler, script
and Colab option: [references/compile-flags.md](references/compile-flags.md).

*   If the compiler crashes with an internal error - or the user reports that it
    did with the defaults (no need to reproduce the crash) - use
    `--truncation-type half --sharding-intensity maximum` (public FAQ guidance)
    and tell the user which flags produced the artifact. `half` changes the
    numerics (16-bit floats on the TPU), so tell the user to validate the
    model's outputs against the original before relying on it.
*   Report a compile-only job from the script's output and sidecar only:

    ```markdown
    **Compiled:** <output path> (<bytes from "Compiled model written to" line>)
    **Target / flags:** <cli_flags from the sidecar, or "--target X (compiler defaults)">
    **Ops offloaded:** <N / M from the "Ops offloaded to the TPU" line>
    **LiteRT:** <"packages" from the sidecar, plus the script's WARNING/NOTE line if it printed one>
    **Verified:** check_compiled.py -> <its Status line>
    ```

Exit codes: 0 ok, 2 bad input, 3 already compiled, 4 unsupported host,
5 LiteRT packages or Beta compiler library missing (message includes the
install fix), 6 compile failed (compiler log errors are printed),
7 output verification failed.

脚本会打印编译报告、再次验证 Tensor stamp、写入 `<output>.compile.json`(flags、卸载到 TPU 的 ops / 总 ops、package versions、SHA-256),并且绝不会覆盖现有输出(退出码 2),除非传入 `--overwrite`。报告中仍有 ops 留在 CPU 上,意味着严格 NPU 基准测试将失败(第 3 步,`--allow-cpu-fallback`)。所有编译器、脚本和 Colab 选项见:[references/compile-flags.md](references/compile-flags.md)。

*   如果编译器因内部错误而崩溃 - 或用户报告使用默认设置时崩溃(无需复现崩溃) - 使用 `--truncation-type half --sharding-intensity maximum`(公共 FAQ 建议),并告知用户生成该产物所使用的 flags。`half` 会改变数值计算方式(TPU 上使用 16 位浮点数),因此应告知用户在依赖该模型之前,将模型输出与原始模型进行验证。
*   仅编译任务只能根据脚本输出和 sidecar 报告:

    ```markdown
    **Compiled:** <output path> (<bytes from "Compiled model written to" line>)
    **Target / flags:** <cli_flags from the sidecar, or "--target X (compiler defaults)">
    **Ops offloaded:** <N / M from the "Ops offloaded to the TPU" line>
    **LiteRT:** <"packages" from the sidecar, plus the script's WARNING/NOTE line if it printed one>
    **Verified:** check_compiled.py -> <its Status line>
    ```

退出码:0 表示成功,2 表示输入无效,3 表示已经编译,4 表示不支持的主机,5 表示缺少 LiteRT packages 或 Beta compiler library(消息中包含安装修复方法),6 表示编译失败(会打印 compiler log 错误),7 表示输出验证失败。

Step 3 - Benchmark on the device

第 3 步 - 在设备上进行基准测试

bash
python3 SKILL_DIR/scripts/benchmark_on_device.py \
  --model /abs/path/compiled/model_Google_Tensor_G5.tflite \
  [--cpu-baseline /abs/path/model.tflite] [--serial SERIAL] \
  [--dispatch-lib /abs/path/libLiteRtDispatch_GoogleTensor.so] \
  [--output-dir /abs/path/results]
The script refuses a model compiled for another SoC than the phone's (exit 4;
--dry-run
never contacts the phone, so it proves nothing about the match), fetches
benchmark_model
and
libLiteRtDispatch_GoogleTensor.so
once from the bucket release in
versions.json
(cached;
downloaded from <url>
means a first download - tell the user what was fetched), runs the strict NPU benchmark, a second hooked NPU pass for TPU tile time and TPU energy, and the optional CPU baseline on the uncompiled model in a fresh
/data/local/tmp/litert_benchmark/run-*
directory, which it deletes even on failure. It writes
report.md
,
results.json
and the logs to
--output-dir
(nothing after exit 2/3/4 or
--dry-run
). Each step and flag: references/benchmark-flags.md.
The script is NPU-only by design (
--extra-arg
cannot switch the accelerator, and an uncompiled model is rejected with exit 2); GPU or non-Tensor runs (
benchmark_model --use_gpu=true
etc. directly) are outside this skill. Other options (
--num-runs
,
--performance-mode
,
--signature
,
--allow-cpu-fallback
,
--no-tile-time
,
--no-power
, ...) are in
--help
.
  • Offline host: pass both
    --benchmark-bin
    and
    --dispatch-lib
    ; a
    --dry-run
    then shows
    user-provided: <path>
    for both binaries without touching the phone or the network - the proof the user wants first.
  • Binary versions: keep
    benchmark_model
    and the dispatch library from the same bucket release (the default does;
    --benchmark-url .../binaries/<release>/android_arm64/benchmark_model
    pins another one and the dispatch URL follows). If a provenance md5 differs from
    verified.binaries
    in
    versions.json
    , rerun with
    --refresh-binary
    (details).
bash
python3 SKILL_DIR/scripts/benchmark_on_device.py \
  --model /abs/path/compiled/model_Google_Tensor_G5.tflite \
  [--cpu-baseline /abs/path/model.tflite] [--serial SERIAL] \
  [--dispatch-lib /abs/path/libLiteRtDispatch_GoogleTensor.so] \
  [--output-dir /abs/path/results]
如果模型编译目标 SoC 与手机不同,脚本会拒绝运行(退出码 4;
--dry-run
不会连接手机,因此无法证明是否匹配)。脚本会从
versions.json
指定的 bucket release 中一次性获取
benchmark_model
和
libLiteRtDispatch_GoogleTensor.so
(带缓存;出现
downloaded from <url>
表示首次下载 - 应告知用户下载了什么),运行严格 NPU 基准测试,再执行第二次 hooked NPU pass 以测量 TPU tile time 和 TPU energy,并在新的
/data/local/tmp/litert_benchmark/run-*
目录中基于未编译模型运行可选的 CPU baseline,即使失败也会删除该目录。它会将
report.md
、
results.json
和日志写入
--output-dir
(退出码 2/3/4 后或使用
--dry-run
时不会产生任何内容)。每个步骤和 flag 见:references/benchmark-flags.md。
脚本在设计上仅使用 NPU(
--extra-arg
无法切换 accelerator,且未编译模型会以退出码 2 被拒绝);GPU 或非 Tensor 运行(直接使用
benchmark_model --use_gpu=true
等)不在此 skill 范围内。其他选项(
--num-runs
、
--performance-mode
、
--signature
、
--allow-cpu-fallback
、
--no-tile-time
、
--no-power
等)见
--help
。
  • **离线主机:**同时传入
    --benchmark-bin
    和
    --dispatch-lib
    ;此时
    --dry-run
    会为两个二进制文件显示
    user-provided: <path>
    ,且不会连接手机或访问网络 - 这正是用户首先需要验证的内容。
  • 二进制文件版本:让
    benchmark_model
    和 dispatch library 来自
    同一个
    bucket release(默认行为如此;
    --benchmark-url .../binaries/<release>/android_arm64/benchmark_model
    可固定其他版本,dispatch URL 会随之变化)。如果 provenance md5 与
    versions.json
    中的
    verified.binaries
    不一致,请使用
    --refresh-binary
    重新运行(详情)。

Step 4 - Report

第 4 步 - 报告

Nothing in the report is typed by hand.
benchmark_on_device.py
prints a Markdown header + table (also saved as
report.md
and as the
markdown
field of
results.json
) in which every value is read from the device or the run. Paste that block verbatim and add only the context the script cannot know:
markdown
**Summary:** <NPU "Inference avg" from the table> ms on the TPU vs <CPU "Inference avg"> ms on the CPU (<the N.NNx value from the script's "NPU speed-up vs CPU" line>); peak memory <NPU> vs <CPU> MB; <strict NPU | CPU fallback allowed>. TPU tile time <"TPU tile time / inference" row> ms per inference. TPU energy <"TPU energy / inference" row> mJ per inference.

<header and table exactly as printed by benchmark_on_device.py, including any "Unexpected log lines" section>

**Notes:** <what the Log check line reports: "none" when both logs show 0 unexpected lines and no profiler retry; otherwise each unexpected line with its meaning from references/benchmark-flags.md or troubleshooting.md, plus anything the user should validate>
When nothing ran (check exit 2, benchmark exit 2/3/4/5, no device), report the same way - from output, not from memory:
markdown
**Not run:** <the script's ERROR line verbatim>

- Device: <the "Device: ..." line printed by benchmark_on_device.py, or the `adb devices -l` line>
- Model: <the "Status: ..." line from check_compiled.py>
- Nothing was pushed to or removed from the phone.

**Options:** <the next actions from step 1 / troubleshooting, most specific first>
  • NEVER type, round, or "tidy up" numbers, and NEVER reuse values from an earlier run or from any example in this skill - copy the script output. Prose numbers (the Summary line) must be character-for-character the table's values. Never edit
    report.md
    (script output); if the user wants the answer saved, write
    summary.md
    next to it.
  • Before writing the Summary and Notes, read Writing the report (no CPU baseline, combined compile + benchmark jobs, run counts, CPU fallback, mixed LiteRT releases, speed-up below 1x, TPU tile time/energy).
报告中的任何内容都不得手动输入。
benchmark_on_device.py
会打印 Markdown 标题和表格(同时保存为
report.md
,并作为
results.json
的
markdown
字段),其中每个值都读取自设备或本次运行。原样粘贴该代码块,并且只添加脚本无法获知的上下文:
markdown
**Summary:** <NPU "Inference avg" from the table> ms on the TPU vs <CPU "Inference avg"> ms on the CPU (<the N.NNx value from the script's "NPU speed-up vs CPU" line>); peak memory <NPU> vs <CPU> MB; <strict NPU | CPU fallback allowed>. TPU tile time <"TPU tile time / inference" row> ms per inference. TPU energy <"TPU energy / inference" row> mJ per inference.

<header and table exactly as printed by benchmark_on_device.py, including any "Unexpected log lines" section>

**Notes:** <what the Log check line reports: "none" when both logs show 0 unexpected lines and no profiler retry; otherwise each unexpected line with its meaning from references/benchmark-flags.md or troubleshooting.md, plus anything the user should validate>
没有运行任何内容时(检查退出码 2、基准测试退出码 2/3/4/5、无设备),也按照相同方式报告 - 依据输出,而不是记忆:
markdown
**Not run:** <the script's ERROR line verbatim>

- Device: <the "Device: ..." line printed by benchmark_on_device.py, or the `adb devices -l` line>
- Model: <the "Status: ..." line from check_compiled.py>
- Nothing was pushed to or removed from the phone.

**Options:** <the next actions from step 1 / troubleshooting, most specific first>
  • 绝不对数字进行输入、取整或“整理”,也绝不复用之前运行或此 skill 示例中的任何值 - 应复制脚本输出。说明性文字中的数字(Summary 行)必须与表格中的值逐字符一致。绝不编辑
    report.md
    (脚本输出);如果用户希望保存答案,则在其旁边写入
    summary.md
    。
  • 编写 Summary 和 Notes 前,阅读 Writing the report(涵盖无 CPU baseline、组合编译 + 基准测试任务、运行次数、CPU fallback、混合 LiteRT releases、低于 1x 的加速比、TPU tile time/energy)。

Step 5 - Package for an app (Gradle)

第 5 步 - 为应用打包(Gradle)

Two cases (Step 0, "New integration or update"); what the package contains and how the script checks it: references/app-integration.md.
  • 5a - New integration: an AI pack written by LiteRT's own exporter from the files as they are (byte-identical, nothing recompiled), the Google Tensor NPU runtime as an install-time feature module (from Google Maven), the original model for the app's CPU path, a Kotlin loader, and an
    INTEGRATION.md
    with every remaining step for the user's project.
  • 5b - Update (the app already ships Google Tensor builds):
    --update-project
    only reads the project and prepares the files to replace, under the app's own paths and names, plus an
    UPDATE.md
    .
bash
undefined
分为两种情况(第 0 步中的“新集成或更新”);package 的内容以及脚本检查方式见:references/app-integration.md。
  • **5a - 新集成:**使用 LiteRT 自带的 exporter,直接根据现有文件生成 AI pack(字节完全一致,不重新编译),将 Google Tensor NPU runtime 作为 install-time feature module(来自 Google Maven),包含应用 CPU path 使用的原始模型、一个 Kotlin loader,以及一份
    INTEGRATION.md
    ,其中列出用户项目的所有剩余步骤。
  • 5b - 更新(应用已经包含 Google Tensor 构建产物):
    --update-project
    只读取项目,并按应用自身的路径和名称准备用于替换的文件,同时生成
    UPDATE.md
    。
bash
undefined

5a -> <fallback dir>/app_package/<pack name>/INTEGRATION.md and the modules

5a -> <fallback dir>/app_package/<pack name>/INTEGRATION.md and the modules

<venv>/bin/python SKILL_DIR/scripts/package_ai_pack.py
--compiled /abs/path/compiled/model_Google_Tensor_G5.tflite
[--compiled /abs/path/compiled/model_Google_Tensor_G6.tflite]
--fallback /abs/path/model.tflite
[--delivery install-time|fast-follow|on-demand] [--pack-name NAME]
[--model-name NAME] [--app-module app] [--kotlin-package com.example.litert]
[--quick-test] [--output-dir /abs/path/app_package/NAME]
<venv>/bin/python SKILL_DIR/scripts/package_ai_pack.py
--compiled /abs/path/compiled/model_Google_Tensor_G5.tflite
[--compiled /abs/path/compiled/model_Google_Tensor_G6.tflite]
--fallback /abs/path/model.tflite
[--delivery install-time|fast-follow|on-demand] [--pack-name NAME]
[--model-name NAME] [--app-module app] [--kotlin-package com.example.litert]
[--quick-test] [--output-dir /abs/path/app_package/NAME]

5b -> <fallback dir>/app_update/<pack>/UPDATE.md, update.json, files/<paths>

5b -> <fallback dir>/app_update/<pack>/UPDATE.md, update.json, files/<paths>

<venv>/bin/python SKILL_DIR/scripts/package_ai_pack.py
--update-project /abs/path/to/app-project
--compiled /abs/path/compiled/model_Google_Tensor_G5.tflite
[--compiled /abs/path/compiled/model_Google_Tensor_G6.tflite]
--fallback /abs/path/model.tflite
[--pack-name NAME] [--model-name NAME] [--cpu-asset PATH|none]

1.  Run `check_compiled.py` on every file first (step 1), and package builds
    that passed the strict benchmark (step 3).
2.  Run with `--dry-run` first (any `python3`), then for real with a Python
    that has `ai-edge-litert`. The script refuses mismatched inputs and never
    replaces a directory it did not write.
3.  For 5b, show the user the verdict and the file list: `drop-in` (only
    model files change), `structural` (`UPDATE.md` section 2 lists the other
    changes; list them and ask how to proceed before anything else) or
    `up-to-date`. A `Runtime` check of `device` means the new builds must pass
    the strict benchmark with the app's own runtime library first (`UPDATE.md`
    section 4).
4.  Paste the script's `**Packaged:** ...` or `**Update plan:** ...` block
    verbatim, add one line (the delivery mode or the verdict in plain words,
    and "next: `<output-dir>/INTEGRATION.md`" or `UPDATE.md`), then ask: "The
    integration plan is ready (`INTEGRATION.md`). Do you want me to apply it
    to your project?" (or "The update plan is ready (`UPDATE.md`). ..."). If
    yes, go to step 6. Publishing to Google Play stays with the user.

Exit codes: 0 ok, 2 bad input (missing file, duplicate SoC, invalid name,
output exists, a build that differs from its sidecar, or a project
`--update-project` cannot update as is), 3 wrong model kind (uncompiled
`--compiled`, compiled `--fallback`), 5 LiteRT not importable or its exporter
API changed (the message has the fix), 7 post-export verification failed
(nothing is left at `--output-dir`).
<venv>/bin/python SKILL_DIR/scripts/package_ai_pack.py
--update-project /abs/path/to/app-project
--compiled /abs/path/compiled/model_Google_Tensor_G5.tflite
[--compiled /abs/path/compiled/model_Google_Tensor_G6.tflite]
--fallback /abs/path/model.tflite
[--pack-name NAME] [--model-name NAME] [--cpu-asset PATH|none]

1.  首先对每个文件运行 `check_compiled.py`(第 1 步),并且只打包通过严格基准测试的构建产物(第 3 步)。
2.  首先使用任意 `python3` 运行 `--dry-run`,然后使用安装了 `ai-edge-litert` 的 Python 实际运行。脚本会拒绝不匹配的输入,并且绝不会替换非由脚本写入的目录。
3.  对于 5b,向用户展示结论和文件列表:`drop-in`(仅模型文件发生变化)、`structural`(`UPDATE.md` 第 2 节列出其他变更;列出这些变更,并在继续操作前询问用户)或 `up-to-date`。如果 `Runtime` 检查结果为 `device`,则新构建产物必须先使用应用自身的 runtime library 通过严格基准测试(`UPDATE.md` 第 4 节)。
4.  原样粘贴脚本的 `**Packaged:** ...` 或 `**Update plan:** ...` 代码块,添加一行(用通俗语言说明交付模式或结论,并注明“next: `<output-dir>/INTEGRATION.md`”或 `UPDATE.md`),然后询问:“集成方案已就绪(`INTEGRATION.md`)。是否要将其应用到你的项目?”(更新场景:“更新方案已就绪(`UPDATE.md`)……”)。如果用户同意,则进入第 6 步。发布到 Google Play 仍由用户负责。

退出码:0 表示成功,2 表示输入无效(缺少文件、SoC 重复、名称无效、输出已存在、构建产物与其 sidecar 不一致,或项目无法直接通过 `--update-project` 更新),3 表示模型类型错误(`--compiled` 传入未编译模型,或 `--fallback` 传入已编译模型),5 表示无法导入 LiteRT 或其 exporter API 已改变(消息中包含修复方法),7 表示导出后验证失败(`--output-dir` 中不会残留任何内容)。

Step 6 - Apply the plan to the user's project (when they say yes)

第 6 步 - 在用户同意后,将方案应用到用户项目

You apply the plan; the user approves it once. Before changing anything, show the whole plan and get one approval. Then ask separately for each risky item (the check marks them): upgrading the Android Gradle plugin or Kotlin, adding Kotlin, raising
minSdk
, limiting the app to arm64-v8a (devices without it can no longer install the app), changing the app's LiteRT version, replacing project files that differ from the package, removing a SoC. A declined item is skipped; say what will not work without it.
  • 6a - New integration: the plan is the
    fail
    items of a read-only check (any
    python3
    , no LiteRT), each with its fix:
    bash
    python3 SKILL_DIR/scripts/package_ai_pack.py --check-project /abs/path/to/app-project \
      --package <output-dir of step 5a> [--app-module app] [--json]
  • 6b - Update: the plan is
    UPDATE.md
    (the file table and section 2).
Follow the procedure in references/app-integration.md (backups, edits merged in the project's own style, re-check, one build). Never delete or reformat unrelated code, and never touch signing configs.
由你应用方案;用户只需批准一次。更改任何内容前,先展示完整方案并取得一次批准。然后针对每个有风险的项目分别询问(检查会将其标记出来):升级 Android Gradle plugin 或 Kotlin、添加 Kotlin、提高
minSdk
、将应用限制为 arm64-v8a(不支持该 ABI 的设备将无法再安装应用)、更改应用的 LiteRT 版本、替换与 package 内容不同的项目文件、移除某个 SoC。用户拒绝某项时跳过该项;并说明缺少它时哪些功能将无法工作。
  • **6a - 新集成:**方案来自只读检查中的
    fail
    项(使用任意
    python3
    ,不需要 LiteRT),每项都附有修复方法:
    bash
    python3 SKILL_DIR/scripts/package_ai_pack.py --check-project /abs/path/to/app-project \
      --package <output-dir of step 5a> [--app-module app] [--json]
  • **6b - 更新:**方案为
    UPDATE.md
    (文件表格和第 2 节)。
按照 references/app-integration.md 中的流程执行(备份、以项目自身风格合并编辑、重新检查、执行一次构建)。绝不删除或重新格式化无关代码,也绝不修改 signing configs。

Step 7 - Install and check on a Pixel (when the user agrees)

第 7 步 - 在 Pixel 上安装并检查(用户同意时)

Builds the user's app, installs it on the attached Pixel the way Google Play would (bundletool local testing, for the phone's device group), and reads the loader's
LITERT_TENSOR_RESULT
line:
bash
python3 SKILL_DIR/scripts/install_and_verify.py --project /abs/path/to/app-project \
  [--app-module app] [--variant debug|release|<flavor>Debug] [--serial <id>] \
  [--uninstall-first] [--dry-run]
Run
--dry-run
first: it shows the phone, its device group and whether the NPU path is expected. Exit 7 means the app is already installed and local testing cannot update it: ask the user (uninstalling deletes the app's data on the phone), then rerun with
--uninstall-first
. The app must call
TensorModelLoader.load()
; if it loads the model on a particular screen, tell the user to open it while the script waits. Report the script's block verbatim:
"path":"npu"
is the pass; for
"path":"cpu"
on a phone the app has a build for, the
npu_error
says why. Requirements and exit codes: references/app-integration.md.
构建用户应用,以类似 Google Play 的方式将其安装到已连接的 Pixel(使用 bundletool local testing,并选择手机的 device group),然后读取 loader 的
LITERT_TENSOR_RESULT
行:
bash
python3 SKILL_DIR/scripts/install_and_verify.py --project /abs/path/to/app-project \
  [--app-module app] [--variant debug|release|<flavor>Debug] [--serial <id>] \
  [--uninstall-first] [--dry-run]
首先运行
--dry-run
:它会显示手机、其 device group,以及是否应走 NPU path。退出码 7 表示应用已经安装,local testing 无法更新它:询问用户(卸载会删除手机上的应用数据),然后使用
--uninstall-first
重新运行。应用必须调用
TensorModelLoader.load()
;如果应用在特定页面加载模型,应告知用户在脚本等待时打开该页面。原样报告脚本代码块:
"path":"npu"
表示通过;对于有对应构建产物的手机,如果出现
"path":"cpu"
,则
npu_error
会说明原因。要求和退出码见:references/app-integration.md。

Rules

规则

  • ALWAYS run
    check_compiled.py
    before compiling or benchmarking, on every model file involved - the compiler cannot recompile a compiled model, and the TPU cannot run an uncompiled one.
  • ALWAYS verify the device SoC matches the model's stamp (the benchmark script does) - a
    Tensor_G5
    model on a
    Tensor_G6
    phone fails with an opaque dispatch error, wasting a debugging round.
  • ALWAYS use the original uncompiled model for
    --cpu-baseline
    - compiled models contain
    DISPATCH_OP
    nodes that only the NPU can execute.
  • ALWAYS clean up the per-run device directory (automatic; after an interrupted run list the base first, then remove only its
    run-*
    entries:
    adb shell ls -la <base>
    and
    adb shell rm -rf <base>/run-*
    , where
    <base>
    is
    /data/local/tmp/litert_benchmark
    or the
    --device-dir
    that was used). NEVER
    rm -rf
    the base directory itself - users keep their own binaries and models there.
  • NEVER combine
    --use_cpu=true
    with
    --require_full_delegation=true
    -
    benchmark_model
    aborts on that combination.
  • NEVER install packages into the system interpreter or without telling the user; use a virtualenv so pinned LiteRT versions do not break other work.
  • NEVER type LiteRT version numbers into install commands or download URLs yourself - take them from
    compile_model.py --print-install
    and the scripts' default URLs, which all read
    versions.json
    , so every job uses the pair that was verified together. Use another release only when the user asks for one (
    --benchmark-url
    , or the pip pair at that version), and say so in the report. Changing the pin is a maintainer task (references/contributing.md).
  • NEVER treat macOS, Windows, or ARM hosts as a compile blocker or give manual browser Colab notebook steps - run
    compile_model.py --colab
    directly.
  • NEVER leave adb, benchmark, or Colab VM sessions running in the background - every command here is bounded by a timeout and returns before the next step (
    compile_model.py --colab
    stops any Colab VM it creates unless
    --keep-colab-session
    is passed).
  • NEVER guess compile flags or overwrite an existing output without asking (
    compile_model.py
    exits 2; use
    --output-name
    or ask before
    --overwrite
    ).
  • ALWAYS package for an app with
    package_ai_pack.py
    (it calls LiteRT's exporter and verifies the result), and with
    --update-project
    (step 5b) when the app already ships Google Tensor builds; NEVER create or edit AI pack folders, the device-targeting XML or the runtime modules by hand, and NEVER copy a new package's
    ai_packs/
    over an existing pack - the app's group names and the place of its CPU copy can differ.
  • ALWAYS ask before changing the user's project, show the whole plan, and get one approval; ask separately for each risky item (step 6) and before
    install_and_verify.py --uninstall-first
    . NEVER change signing configs.
  • NEVER put a compiled build where the app runs on the CPU (base assets,
    --fallback
    ): only the original uncompiled model.
  • NEVER publish to Google Play, sign, or upload anything for the user; the skill stops at the package and the local-testing steps.
  • 在编译或基准测试之前,始终对涉及的每个模型文件运行
    check_compiled.py
    - 编译器无法重新编译已编译模型,而 TPU 无法运行未编译模型。
  • 始终验证设备 SoC 与模型 stamp 是否匹配(基准测试脚本会执行验证) - 在
    Tensor_G6
    手机上运行
    Tensor_G5
    模型会触发难以理解的 dispatch error,白白浪费一轮调试。
  • --cpu-baseline
    始终使用原始未编译模型 - 编译后的模型包含只能由 NPU 执行的
    DISPATCH_OP
    节点。
  • 始终清理每次运行的设备目录(自动完成;运行被中断后,先列出 base,再仅删除其中的
    run-*
    项:
    adb shell ls -la <base>
    和
    adb shell rm -rf <base>/run-*
    ,其中
    <base>
    为
    /data/local/tmp/litert_benchmark
    或所使用的
    --device-dir
    )。绝不删除 base 目录本身 - 用户会在其中保存自己的二进制文件和模型。
  • 绝不将
    --use_cpu=true
    与
    --require_full_delegation=true
    结合使用 -
    benchmark_model
    会因这种组合而中止。
  • 绝不在不告知用户的情况下将 packages 安装到 system interpreter 中;使用 virtualenv,避免固定版本的 LiteRT 破坏其他工作。
  • 绝不在安装命令或下载 URL 中手动输入 LiteRT 版本号 - 应从
    compile_model.py --print-install
    和脚本默认 URL 中获取;这些内容都读取
    versions.json
    ,从而确保每个任务使用经过共同验证的版本组合。仅当用户要求使用其他 release 时才使用(
    --benchmark-url
    ,或该版本的 pip pair),并在报告中说明。更改固定版本是维护者任务(references/contributing.md)。
  • 绝不将 macOS、Windows 或 ARM 主机视为编译阻塞条件,也绝不提供手动操作浏览器 Colab notebook 的步骤 - 直接运行
    compile_model.py --colab
    。
  • 绝不留下在后台运行的 adb、基准测试或 Colab VM sessions - 此处的每个命令都有 timeout 限制,并会在下一步骤前返回(
    compile_model.py --colab
    会停止其创建的 Colab VM,除非传入
    --keep-colab-session
    )。
  • 绝不猜测编译 flags,也绝不在未询问的情况下覆盖现有输出(
    compile_model.py
    会以退出码 2 退出;使用
    --output-name
    或询问后使用
    --overwrite
    )。
  • 始终使用
    package_ai_pack.py
    为应用打包(它会调用 LiteRT exporter 并验证结果);当应用已经包含 Google Tensor 构建产物时,使用
    --update-project
    (第 5b 步)。绝不手动创建或编辑 AI pack 文件夹、设备定向 XML 或 runtime modules,也绝不将新 package 的
    ai_packs/
    复制到现有 pack 上 - 应用的 group 名称和 CPU 副本位置可能不同。
  • 更改用户项目之前始终先询问,展示完整方案并取得一次批准;针对每个有风险的项目分别询问(第 6 步),并在执行
    install_and_verify.py --uninstall-first
    前询问。绝不修改 signing configs。
  • 绝不将编译后的构建产物放在应用运行 CPU path 的位置(base assets、
    --fallback
    );那里只能使用原始未编译模型。
  • 绝不替用户发布到 Google Play、签名或上传任何内容;此 skill 止步于打包和 local testing 步骤。

Troubleshooting

Troubleshooting

Every script error names its fix; look up the error line or exit code in references/troubleshooting.md.
每个脚本错误都会指出修复方法;在 references/troubleshooting.md 中查找错误行或退出码。

Public references

公共参考

Contributions

贡献

To contribute or modify this skill or its references, follow the contribution guidelines before making changes.
如需为此 skill 或其参考资料贡献内容或进行修改,请先遵循 contribution guidelines。