local-ai-app-integration

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Local AI App Integration (Embeddable Lemonade)

本地AI应用集成（Embeddable Lemonade）

Add a local AI mode to an existing app that already talks to a cloud AI API (OpenAI, Anthropic, or Ollama-compatible). The app launches

lemond

, the Embeddable Lemonade binary, as a private subprocess and the existing client talks to it on

http://localhost:PORT/api/v1

. The user gets local, private, hardware-optimized inference (CPU, AMD iGPU/dGPU, XDNA2 NPU) with no separate install.

What you'll end up with: one new launcher module (~30 lines), three mandatory changes to the existing HTTP client (

base_url

api_key

, and a 120-second HTTP timeout), one vendored binary under

vendor/lemonade/

为已对接云AI API（OpenAI、Anthropic或兼容Ollama）的现有应用添加本地AI模式。应用会启动

lemond

（Embeddable Lemonade的二进制文件）作为私有子进程，现有客户端通过

http://localhost:PORT/api/v1

与其通信。用户无需单独安装，即可获得本地、私有、硬件优化的推理能力（支持CPU、AMD iGPU/dGPU、XDNA2 NPU）。

最终成果： 一个新的启动模块（约30行代码），对现有HTTP客户端的三处必填修改（

base_url

、

api_key

和120秒HTTP超时），以及

vendor/lemonade/

目录下的一个内置二进制文件。

When this skill is the right tool

何时适合使用该技能

Use this skill when all of the following are true:

The app already calls a cloud AI service over HTTP (OpenAI Chat Completions, Anthropic Messages, or Ollama).
The user wants that AI to run on the end-user's PC, with the AI engine bundled into the app, not as a separate user install.
The target platform is Windows x64 or Linux x64 (macOS embeddable is in beta).

If the user instead wants a system-wide Lemonade Server (one install, shared across apps), do not use this skill; point them at

https://lemonade-server.ai/install_options.html

and the standard OpenAI base URL

http://localhost:13305/api/v1

当所有以下条件都满足时，使用该技能：

应用已通过HTTP调用云AI服务（OpenAI Chat Completions、Anthropic Messages或Ollama）。
用户希望AI运行在终端用户的PC上，且AI引擎被打包到应用中，而非让用户单独安装。
目标平台为Windows x64或Linux x64（macOS嵌入式版本处于测试阶段）。

如果用户需要的是系统级的Lemonade Server（一次安装，多应用共享），请勿使用该技能；请引导他们访问

https://lemonade-server.ai/install_options.html

，并使用标准OpenAI基础URL

http://localhost:13305/api/v1

。

The opinionated path

标准化实施流程

This skill follows one fixed sequence. Do not deviate without a stated reason.

[ ] 1. Survey the app's current AI integration
[ ] 2. Pick a model + backend profile
[ ] 3. Place Embeddable Lemonade in the app's tree (full package, not just the binary)
[ ] 4. Add a `lemond` launcher (subprocess + API key + port + per-stage logging)
[ ] 5. Re-point the existing client at lemond (base_url, api_key, 120s timeout — all three required)
[ ] 6. Wait for /api/v1/health, install backend, then PULL the model before first use
[ ] 7. Wire shutdown and error recovery

Track progress against this checklist. Move on only when each step verifies.

Log every stage. A local integration has many silent failure points — spawn, health, backend install, model download, first inference. Without a log line at each transition, "nothing happened" is indistinguishable from "broke at stage 3." Emit one clear line per stage as you build (see Step 4); the most common dead-end in this integration — a blank result with no error — is invisible without them.

该技能遵循固定的实施步骤，无特殊原因请勿偏离。

[ ] 1. 调研应用当前的AI集成情况
[ ] 2. 选择模型+后端配置
[ ] 3. 将Embeddable Lemonade放置到应用目录树中（完整包，而非仅二进制文件）
[ ] 4. 添加`lemond`启动器（子进程+API密钥+端口+分阶段日志）
[ ] 5. 将现有客户端指向lemond（base_url、api_key、120秒超时——三者均为必填）
[ ] 6. 等待/api/v1/health接口返回成功，安装后端，然后在首次使用前拉取模型
[ ] 7. 配置关闭流程与错误恢复机制

对照此清单跟踪进度，只有当前步骤验证通过后才能进入下一步。

记录每个阶段的日志。 本地集成存在许多无提示的故障点——进程启动、健康检查、后端安装、模型下载、首次推理。如果每个阶段没有日志记录，“无响应”无法区分是“在第3阶段崩溃”还是其他问题。构建时为每个阶段输出清晰的日志行（参见步骤4）；集成中最常见的死胡同——无错误提示的空白结果——没有日志就无法排查。

Step 1: Survey the app

步骤1：调研应用

Find every place the app currently calls a cloud AI API. Search the repo for:

openai

OpenAI(

chat.completions

responses.create

```
anthropic
```
,
```
Anthropic(
```
,
```
messages.create
```

api.openai.com

api.anthropic.com

localhost:11434

(Ollama)

```
OPENAI_API_KEY
```
,
```
ANTHROPIC_API_KEY
```

Record three things before continuing:

Client library and language (e.g.,

openai-python

openai-node

@anthropic-ai/sdk

go-openai

, raw

fetch

Modalities used: text chat, tool calling, embeddings, image gen, transcription, TTS. This drives the model + backend choice in Step 2.
One single place where the base URL and API key are constructed. If there isn't one, refactor to one before going further. Local-mode toggling must flip exactly one config object.
Any API-key gating that blocks the app before a key is entered (onboarding walls, validators that reject empty keys, startup checks that disable AI until a key exists). Note each one — Step 5 bypasses them in local mode.

找出应用中所有调用云AI API的位置。在代码库中搜索：

openai

、

OpenAI(

、

chat.completions

、

responses.create

```
anthropic
```
、
```
Anthropic(
```
、
```
messages.create
```

api.openai.com

、

api.anthropic.com

、

localhost:11434

（Ollama）

```
OPENAI_API_KEY
```
、
```
ANTHROPIC_API_KEY
```

继续下一步前，记录以下三点：

客户端库与语言（例如：
```
openai-python
```
、
```
openai-node
```
、
```
@anthropic-ai/sdk
```
、
```
go-openai
```
、原生
```
fetch
```
）。
使用的模态：文本聊天、工具调用、嵌入、图像生成、转录、TTS。这将决定步骤2中的模型+后端选择。
构建base URL和API密钥的统一位置。如果没有，先重构为统一位置再继续。本地模式切换必须仅修改一个配置对象。
任何API密钥校验逻辑（引导页拦截、拒绝空密钥的验证器、启动时检查AI功能是否禁用直到密钥存在）。记录每一处——步骤5将在本地模式下绕过这些逻辑。

Step 2: Pick a model + backend profile

步骤2：选择模型+后端配置

Choose one default profile based on the app's primary modality. Do not ship a buffet. Ship one good default and document how the user can override it.

App's primary need	Default model	Recipe	Why
General chat / assistant	`Qwen3-4B-GGUF`	`llamacpp`	Small, fast, good tool calling, fits 8GB systems
Coding assistant	`Qwen2.5-Coder-7B-Instruct-GGUF`	`llamacpp`	Strong code, runs on iGPU
Vision / multimodal chat	`Gemma-4-E2B-it-GGUF`	`llamacpp`	Small multimodal default
NPU-first on Ryzen AI	`Llama-3.2-3B-Instruct-Hybrid`	`ryzenai-llm`	XDNA2 NPU on Windows
Speech-to-text (Windows)	`Whisper-Large-v3-Turbo`	`whispercpp`	One model; probe picks NPU → iGPU/dGPU → CPU automatically
Speech-to-text (Linux NPU)	`whisper-v3-turbo-FLM`	`flm`	Linux NPU path; falls back to `whispercpp` iGPU/CPU off-NPU
Text-to-speech	`kokoro-v1`	`kokoro`	CPU-only, low latency
Image generation	`SDXL-Turbo`	`sd-cpp`	Single-step generation

For the LLM backend, default to

llamacpp

and let

lemond

pick

rocm

→

vulkan

→

cpu

automatically by leaving

llamacpp_backend

unset. Override only if the app has hard hardware requirements.

Scope: this skill selects a backend once at integration time on the developer's machine. Runtime fallback based on the end user's hardware is out of scope. Bundle

vulkan

as the universal fallback so the app works on any machine. If the dev machine has an NPU and the chosen recipe supports it, the skill will use the NPU backend — otherwise it falls back to

vulkan

Note: having an NPU does not mean every recipe supports NPU. Confirm the recipe/backend pair is
installed
or
installable
via
GET /api/v1/system-info
before committing to it. See reference.md for per-recipe decision rules.

For more options and tradeoffs, see reference.md.

根据应用的主要模态选择一个默认配置。不要提供多种选项，只需提供一个优质默认配置，并记录用户如何自定义配置。

应用主要需求	默认模型	方案	原因
通用聊天/助手	`Qwen3-4B-GGUF`	`llamacpp`	体积小、速度快、工具调用能力强，适配8GB内存的系统
编码助手	`Qwen2.5-Coder-7B-Instruct-GGUF`	`llamacpp`	代码能力强，可在iGPU上运行
视觉/多模态聊天	`Gemma-4-E2B-it-GGUF`	`llamacpp`	轻量多模态默认选项
优先使用Ryzen AI的NPU	`Llama-3.2-3B-Instruct-Hybrid`	`ryzenai-llm`	适配Windows系统的XDNA2 NPU
语音转文本（Windows）	`Whisper-Large-v3-Turbo`	`whispercpp`	单模型；自动优先选择NPU → iGPU/dGPU → CPU
语音转文本（Linux NPU）	`whisper-v3-turbo-FLM`	`flm`	Linux NPU路径；非NPU环境下 fallback 到 `whispercpp` iGPU/CPU
文本转语音	`kokoro-v1`	`kokoro`	仅CPU运行，低延迟
图像生成	`SDXL-Turbo`	`sd-cpp`	单步生成

对于LLM后端，默认使用

llamacpp

，并通过留空

llamacpp_backend

让

lemond

自动选择

rocm

→

vulkan

→

cpu

。仅当应用有严格硬件要求时才手动指定。

范围： 该技能在开发者机器上集成时一次性选择后端。基于终端用户硬件的运行时 fallback 不在本技能范围内。打包时包含

vulkan

作为通用 fallback，确保应用可在任意机器上运行。如果开发者机器有NPU且所选方案支持，则技能会使用NPU后端——否则 fallback 到

vulkan

。

注意： 拥有NPU不代表所有方案都支持NPU。在确定方案前，需通过
GET /api/v1/system-info
确认方案/后端组合是已安装或可安装的。请查看reference.md了解各方案的决策规则。

如需更多选项和权衡，请查看reference.md。

Step 3: Place Embeddable Lemonade in the app's tree and install backends

步骤3：将Embeddable Lemonade放置到应用目录树并安装后端

Get the embeddable artifact from the latest Lemonade release:

https://github.com/lemonade-sdk/lemonade/releases/latest

Download the file matching your target OS:

Windows:

lemonade-embeddable-{VERSION}-windows-x64.zip

Linux:

lemonade-embeddable-{VERSION}-ubuntu-x64.tar.gz

Don't hand-build the download URL from the tag. The git tag carries a leading
v
(e.g.
v10.8.0
) but the asset filename strips it (
lemonade-embeddable-10.8.0-...
), so using the tag verbatim 404s. Ask the GitHub API for the asset by its stable name pattern and use the URL it returns, as below — this stays correct across version and naming changes.

First, create the target directory — it does not exist in a fresh repo:

powershell

undefined

从Lemonade最新版本获取嵌入式包：

https://github.com/lemonade-sdk/lemonade/releases/latest

下载匹配目标操作系统的文件：

Windows：

lemonade-embeddable-{VERSION}-windows-x64.zip

Linux：

lemonade-embeddable-{VERSION}-ubuntu-x64.tar.gz

不要通过标签手动构建下载URL。 Git标签带有前缀
v
（例如
v10.8.0
），但资产文件名会去掉该前缀（
lemonade-embeddable-10.8.0-...
），直接使用标签会导致404错误。请通过GitHub API按稳定名称模式获取资产URL，如下所示——这样可确保在版本和命名变更时依然正确。

首先创建目标目录——全新代码库中该目录不存在：

powershell

undefined

Windows

New-Item -ItemType Directory -Force vendor\lemonade


```bash

New-Item -ItemType Directory -Force vendor\lemonade


```bash

Linux

mkdir -p vendor/lemonade


Then download and unpack on Windows (PowerShell):

```powershell
$rel = Invoke-RestMethod https://api.github.com/repos/lemonade-sdk/lemonade/releases/latest
$asset = $rel.assets | Where-Object { $_.name -like "lemonade-embeddable-*-windows-x64.zip" } | Select-Object -First 1
Invoke-WebRequest $asset.browser_download_url -OutFile lemond.zip
Expand-Archive lemond.zip -DestinationPath "$env:TEMP\lemond-unpack"
$folder = $asset.name -replace '\.zip$',''   # unpacked dir = asset name without .zip
Copy-Item -Recurse "$env:TEMP\lemond-unpack\$folder\*" vendor\lemonade\

mkdir -p vendor/lemonade


然后在Windows（PowerShell）中下载并解压：

```powershell
$rel = Invoke-RestMethod https://api.github.com/repos/lemonade-sdk/lemonade/releases/latest
$asset = $rel.assets | Where-Object { $_.name -like "lemonade-embeddable-*-windows-x64.zip" } | Select-Object -First 1
Invoke-WebRequest $asset.browser_download_url -OutFile lemond.zip
Expand-Archive lemond.zip -DestinationPath "$env:TEMP\lemond-unpack"
$folder = $asset.name -replace '\.zip$',''   # 解压后的目录 = 资产名称去掉.zip
Copy-Item -Recurse "$env:TEMP\lemond-unpack\$folder\*" vendor\lemonade\

Sanity check: resources/ must be nested under vendor\lemonade\ (not flattened)

完整性检查：resources/必须嵌套在vendor\lemonade\下（不能扁平化）

if (-not (Test-Path vendor\lemonade\resources*.json)) { throw "resources/ missing — re-extract and copy again" }


On Linux (bash):

```bash
URL=$(curl -s https://api.github.com/repos/lemonade-sdk/lemonade/releases/latest \
  | grep browser_download_url | grep ubuntu-x64.tar.gz | cut -d'"' -f4)
curl -L "$URL" | tar -xz --strip-components=1 -C vendor/lemonade

Copy the full package, not just the binary. The archive contains
lemond[.exe]
,
lemonade[.exe]
,
LICENSE
, and
resources/
. The
resources/
directory is required — without it lemond starts and passes the health check but fails on every model and backend request. Copying only the binary produces a server that looks healthy but cannot function.

lemond
vs
lemonade
CLI:
lemond
is the embedded server binary that ships with the app. The
lemonade
CLI is a separate packaging tool used only during development/build time to install backends. The same embeddable archive unpacked above already contains a matching
lemonade[.exe]
next to
lemond[.exe]
, so its version aligns with the bundled
lemond
. Do not
pip install lemonade-sdk
to get it: the PyPI package is a separate, older release line whose ports, model names, and install API do not match the
lemond
bundled here, and mixing the two is a known source of silent version mismatches. Keep the
lemonade
CLI,
lemond
, and the backends all from the one release downloaded in this step so their versions stay aligned.

The expected layout after setup (first run + backend install). A freshly unzipped package contains only

lemond[.exe]

lemonade[.exe]

LICENSE

, and

resources/

— the items below are created later, as their comments note:

vendor/lemonade/
  lemond[.exe]                     # the only binary the app ships
  LICENSE
  config.json                      # generated on first run; commit a seed copy
  resources/
    server_models.json             # do not edit; use GET /api/v1/models at runtime
    backend_versions.json
  bin/                             # backends bundled at packaging time
    llamacpp/vulkan/llama-server[.exe]
  models/                          # pre-bundled model weights (optional)
    models--unsloth--Qwen3-4B-GGUF/

server_models.json
: Do not edit or rely on this file. It can be stale. The only authoritative model list is
GET /api/v1/models
on a running
lemond
instance with the backend already installed.

Bundle decisions: pick deliberately

Backends: Bundle
```
llamacpp:vulkan
```
at packaging time (works on every GPU). Install
```
llamacpp:rocm
```
at first run on supported AMD systems via
```
POST /api/v1/install
```
after probing
```
GET /api/v1/system-info
```
. Never ship every backend, or the artifact balloons.
Models: Either bundle the default model under
```
models/
```
(offline install, larger installer) or pull on first run with
```
POST /api/v1/pull
```
(smaller installer, needs network). Pick one and document it.
models_dir
: Set to
```
./models
```
in
```
config.json
```
to keep weights private to the app. Leave as
```
auto
```
only if the user explicitly wants to share weights with other apps.

Backend install timing — two distinct paths:

Packaging time (developer machine, before bundling). Use the lemonade CLI that shipped inside
vendor/lemonade/
so it matches the bundled
lemond
version (prefix with
./
or the full path):
vendor/lemonade/lemonade backends install llamacpp:vulkan
vendor/lemonade/lemonade backends install flm:npu    # Windows NPU path only
This bakes the backend binaries into
vendor/lemonade/bin/
before the app ships.
lemond
does not need to be running. Use a modern
lemonade
CLI whose version matches the bundled
lemond
(the copy in the archive you unpacked works); do not
pip install lemonade-sdk
for it.
First-run / runtime (user's machine, after
lemond
is running):
http
POST /api/v1/install
{"recipe": "llamacpp", "backend": "rocm"}
Use this for hardware-specific backends (e.g.
llamacpp:rocm
) that cannot be bundled universally.
lemond
must already be running (Step 4 complete).

if (-not (Test-Path vendor\lemonade\resources*.json)) { throw "resources/缺失——重新解压并复制" }


在Linux（bash）中：

```bash
URL=$(curl -s https://api.github.com/repos/lemonade-sdk/lemonade/releases/latest \
  | grep browser_download_url | grep ubuntu-x64.tar.gz | cut -d'"' -f4)
curl -L "$URL" | tar -xz --strip-components=1 -C vendor/lemonade

复制完整包，而非仅二进制文件。 压缩包包含
lemond[.exe]
、
lemonade[.exe]
、
LICENSE
和
resources/
。
resources/
目录是必需的——没有它，lemond可以启动并通过健康检查，但所有模型和后端请求都会失败。仅复制二进制文件会导致服务器看似正常但无法运行。

lemond
vs
lemonade
CLI：
lemond
是随应用打包的嵌入式服务器二进制文件。
lemonade
CLI是仅在开发/构建时使用的独立打包工具，用于安装后端。上述解压的嵌入式包中已包含与
lemond
版本匹配的
lemonade[.exe]
，因此版本保持一致。请勿通过
pip install lemonade-sdk
获取该工具：PyPI包是独立的旧版本分支，其端口、模型名称和安装API与此处打包的
lemond
不匹配，混用会导致无提示的版本不兼容问题。确保
lemonade
CLI、
lemond
和后端均来自本步骤下载的同一版本，以保持版本一致。

设置完成后的预期目录结构（首次运行+后端安装后）。刚解压的包仅包含

lemond[.exe]

、

lemonade[.exe]

、

LICENSE

和

resources/

——以下项目会在后续创建，注释说明其创建时机：

vendor/lemonade/
  lemond[.exe]                     # 应用仅需打包该二进制文件
  LICENSE
  config.json                      # 首次运行时生成；提交一个初始副本
  resources/
    server_models.json             # 请勿编辑；运行时使用GET /api/v1/models获取
    backend_versions.json
  bin/                             # 打包时内置的后端
    llamacpp/vulkan/llama-server[.exe]
  models/                          # 预打包的模型权重（可选）
    models--unsloth--Qwen3-4B-GGUF/

server_models.json
：请勿编辑或依赖该文件，它可能已过期。唯一权威的模型列表是运行中的
lemond
实例上的
GET /api/v1/models
接口（需已安装后端）。

打包决策：谨慎选择

后端： 打包时内置
```
llamacpp:vulkan
```
（适用于所有GPU）。在支持AMD系统的首次运行时，通过
```
GET /api/v1/system-info
```
探测后，调用
```
POST /api/v1/install
```
安装
```
llamacpp:rocm
```
。切勿打包所有后端，否则包体积会大幅增加。
模型： 要么将默认模型打包在
```
models/
```
下（离线安装，安装包更大），要么在首次运行时通过
```
POST /api/v1/pull
```
拉取（安装包更小，需要网络）。二选一并记录决策。
models_dir
：在
```
config.json
```
中设置为
```
./models
```
，使权重仅对当前应用私有。仅当用户明确希望与其他应用共享权重时，才保留为
```
auto
```
。

后端安装时机——两种不同路径：

打包时（开发者机器，打包前）。使用
vendor/lemonade/
目录下的lemonade CLI，确保其与打包的
lemond
版本匹配（前缀加
./
或完整路径）：
vendor/lemonade/lemonade backends install llamacpp:vulkan
vendor/lemonade/lemonade backends install flm:npu    # 仅适用于Windows NPU路径
这会在应用打包前将后端二进制文件嵌入
vendor/lemonade/bin/
。无需运行
lemond
。使用与打包的
lemond
版本匹配的最新lemonade CLI（解压包中的副本即可）；请勿通过
pip install lemonade-sdk
获取。
首次运行/运行时（用户机器，
lemond
启动后）：
http
POST /api/v1/install
{"recipe": "llamacpp", "backend": "rocm"}
适用于无法通用打包的硬件特定后端（例如
llamacpp:rocm
）。
lemond
必须已启动（步骤4完成）。

Step 4: Add a

lemond

launcher

步骤4：添加

lemond

启动器

Write the launcher as a new module named lemond_launcher.py
(or

lemond_launcher.<ext>

for the app's language). It is a thin process supervisor. Its only jobs:

Generate a fresh random API key:
```
key = secrets.token_urlsafe(32)
```
Pick a free localhost port: bind a
```
socket
```
to port 0, read back the assigned port, close it.

Spawn lemond as a

subprocess

subprocess.Popen([LEMOND_BIN, LEMOND_DIR, "--port", str(port)], env={**os.environ, "LEMONADE_API_KEY": key})

Poll
```
GET /api/v1/health
```
with
```
Authorization: Bearer {key}
```
in a loop until HTTP 200 — this is the only correct readiness check.
Expose the chosen
```
port
```
and
```
key
```
to the rest of the app.

Log one line per lifecycle stage. Build the logging in from the start — not as an afterthought when something breaks. Each silent transition needs a visible marker so a failure points at the exact stage. Aim for:
[lemond] Starting on port <port>
[lemond] Healthy on port <port>
[lemond] <recipe>:<backend> installed        (or: already installed / install failed)
[lemond] Pulling model <name>...             then: Model <name> ready  (or: pull returned <status>)
[local]  <modality> result: <value>          (first inference output — empty string here = unpulled model)
Logging the first inference result verbatim is what turns the silent-empty failure (Step 6) from a multi-hour mystery into a one-line diagnosis. Route these through the app's normal logging so they can be quieted for release.

Dev-mode file watchers: If the app runs with a file watcher (Tauri, Electron, Next.js, Vite, etc.) that watches the source tree, ensure
vendor/lemonade/
is excluded from the watched paths. Lemond writes config and cache files at runtime; a watcher that picks these up will restart the app, kill the lemond subprocess, and spawn a new one on a new port — silently breaking any in-flight transcription. Add
vendor/
(or the equivalent) to the watcher's ignore list before testing.

Use the reference implementation from reference.md § Reference launchers directly — copy it verbatim and adapt only the

LEMOND_DIR

path. Do not write a launcher from scratch. The reference Python launcher uses

secrets

(for the API key),

socket

(for the free-port probe), and

subprocess

(to spawn lemond); the Node.js launcher uses the equivalent stdlib modules. Both handle port-race retries and health polling correctly.

Readiness is always determined by polling the exact endpoint

GET http://127.0.0.1:<port>/api/v1/health

and checking for HTTP 200 — never by reading

lemond

's stdout or stderr. Any health-check helper you write must hit that

/api/v1/health

path.

将启动器编写为名为**

lemond_launcher.py

**（或对应应用语言的

lemond_launcher.<ext>

）的新模块。它是一个轻量级进程管理器，仅需完成以下工作：

生成新的随机API密钥：
```
key = secrets.token_urlsafe(32)
```
选择一个空闲的本地端口：将
```
socket
```
绑定到端口0，读取分配的端口，然后关闭。

启动lemond作为子进程：

subprocess.Popen([LEMOND_BIN, LEMOND_DIR, "--port", str(port)], env={**os.environ, "LEMONADE_API_KEY": key})

循环调用
```
GET /api/v1/health
```
并携带
```
Authorization: Bearer {key}
```
，直到返回HTTP 200——这是唯一正确的就绪检查方式。
向应用其他部分暴露所选的
```
port
```
和
```
key
```
。

记录每个生命周期阶段的日志。 从一开始就加入日志功能——不要等到出现问题才添加。每个无提示的阶段转换都需要可见标记，以便故障能定位到具体阶段。目标日志如下：
[lemond] 在端口<port>启动
[lemond] 端口<port>健康检查通过
[lemond] <recipe>:<backend>已安装        （或：已安装/安装失败）
[lemond] 拉取模型<name>...             然后：模型<name>就绪 （或：拉取返回<status>）
[local]  <modality>结果：<value>          （首次推理输出——此处为空字符串表示模型未拉取）
记录首次推理结果原文可将无提示的空白结果（步骤6）从数小时的排查难题转化为一行诊断信息。将这些日志接入应用的常规日志系统，以便在发布版本中关闭日志输出。

开发模式文件监视器： 如果应用使用文件监视器（Tauri、Electron、Next.js、Vite等）监视源码目录，请确保
vendor/lemonade/
被排除在监视路径之外。Lemond运行时会写入配置和缓存文件；监视器检测到这些文件变更会重启应用，杀死lemond子进程，并在新端口启动新进程——这会无声地中断正在进行的转录。测试前将
vendor/
（或等效目录）添加到监视器的忽略列表。

直接使用reference.md § Reference launchers中的参考实现——原样复制并仅修改

LEMOND_DIR

路径。请勿从头编写启动器。参考Python启动器使用

secrets

（生成API密钥）、

socket

（探测空闲端口）和

subprocess

（启动lemond）；Node.js启动器使用等效的标准库模块。两者均正确处理端口冲突重试和健康检查轮询。

就绪状态始终通过轮询精确的端点

GET http://127.0.0.1:<port>/api/v1/health

并检查HTTP 200来确定——绝不要通过读取

lemond

的stdout或stderr判断。任何自定义的健康检查工具都必须调用该

/api/v1/health

路径。

Step 5: Re-point the existing client at

lemond

步骤5：将现有客户端指向

lemond

Make three changes to the app's existing client construction — all three are required, not optional:

Set
```
base_url
```
to
```
http://127.0.0.1:{port}/api/v1
```
Set
```
api_key
```
to the launcher key
Set the HTTP timeout to 120 seconds — this is mandatory, not optional

The 120-second timeout is not a tuning suggestion. The default on most HTTP clients is 30s, which is shorter than lemond's first-run model load time on real hardware. Without it the request silently times out and the UI shows nothing, which is indistinguishable from a broken integration.

Python (openai) — the exact change to make:

python

import httpx
from openai import OpenAI

proc, key, port = start_lemond()
client = OpenAI(
    base_url=f"http://127.0.0.1:{port}/api/v1",
    api_key=key,
    http_client=httpx.Client(timeout=120),  # required: 120s for first-run model load
)

For other clients:

Existing client	New `base_url`	New auth	Timeout
`openai-python`	`http://127.0.0.1:{port}/api/v1`	`api_key=key`	`httpx.Client(timeout=120)`
`openai-node`	`http://127.0.0.1:{port}/api/v1`	`apiKey: key`	`timeout: 120000`
`@anthropic-ai/sdk`	`http://127.0.0.1:{port}/api/v1`	`apiKey: key`	`timeout: 120000`
Raw `fetch` / `requests`	same	`Authorization: Bearer {key}`	set per-request
Ollama-compatible code	`http://127.0.0.1:{port}/api/v0`	pass key anyway	120s

The model identifier on requests stays a Lemonade model name (e.g.

Qwen3-4B-GGUF

), not the cloud name.

Local mode needs no cloud API key — at all. This is a defining property of local mode, not an edge case: there is no cloud service to authenticate to, so nothing should ever ask the user for a key. Any onboarding wall, validator, or startup check that demands one must not block local-mode users. Concretely:

Skip or auto-satisfy the key-entry screen in local mode.
Treat local mode as already-authorized in every validation path — an empty-key check must short-circuit to "valid" when the active mode is local, never throw "API key not configured".
Re-enable the gate only for cloud mode.

The

lemond

key from Step 4 is generated internally by the launcher and used only for the local loopback connection, so the user never sees or enters one; any UI placeholder (e.g.

"local"

) is fine. Flipping into local mode should never strand the user on a key-entry wall.

对应用现有客户端的构造进行三处修改——三者均为必填，不可省略：

将
```
base_url
```
设置为
```
http://127.0.0.1:{port}/api/v1
```
将
```
api_key
```
设置为启动器生成的密钥
将HTTP超时设置为120秒——这是必填项，不可省略

120秒超时不是调优建议。大多数HTTP客户端的默认超时为30秒，短于lemond在真实硬件上首次运行时的模型加载时间。如果不设置，请求会无声超时，UI无任何显示，无法与集成失败区分。

Python（openai）——具体修改：

python

import httpx
from openai import OpenAI

proc, key, port = start_lemond()
client = OpenAI(
    base_url=f"http://127.0.0.1:{port}/api/v1",
    api_key=key,
    http_client=httpx.Client(timeout=120),  # 必填：首次运行模型加载需120秒
)

其他客户端的修改：

现有客户端	新 `base_url`	新认证方式	超时设置
`openai-python`	`http://127.0.0.1:{port}/api/v1`	`api_key=key`	`httpx.Client(timeout=120)`
`openai-node`	`http://127.0.0.1:{port}/api/v1`	`apiKey: key`	`timeout: 120000`
`@anthropic-ai/sdk`	`http://127.0.0.1:{port}/api/v1`	`apiKey: key`	`timeout: 120000`
原生 `fetch` / `requests`	同上	`Authorization: Bearer {key}`	按请求设置
兼容Ollama的代码	`http://127.0.0.1:{port}/api/v0`	仍需传入密钥	120秒

请求中的模型标识符需使用Lemonade模型名称（例如

Qwen3-4B-GGUF

），而非云服务中的模型名称。

本地模式完全不需要云API密钥。 这是本地模式的核心特性，而非边缘情况：无需对接云服务，因此绝不应要求用户提供密钥。任何引导页拦截、验证器或启动检查都不得阻止本地模式用户。具体实现：

本地模式下跳过或自动通过密钥输入界面。
在所有验证路径中将本地模式视为已授权——当启用本地模式时，空密钥检查必须直接判定为“有效”，绝不能抛出“API密钥未配置”错误。
仅在云模式下重新启用密钥校验。

步骤4中生成的

lemond

密钥由启动器内部生成，仅用于本地回环连接，用户永远不会看到或输入该密钥；UI中可显示占位符（例如

"local"

）。切换到本地模式绝不应让用户卡在密钥输入界面。

Step 6: Health, backend, then pull the model — before first inference

步骤6：健康检查、后端安装、拉取模型——在首次推理前完成

GET /api/v1/health

returning 200 means the server is up. It does not mean inference will work. Before the first real request succeeds, three more things must be true: the backend for your modality is installed, the model's weights are downloaded to disk, and (on the first call) the model is loaded into memory. Treating health=200 as "ready" is the single biggest cause of a broken-looking integration.

Do not call
POST /api/v1/load
at startup. Lemond lazy-loads the model into memory on the first inference request and handles that step on its own. Pre-loading is unreliable across lemond versions (the

/load

request body shape has changed between releases) and a malformed call can crash or destabilise the server before the user takes any action. Loading is the one step you let lemond do lazily — pulling is not.

GET /api/v1/health

返回200仅表示服务器已启动，不代表推理可以正常运行。首次真实请求成功前，还需满足三个条件：对应模态的后端已安装、模型权重已下载到磁盘、（首次调用时）模型已加载到内存。将健康检查返回200视为“就绪”是导致集成看似失败的最常见原因。

启动时请勿调用
POST /api/v1/load
。 Lemond会在首次推理请求时自动将模型懒加载到内存，无需手动处理。预加载在不同lemond版本中不可靠（

/load

请求体格式在版本间已变更），格式错误的调用可能在用户操作前导致服务器崩溃或不稳定。加载是唯一可交给lemond自动处理的步骤——拉取模型则必须主动执行。

Pull the model so it exists on disk

拉取模型到磁盘

Lazy-load only loads weights that are already downloaded. If the model was never pulled, the first inference does not error — lemond returns an empty / blank result with HTTP 200. So after health passes and the backend is installed, proactively pull the model:

http

POST /api/v1/pull
{"model": "Whisper-Large-v3-Turbo"}

This is idempotent — a no-op if the weights are already present, a download if they are not. Run it once during setup (after backend install, before the first user-triggered inference) and log the result.

Default model (the one you chose in Step 2): pull it by name as above.
Custom / user-overridden model: do not assume it exists. Confirm it is a real Lemonade model first via
```
GET /api/v1/models
```
(the only trusted catalog — see reference.md), then pull it the same way. A model appearing in the catalog is not proof its weights are downloaded; a successful pull is.

Silent-empty is almost always an unpulled model. If inference returns an empty string / blank output with no HTTP error, the model was not downloaded. Check your pull step before debugging anything else — this is the failure mode that wastes the most time. Log the pull result and the first inference result (see Step 4) so this is diagnosable from the console, not by guesswork.

懒加载仅加载已下载的权重。如果模型从未拉取，首次推理不会报错——lemond会返回空/空白结果并伴随HTTP 200。因此，在健康检查通过且后端安装完成后，需主动拉取模型：

http

POST /api/v1/pull
{"model": "Whisper-Large-v3-Turbo"}

该操作是幂等的——如果权重已存在则无操作，否则进行下载。在设置阶段运行一次（后端安装后，首次用户触发的推理前）并记录结果。

默认模型（步骤2中选择的模型）：按上述方式通过名称拉取。
自定义/用户覆盖的模型：不要假设模型已存在。首先通过
```
GET /api/v1/models
```
确认其为有效的Lemonade模型（这是唯一可信的模型目录——参见reference.md），然后按相同方式拉取。模型出现在目录中不代表其权重已下载；拉取成功才是权重存在的证明。

无提示空白结果几乎总是因为模型未拉取。 如果推理返回空字符串/空白输出且无HTTP错误，说明模型未下载。在调试其他问题前先检查拉取步骤——这是最浪费时间的故障模式。记录拉取结果和首次推理结果（参见步骤4），以便通过控制台排查问题，而非猜测。

Surface the whole setup, not just model load

展示完整设置流程，而非仅模型加载

First-run cold start is more than a model load. The full sequence is:

server spawn  →  health 200  →  backend install  →  model download  →  model load  →  first result

On a fresh machine, backend install and model download can each take from tens of seconds to several minutes (multi-GB weights over the network). Model load alone is 10–30s. An app that shows nothing during this will look frozen.

Minimum: show a loading indicator or status message ("Setting up local AI…") from the moment setup begins until the first response arrives — covering the entire sequence above, not just the final load. The simplest implementation is a flag set when setup/first-request starts and cleared when the first response arrives. Once the model is pulled and loaded once, subsequent runs are fast; the long wait is first-run only.

首次运行冷启动不仅是模型加载。完整流程为：

服务器启动 → 健康检查200 → 后端安装 → 模型下载 → 模型加载 → 首次结果返回

在全新机器上，后端安装和模型下载可能需要数十秒到数分钟（多GB权重通过网络下载）。仅模型加载就需要10–30秒。如果应用在此期间无任何显示，会看似冻结。

最低要求：从设置开始到首次结果返回期间，显示加载指示器或状态消息（“正在设置本地AI…”）——覆盖上述整个流程，而非仅最终的加载步骤。最简单的实现方式是设置一个标志，在设置/首次请求开始时置位，首次结果返回时复位。模型拉取并加载一次后，后续运行会很快；长时间等待仅出现在首次运行时。

Step 7: Lifecycle and recovery

步骤7：生命周期与恢复

These are the only failure modes worth handling. Do not over-engineer.

Symptom	Cause	Recovery
Inference returns empty / blank with HTTP 200, no error	Model never pulled: backend is installed but weights are absent, so lazy-load has nothing to load	`POST /api/v1/pull` with `{"model":"..."}` , wait for success, retry. Log the pulled result and the first inference result. This is the most common silent failure — see Step 6
`POST /api/v1/load` returns 404 / model not found	Model not pulled yet (same root cause as the empty-result row above)	`POST /api/v1/pull` with `{"model": "..."}` then retry `/api/v1/load`
`POST /api/v1/load` returns 500 with backend error	Backend not installed for this hardware	`GET /api/v1/system-info` , pick a supported backend, `POST /api/v1/install` with `{"recipe": "...", "backend": "..."}` , retry
Subprocess exits immediately	Port race: another process grabbed the port between `freePort()` and lemond binding	The reference launcher retries with a fresh port automatically (3 attempts)
`/api/v1/health` never returns 200	First-run backend extraction is slow on cold disk	Extend timeout to 90s on first launch, 30s after
HTTP 401 on every request	Forgot the `Authorization: Bearer` header	Audit the client config because Lemonade rejects unauth'd calls when `LEMONADE_API_KEY` is set

Shutdown: On app exit,

proc.terminate()

(Unix) or

proc.kill()

(Windows).

lemond

flushes config and exits cleanly within a couple of seconds. Always wait on the process; never orphan it.

Do not parse

lemond

stdout to detect readiness; use the HTTP

/api/v1/health

probe. Stdout format is not a stable contract.

只需处理以下故障模式，无需过度设计。

症状	原因	恢复方式
推理返回空/空白结果，HTTP 200，无错误	模型从未拉取：后端已安装但权重缺失，因此懒加载无内容可加载	调用 `POST /api/v1/pull` 并传入 `{"model":"..."}` ，等待成功后重试。记录拉取结果和首次推理结果。这是最常见的无提示故障——参见步骤6
`POST /api/v1/load` 返回404/模型未找到	模型未拉取（与上述空白结果的根本原因相同）	调用 `POST /api/v1/pull` 传入 `{"model": "..."}` ，然后重试 `/api/v1/load`
`POST /api/v1/load` 返回500并伴随后端错误	当前硬件未安装对应后端	调用 `GET /api/v1/system-info` ，选择支持的后端，调用 `POST /api/v1/install` 传入 `{"recipe": "...", "backend": "..."}` ，然后重试
子进程立即退出	端口冲突：在 `freePort()` 和lemond绑定端口之间，其他进程占用了该端口	参考启动器会自动使用新端口重试（最多3次）
`/api/v1/health` 始终不返回200	首次运行时后端在冷磁盘上提取速度慢	首次启动时将超时延长至90秒，后续启动为30秒
所有请求返回HTTP 401	忘记添加 `Authorization: Bearer` 头	检查客户端配置，因为当设置 `LEMONADE_API_KEY` 时，Lemonade会拒绝未授权的调用

关闭流程： 应用退出时，调用

proc.terminate()

（Unix）或

proc.kill()

（Windows）。

lemond

会刷新配置并在几秒内干净退出。始终等待进程结束；切勿让其成为孤儿进程。

请勿通过解析

lemond

的stdout来检测就绪状态；请使用HTTP

/api/v1/health

探测。stdout格式并非稳定的契约。

Verification checklist

验证清单

The integration is done when all of these are true:

vendor/lemonade/

contains the full package:

lemond[.exe]

lemonade[.exe]

LICENSE

, and

resources/

— not just the binary.

```
lemond
```
starts as a subprocess with a fresh API key per launch.
```
GET /api/v1/health
```
returns 200 within the timeout.
The default model is pulled (or bundled) before the first inference; a custom/overridden model is confirmed via
```
GET /api/v1/models
```
and then pulled. A blank result with no error means this step was skipped.
Each lifecycle stage logs a clear line (spawn, health, backend install, model pull, first result) so a failure is diagnosable from the console.
The existing client's chat / image / speech call returns a valid response with the base URL and key swapped, with no other code changed.
First-run latency is surfaced: the interface shows a loading state from the moment the first inference request is sent until the response arrives.
The HTTP client timeout is set to 120 seconds.
In local mode the app requires no cloud API key: no onboarding wall, validator, or startup check blocks the user, and no code path throws "API key not configured" when the active mode is local.
If the app uses a dev-mode file watcher,
```
vendor/lemonade/
```
is excluded from the watched paths so runtime writes by lemond do not trigger restarts.
Killing the parent process leaves no
```
lemond
```
subprocess behind.
On a fresh machine without the optimal backend, the app still works via the Vulkan fallback bundled in
```
bin/
```
.

If any box is unchecked, do not declare the task complete.

当所有以下条件满足时，集成完成：

```
vendor/lemonade/
```
包含完整包：
```
lemond[.exe]
```
、
```
lemonade[.exe]
```
、
```
LICENSE
```
和
```
resources/
```
——而非仅二进制文件。
```
lemond
```
作为子进程启动，每次启动生成新的API密钥。
```
GET /api/v1/health
```
在超时时间内返回200。
默认模型在首次推理前已拉取（或已打包）；自定义/覆盖的模型先通过
```
GET /api/v1/models
```
确认，然后拉取。无错误提示的空白结果表示此步骤被跳过。
每个生命周期阶段都有清晰的日志记录（启动、健康检查、后端安装、模型拉取、首次结果），以便通过控制台排查故障。
现有客户端的聊天/图像/语音调用在仅替换base URL和密钥的情况下返回有效响应，无需修改其他代码。
首次运行延迟已展示：界面从首次推理请求发送到结果返回期间显示加载状态。
HTTP客户端超时已设置为120秒。
本地模式下应用无需云API密钥：无引导页拦截、验证器或启动检查阻止用户，且本地模式下无代码路径抛出“API密钥未配置”错误。
如果应用使用开发模式文件监视器，
```
vendor/lemonade/
```
已被排除在监视路径之外，避免lemond运行时写入触发应用重启。
杀死父进程后无
```
lemond
```
子进程残留。
在无最优后端的全新机器上，应用仍可通过
```
bin/
```
中打包的Vulkan fallback正常运行。

如果有任何一项未勾选，请勿宣布任务完成。

Reference

参考资料

For detailed model catalog, backend selection matrix, full endpoint reference, config keys, and per-model

recipe_options.json

tuning, see reference.md.

如需详细的模型目录、后端选择矩阵、完整端点参考、配置项和各模型的

recipe_options.json

调优，请查看reference.md。

local-ai-app-integration

Original

Translation

Local AI App Integration (Embeddable Lemonade)

本地AI应用集成（Embeddable Lemonade）

When this skill is the right tool

何时适合使用该技能

The opinionated path

标准化实施流程

Step 1: Survey the app

步骤1：调研应用

Step 2: Pick a model + backend profile

步骤2：选择模型+后端配置

Step 3: Place Embeddable Lemonade in the app's tree and install backends

步骤3：将Embeddable Lemonade放置到应用目录树并安装后端

Windows

Windows

Linux

Linux

Sanity check: resources/ must be nested under vendor\lemonade\ (not flattened)

完整性检查：resources/必须嵌套在vendor\lemonade\下（不能扁平化）

Step 4: Add a
`lemond`
launcher

步骤4：添加
`lemond`
启动器

Step 5: Re-point the existing client at
`lemond`

步骤5：将现有客户端指向
`lemond`

Step 6: Health, backend, then pull the model — before first inference

步骤6：健康检查、后端安装、拉取模型——在首次推理前完成

Pull the model so it exists on disk

拉取模型到磁盘

Surface the whole setup, not just model load

展示完整设置流程，而非仅模型加载

Step 7: Lifecycle and recovery

步骤7：生命周期与恢复

Verification checklist

验证清单

Reference

参考资料

local-ai-app-integration

Original

Translation

Local AI App Integration (Embeddable Lemonade)

本地AI应用集成（Embeddable Lemonade）

When this skill is the right tool

何时适合使用该技能

The opinionated path

标准化实施流程

Step 1: Survey the app

步骤1：调研应用

Step 2: Pick a model + backend profile

步骤2：选择模型+后端配置

Step 3: Place Embeddable Lemonade in the app's tree and install backends

步骤3：将Embeddable Lemonade放置到应用目录树并安装后端

Windows

Windows

Linux

Linux

Sanity check: resources/ must be nested under vendor\lemonade\ (not flattened)

完整性检查：resources/必须嵌套在vendor\lemonade\下（不能扁平化）

Step 4: Add a lemond launcher

步骤4：添加lemond启动器

Step 5: Re-point the existing client at lemond

步骤5：将现有客户端指向lemond

Step 6: Health, backend, then pull the model — before first inference

步骤6：健康检查、后端安装、拉取模型——在首次推理前完成

Pull the model so it exists on disk

拉取模型到磁盘

Surface the whole setup, not just model load

展示完整设置流程，而非仅模型加载

Step 7: Lifecycle and recovery

步骤7：生命周期与恢复

Verification checklist

验证清单

Reference

参考资料

Step 4: Add a
`lemond`
launcher

步骤4：添加
`lemond`
启动器

Step 5: Re-point the existing client at
`lemond`

步骤5：将现有客户端指向
`lemond`