Back to Details

xiao-edgeimpulse-train

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Edge Impulse training pipeline for XIAO ESP32S3 (REST API, verified)

适用于XIAO ESP32S3的Edge Impulse训练流程（基于REST API，已验证）

Everything below was executed and verified end-to-end (2026-07). The edge-impulse-cli is NOT needed — do not try to install it (its serialport@8 dependency needs VS C++ build tools on Node 22+).
以下所有操作均已完成端到端执行与验证（2026年7月）。无需使用edge-impulse-cli——请勿尝试安装它（在Node 22+环境下，其serialport@8依赖项需要VS C++构建工具）。

Step 0 — API key (ask the user)

步骤0 — 获取API密钥（向用户索要）

Ask the user for an Edge Impulse admin API key (
ei_...
), or read it from the project's 
.env
(
EI_API_KEY=...
) if the user points you there.
  • The auto-created project key has role "Ingestion & deployment" — it can upload data but cannot create impulses or start training (symptom: 
    insufficient permissions (valid roles: [admin])
    ).
  • If needed, tell the user: Studio → Dashboard → Keys → Add new API key → Role Admin (full access) → copy the full key immediately (it is truncated in the list afterwards).
  • Get the project id: 
    GET https://studio.edgeimpulse.com/v1/api/projects
    with header 
    x-api-key
    → 
    projects[0].id
    .
  • PowerShell warning: 
    $pid
    is a READ-ONLY reserved variable — use 
    $proj
    .
向用户索要Edge Impulse的管理员API密钥（格式为
ei_...
），如果用户指定了项目的
.env
文件，则从中读取
EI_API_KEY=...
的值。
  • 自动创建的项目密钥角色为“Ingestion & deployment”（数据导入与部署）——它可以上传数据，但无法创建脉冲或启动训练（报错症状：
    insufficient permissions (valid roles: [admin])
    ）。
  • 若需要指导用户创建密钥：进入Studio → Dashboard（仪表盘）→ Keys（密钥）→ Add new API key（添加新API密钥）→ 选择角色Admin (full access)（管理员，完全权限）→ 立即复制完整密钥（后续在列表中会被截断）。
  • 获取项目ID：发送
    GET https://studio.edgeimpulse.com/v1/api/projects
    请求，请求头携带
    x-api-key
    ，从返回结果的
    projects[0].id
    中获取。
  • PowerShell注意事项：
    $pid
    是只读保留变量，请使用
    $proj
    代替。

Step 1 — upload dataset (ingestion API)

步骤1 — 上传数据集（导入API）

One POST per file; label comes from the 
x-label
header:
powershell
curl.exe -s -X POST -H "x-api-key: $key" -H "x-label: $cls" `
  -F "data=@$($f.FullName)" https://ingestion.edgeimpulse.com/api/training/files
WAV: 16 kHz 16-bit mono. Images: jpg/png as-is. After uploading everything, auto-split train/test: 
POST /v1/api/$proj/rebalance
.
One project = one model. To retrain for a different task first wipe: 
POST /v1/api/$proj/raw-data/delete-all
(archive any deployed lib zip first).
每个文件对应一次POST请求；标签来自请求头
x-label
：
powershell
curl.exe -s -X POST -H "x-api-key: $key" -H "x-label: $cls" `
  -F "data=@$($f.FullName)" https://ingestion.edgeimpulse.com/api/training/files
WAV音频要求：16 kHz采样率、16位单声道。图片支持jpg/png格式，直接上传即可。全部文件上传完成后，自动划分训练/测试集：发送
POST /v1/api/$proj/rebalance
请求。
一个项目对应一个模型。若要针对不同任务重新训练，需先清空现有数据：发送
POST /v1/api/$proj/raw-data/delete-all
请求（请先备份已部署的库压缩包）。

Step 2 — create the impulse

步骤2 — 创建脉冲

POST https://studio.edgeimpulse.com/v1/api/$proj/impulse
(POST, not PUT).
Audio (keyword spotting) — 
implementationVersion: 4
on the MFCC block is REQUIRED; without it you get v1 and feature generation fails with "mel filterbank contains all zeros":
json
{"inputBlocks":[{"id":1,"type":"time-series","name":"Time series data","title":"Time series data","windowSizeMs":1000,"windowIncreaseMs":500,"frequencyHz":16000,"padZeros":true}],
 "dspBlocks":[{"id":2,"type":"mfcc","name":"MFCC","axes":["audio"],"title":"MFCC","implementationVersion":4}],
 "learnBlocks":[{"id":3,"type":"keras","name":"Classifier","dsp":[2],"title":"Classification"}]}
Vision (image classification, MobileNet transfer learning):
json
{"inputBlocks":[{"id":1,"type":"image","name":"Images","title":"Image data","imageWidth":96,"imageHeight":96,"resizeMode":"squash"}],
 "dspBlocks":[{"id":2,"type":"image","name":"Image","axes":["image"],"title":"Image","implementationVersion":1}],
 "learnBlocks":[{"id":3,"type":"keras-transfer-image","name":"Transfer learning","dsp":[2],"title":"Transfer learning (Images)"}]}
发送
POST https://studio.edgeimpulse.com/v1/api/$proj/impulse
请求（使用POST，而非PUT）。
音频（关键词识别）——MFCC模块必须设置
implementationVersion: 4
；若不设置则会使用v1版本，导致特征生成失败，报错为"mel filterbank contains all zeros"：
json
{"inputBlocks":[{"id":1,"type":"time-series","name":"Time series data","title":"Time series data","windowSizeMs":1000,"windowIncreaseMs":500,"frequencyHz":16000,"padZeros":true}],
 "dspBlocks":[{"id":2,"type":"mfcc","name":"MFCC","axes":["audio"],"title":"MFCC","implementationVersion":4}],
 "learnBlocks":[{"id":3,"type":"keras","name":"Classifier","dsp":[2],"title":"Classification"}]}
视觉（图像分类，基于MobileNet迁移学习）：
json
{"inputBlocks":[{"id":1,"type":"image","name":"Images","title":"Image data","imageWidth":96,"imageHeight":96,"resizeMode":"squash"}],
 "dspBlocks":[{"id":2,"type":"image","name":"Image","axes":["image"],"title":"Image","implementationVersion":1}],
 "learnBlocks":[{"id":3,"type":"keras-transfer-image","name":"Transfer learning","dsp":[2],"title":"Transfer learning (Images)"}]}

Step 3 — generate features, then train

步骤3 — 生成特征，然后训练

powershell
undefined
powershell
undefined

start: returns a job id

启动任务：返回作业ID

POST /v1/api/$proj/jobs/generate-features body: {"dspId":2,"calculateFeatureImportance":false}
POST /v1/api/$proj/jobs/generate-features 请求体: {"dspId":2,"calculateFeatureImportance":false}

poll until finished (10-15 s interval):

轮询直到任务完成（间隔10-15秒）:

GET /v1/api/$proj/jobs/$jobId/status -> job.finished / job.finishedSuccessful
GET /v1/api/$proj/jobs/$jobId/status -> 查看job.finished / job.finishedSuccessful状态

on failure read newest-first logs:

任务失败时，按从新到旧顺序读取日志:

GET /v1/api/$proj/jobs/$jobId/stdout -> stdout[0..] .data
GET /v1/api/$proj/jobs/$jobId/stdout -> 读取stdout[0..] .data字段

train (after features succeed):

训练（特征生成成功后执行）:

POST /v1/api/$proj/jobs/train/keras/3
POST /v1/api/$proj/jobs/train/keras/3

audio body: {"trainingCycles":100,"learningRate":0.005}

音频任务请求体: {"trainingCycles":100,"learningRate":0.005}

vision body: {"trainingCycles":20,"learningRate":0.0005}

视觉任务请求体: {"trainingCycles":20,"learningRate":0.0005}


Accuracy is in the train job stdout (`val_accuracy` lines). Keep each poll
loop's total wait under your tool timeout — resume polling in a new call
rather than one giant sleep.

模型准确率可在训练作业的标准输出中查看（含`val_accuracy`的行）。确保每次轮询循环的总等待时间不超过工具超时时间——若未完成，可发起新请求继续轮询，而非一次性长时间等待。

Step 4 — build + download the Arduino library

步骤4 — 构建并下载Arduino库

powershell
POST /v1/api/$proj/jobs/build-ondevice-model?type=arduino   body: {"engine":"tflite-eon"}
powershell
POST /v1/api/$proj/jobs/build-ondevice-model?type=arduino   请求体: {"engine":"tflite-eon"}

poll job, then:

轮询作业状态，完成后:

GET /v1/api/$proj/deployment/download?type=arduino -> save as <name>.zip

Install: find the sketchbook with `arduino-cli config get directories.user`,
extract into `<sketchbook>\libraries\<project>_inferencing`. The zip usually
nests one folder — flatten so `src\<project>_inferencing.h` exists.
Archive the zip per task (audio/vision) — swapping tasks overwrites the lib.
GET /v1/api/$proj/deployment/download?type=arduino -> 保存为<name>.zip

安装方法：使用`arduino-cli config get directories.user`命令找到Arduino草图目录，将压缩包解压到`<sketchbook>\libraries\<project>_inferencing`路径下。压缩包通常会嵌套一层文件夹，请将其展平，确保`src\<project>_inferencing.h`文件存在。请按任务类型（音频/视觉）分别备份压缩包——切换任务时会覆盖现有库文件。

Step 5 — on-device fixes (ESP32-S3, REQUIRED for vision)

步骤5 — 设备端修复（ESP32-S3，视觉任务必须执行）

  1. --clean
    on the first compile after swapping the library.     The arduino-cli cache keeps stale objects for the same library name/version — symptoms: 
    objs.a ... is not an object
    link errors, or edits that appear to change nothing (identical byte sizes).
  2. Tensor arena overflow crash (
    Failed to allocate persistent buffer ... EI_MAX_OVERFLOW_BUFFER_COUNT
    + Guru Meditation on boot): edit 
    src\edge-impulse-sdk\porting\ei_classifier_porting.h
    — near the end, 
    #if defined(CONFIG_IDF_TARGET_ESP32S3)
    hard-defines 
    EI_MAX_OVERFLOW_BUFFER_COUNT 30
    with no #ifndef guard, so -D flags and model-file defaults are ALL ignored. Change 
    30
    to 
    2048
    , then 
    --clean
    .
  3. Put the arena in PSRAM — add to the sketch (the SDK's allocators are weak symbols) and build with 
    --board-options PSRAM=opi
    :
cpp
#include "esp_heap_caps.h"
void *ei_malloc(size_t size) {
  void *p = heap_caps_aligned_alloc(16, size, MALLOC_CAP_SPIRAM);
  if (!p) p = heap_caps_aligned_alloc(16, size, MALLOC_CAP_DEFAULT);
  return p;
}
void *ei_calloc(size_t n, size_t s) { void *p = ei_malloc(n*s); if (p) memset(p,0,n*s); return p; }
void ei_free(void *ptr) { heap_caps_free(ptr); }
  1. 更换库后首次编译需添加
    --clean
    参数    。arduino-cli的缓存会保留同名同版本库的陈旧对象——报错症状：
    objs.a ... is not an object
    链接错误，或修改代码后编译结果无变化（字节大小完全相同）。
  2. Tensor arena溢出崩溃（报错信息：
    Failed to allocate persistent buffer ... EI_MAX_OVERFLOW_BUFFER_COUNT
    + 启动时出现Guru Meditation错误）：编辑
    src\edge-impulse-sdk\porting\ei_classifier_porting.h
    文件——在文件末尾，
    #if defined(CONFIG_IDF_TARGET_ESP32S3)
    代码块硬定义了
    EI_MAX_OVERFLOW_BUFFER_COUNT 30
    ，且没有
    #ifndef
    保护，导致编译时的-D参数和模型文件默认值全部被忽略。将
    30
    改为
    2048
    ，然后执行
    --clean
    编译。
  3. 将arena放入PSRAM——将以下代码添加到草图中（SDK的分配器为弱符号），并使用
    --board-options PSRAM=opi
    参数编译：
cpp
#include "esp_heap_caps.h"
void *ei_malloc(size_t size) {
  void *p = heap_caps_aligned_alloc(16, size, MALLOC_CAP_SPIRAM);
  if (!p) p = heap_caps_aligned_alloc(16, size, MALLOC_CAP_DEFAULT);
  return p;
}
void *ei_calloc(size_t n, size_t s) { void *p = ei_malloc(n*s); if (p) memset(p,0,n*s); return p; }
void ei_free(void *ptr) { heap_caps_free(ptr); }

Step 6 — inference sketch integration

步骤6 — 推理草图集成

  • Feed data via the static-buffer pattern (
    signal.get_data
    ), NOT the EI example's 
    <I2S.h>
    code (breaks on esp32 core 3.x — use 
    ESP_I2S.h
    ).
  • Vision feature packing: one float per pixel = 
    (r<<16)|(g<<8)|b
    .
  • Camera on XIAO Sense renders rotated 90°: rotate CW while downscaling (
    sx = y*W/H; sy = H-1-(x*H/W)
    ) or accuracy collapses. Verify orientation empirically by classifying one frame 3 ways (none/cw/ccw) and comparing.
  • Audio level gap: mic audio is far quieter than full-scale training WAVs — best fix is collecting training data through the device mic itself.
  • Camera AWB needs warmup: grab and discard ~8 frames before the real capture.
  • 使用静态缓冲区模式（
    signal.get_data
    ）输入数据，而非EI示例中的
    <I2S.h>
    代码（在esp32核心3.x版本中会失效——请使用
    ESP_I2S.h
    ）。
  • 视觉特征打包：每个像素对应一个浮点数，计算公式为
    (r<<16)|(g<<8)|b
    。
  • XIAO Sense的摄像头输出画面旋转了90°：在缩小画面时顺时针旋转（公式：
    sx = y*W/H; sy = H-1-(x*H/W)
    ），否则模型准确率会大幅下降。可通过对同一帧画面分别进行无旋转、顺时针旋转、逆时针旋转三种分类，对比结果来验证旋转方向是否正确。
  • 音频电平差异：麦克风采集的音频远低于训练用WAV文件的满量程——最佳解决方案是直接通过设备麦克风采集训练数据。
  • 摄像头自动白平衡（AWB）需要预热：在实际采集前，先抓取并丢弃约8帧画面。