getitune-optimizing-a-model

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Optimizing (quantizing) a model with getitune

使用getitune优化（量化）模型

getitune

applies post-training quantization (PTQ) via NNCF to shrink an exported OpenVINO model and speed up inference. Quantization runs on an OpenVINO model (an exported

.xml

), producing an INT8 version.

Run everything from

library/

getitune

通过NNCF应用后训练量化（PTQ）来压缩导出的OpenVINO模型并加速推理。量化操作基于OpenVINO模型（导出的

.xml

文件）进行，生成INT8版本的模型。

所有操作请在

library/

目录下执行。

Workflow

工作流程

python

from getitune.engine import create_engine

python

from getitune.engine import create_engine

Load an exported OpenVINO model, then quantize it

加载导出的OpenVINO模型，然后对其进行量化

ov_engine = create_engine( model="/path/to/exported_model.xml", data="/path/to/dataset", ) ov_engine.optimize() # INT8 post-training quantization via NNCF int8_metrics = ov_engine.test() # validate the quantized model predictions = ov_engine.predict() # run inference with the quantized model


1. **Start from an exported OpenVINO model** (`.xml`). If you only have a
   checkpoint, export it first with the `getitune-exporting-a-model` skill.
   - Done when: `create_engine(model="....xml", data=...)` builds an `OVEngine`.
2. **Provide a calibration dataset.** Calibration images are taken automatically
   from the training subset; 200-500 images is the recommended calibration size.
   - Done when: `optimize()` runs without a "not enough calibration data" issue.
3. **Run `optimize()`.** This replaces the engine's model in place with the INT8
   version.
   - Done when: the call completes and subsequent `test()`/`predict()` use INT8.
4. **Re-validate accuracy** with `test()` and compare against the FP32/FP16
   baseline; a small accuracy drop is expected in exchange for size/latency.
   - Done when: the INT8 metric is within your acceptable tolerance of baseline.

ov_engine = create_engine( model="/path/to/exported_model.xml", data="/path/to/dataset", ) ov_engine.optimize() # 通过NNCF执行INT8后训练量化 int8_metrics = ov_engine.test() # 验证量化后的模型 predictions = ov_engine.predict() # 使用量化模型运行推理


1. **从导出的OpenVINO模型（`.xml`）开始**。如果只有模型 checkpoint，请先使用`getitune-exporting-a-model`技能导出模型。
   - 完成标志：`create_engine(model="....xml", data=...)`成功构建`OVEngine`。
2. **提供校准数据集**。校准图像会自动从训练子集中选取；推荐校准集大小为200-500张图像。
   - 完成标志：`optimize()`运行时未出现“校准数据不足”的问题。
3. **运行`optimize()`**。该操作会将引擎中的模型替换为INT8版本。
   - 完成标志：调用完成后，后续的`test()`/`predict()`使用的是INT8模型。
4. **使用`test()`重新验证精度**，并与FP32/FP16基线模型对比；通常会以微小的精度下降换取模型体积和延迟的优化。
   - 完成标志：INT8模型的指标在可接受的基线误差范围内。

Comparing against the original model

与原始模型对比

After

optimize()

the engine holds the INT8 model. To re-check the original FP32/FP16 model, either pass the original

.xml

path directly to

.test()

.predict()

, or create the engine again from the original

.xml

执行

optimize()

后，引擎中保存的是INT8模型。如需重新检查原始FP32/FP16模型，可直接将原始

.xml

路径传入

.test()

.predict()

，或重新从原始

.xml

创建引擎。

Notes

注意事项

Quantization is OpenVINO/NNCF-based and applies to exported IR models — it is not a training-time step.
Only OpenVINO IR (
.xml
) is supported. An ONNX model must be converted to OpenVINO IR first before it can be optimized.
In the Geti application this is exposed as the
```
quantize
```
job (
```
application/backend/app/execution/quantization/
```
); library
```
optimize()
```
is the same capability without the job/queue wrapper.

量化基于OpenVINO/NNCF实现，针对导出的IR模型进行——这不是训练阶段的操作。
仅支持OpenVINO IR（
.xml
）。ONNX模型必须先转换为OpenVINO IR才能进行优化。
在Geti应用中，该功能以
```
quantize
```
任务的形式提供（路径：
```
application/backend/app/execution/quantization/
```
）；库中的
```
optimize()
```
具备相同功能，只是没有任务/队列封装。

Verify

验证

bash

undefined