ffmpeg-opencv-integration

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

FFmpeg + OpenCV Integration Guide

FFmpeg + OpenCV 集成指南

Use this skill when FFmpeg handles video I/O and OpenCV handles image processing. This SKILL is a lean orchestrator; full pipe patterns, library examples, and Modal recipes are preserved in

references/opencv-pipelines-and-libraries.md

当FFmpeg负责视频I/O、OpenCV负责图像处理时，可使用本技能。本SKILL是一个精简的编排器；完整的管道模式、库示例和Modal 方案保存在

references/opencv-pipelines-and-libraries.md

中。

When to Use

使用场景

Decode with FFmpeg and process frames with OpenCV
Encode OpenCV-generated/processed frames with FFmpeg
Compare
```
cv2.VideoCapture
```
, subprocess pipes, PyAV, ffmpegcv, VidGear, and Decord
Fix RGB/BGR color bugs or
```
(height, width)
```
dimension-order bugs
Preserve audio while replacing processed video frames
Build GPU-assisted video I/O around CPU OpenCV processing

用FFmpeg解码，用OpenCV处理帧
用FFmpeg编码OpenCV生成/处理的帧
对比
```
cv2.VideoCapture
```
、子进程管道、PyAV、ffmpegcv、VidGear和Decord
修复RGB/BGR颜色错误或
```
(height, width)
```
维度顺序错误
替换处理后的视频帧时保留音频
围绕CPU端OpenCV处理构建GPU辅助的视频I/O

Library Selection

库选择

Need	Best option	Why
Simple local file	`cv2.VideoCapture`	Built-in and simple
Full FFmpeg format/protocol support	subprocess pipe	Exact CLI behavior
GPU video I/O	ffmpegcv	NVDEC/NVENC with OpenCV-like API
Network/RTSP streaming	VidGear	Threaded capture and stream helpers
ML batch loading	Decord	Fast random/batch frame access
Frame-level libav control	PyAV	Direct FFmpeg library access

需求	最佳选项	原因
简单本地文件	`cv2.VideoCapture`	内置且易用
完整FFmpeg格式/协议支持	子进程管道	实现与CLI完全一致的行为
GPU视频I/O	ffmpegcv	带有类OpenCV API的NVDEC/NVENC支持
网络/RTSP流处理	VidGear	线程化捕获和流处理辅助工具
机器学习批量加载	Decord	快速随机/批量帧访问
帧级libav控制	PyAV	直接访问FFmpeg库

Critical Gotchas

关键陷阱

OpenCV is BGR. FFmpeg/PyAV/PIL/Decord often produce RGB. Convert explicitly.

NumPy dimensions are
(height, width, channels)
. Pixel access is

img[y, x]

, not

img[x, y]

Video filters can drop audio. Preserve or remux original audio intentionally.
Release resources. Always close
```
VideoCapture
```
, pipes, writers, and PyAV containers.
Rawvideo pipes require exact frame size.
```
width * height * channels
```
must match
```
-pix_fmt
```
.

OpenCV采用BGR格式。FFmpeg/PyAV/PIL/Decord通常生成RGB格式，需显式转换。
NumPy维度为
(height, width, channels)
。像素访问使用
```
img[y, x]
```
，而非
```
img[x, y]
```
。
视频滤镜可能丢失音频。需有意保留或重新复用原始音频。
释放资源。务必关闭
```
VideoCapture
```
、管道、写入器和PyAV容器。
Rawvideo管道要求精确帧大小。
```
width * height * channels
```
必须与
```
-pix_fmt
```
匹配。

Minimal Pipe Patterns

最简管道模式

FFmpeg to OpenCV using BGR frames:

python

cmd = [
    "ffmpeg", "-i", input_path,
    "-f", "rawvideo", "-pix_fmt", "bgr24", "-"
]

OpenCV to FFmpeg using BGR frames:

python

cmd = [
    "ffmpeg", "-y",
    "-f", "rawvideo", "-vcodec", "rawvideo",
    "-s", f"{width}x{height}", "-pix_fmt", "bgr24",
    "-r", str(fps), "-i", "-",
    "-c:v", "libx264", "-preset", "fast", "-crf", "23",
    "-pix_fmt", "yuv420p", output_path
]

使用BGR帧实现FFmpeg到OpenCV的传输：

python

cmd = [
    "ffmpeg", "-i", input_path,
    "-f", "rawvideo", "-pix_fmt", "bgr24", "-"
]

使用BGR帧实现OpenCV到FFmpeg的传输：

python

cmd = [
    "ffmpeg", "-y",
    "-f", "rawvideo", "-vcodec", "rawvideo",
    "-s", f"{width}x{height}", "-pix_fmt", "bgr24",
    "-r", str(fps), "-i", "-",
    "-c:v", "libx264", "-preset", "fast", "-crf", "23",
    "-pix_fmt", "yuv420p", output_path
]

Core Workflow

核心工作流

Probe input dimensions, fps, duration, and audio streams.
Pick the I/O library based on the selection table.
Lock pixel format at boundaries (
```
bgr24
```
for OpenCV pipes;
```
yuv420p
```
for final H.264 output).
Process frames in generators/batches; avoid loading full videos unless they are small.
Preserve or re-encode audio explicitly.
Verify output duration, fps, resolution, codec, and A/V sync.

探测输入的维度、帧率、时长和音频流信息。
根据选择表挑选I/O库。
在边界处锁定像素格式（OpenCV管道使用
```
bgr24
```
；最终H.264输出使用
```
yuv420p
```
）。
以生成器/批量方式处理帧；除非视频很小，否则避免加载完整视频。
显式保留或重新编码音频。
验证输出的时长、帧率、分辨率、编解码器和音视频同步。

Reference Map

参考映射

```
references/opencv-pipelines-and-libraries.md
```
- Full preserved reference: color/dimension gotchas, cleanup patterns, FFmpeg-to-OpenCV and OpenCV-to-FFmpeg pipes, bidirectional pipeline, ffmpegcv, VidGear, Decord, PyAV, Modal.com examples, GPU pipeline, cheat sheets, sources.

```
references/opencv-pipelines-and-libraries.md
```
- 完整保留的参考文档：颜色/维度陷阱、清理模式、FFmpeg到OpenCV和OpenCV到FFmpeg的管道、双向流水线、ffmpegcv、VidGear、Decord、PyAV、Modal.com示例、GPU流水线、速查表、来源。

ffmpeg-opencv-integration

Original

Translation

FFmpeg + OpenCV Integration Guide

FFmpeg + OpenCV 集成指南

When to Use

使用场景

Library Selection

库选择

Critical Gotchas

关键陷阱

Minimal Pipe Patterns

最简管道模式

Core Workflow

核心工作流

Reference Map

参考映射

Related Skills

相关技能