io-uring

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

io_uring

io_uring

Purpose

目的

Guide agents through Linux io_uring: the submission/completion queue model (SQE/CQE), liburing API, multi-shot accept/recv, provided buffer rings, fixed files and registered buffers, zero-copy send, Rust integration with tokio-uring, performance comparison with epoll, and security considerations.
指导 Agent 深入理解 Linux io_uring:submission/completion queue 模型(SQE/CQE)、liburing API、multi-shot accept/recv、provided buffer ring、fixed files 和 registered buffers、zero-copy send、通过 tokio-uring 实现 Rust 集成、与 epoll 的性能对比,以及安全注意事项。

When to Use

何时使用

  • Building a high-throughput network or disk server on Linux 5.1+
  • Replacing epoll + thread pool with fewer syscalls
  • Implementing multi-shot accept/recv for connection-heavy services
  • Using zero-copy networking with
    IORING_OP_SEND_ZC
  • Integrating async I/O in Rust via
    tokio-uring
  • Evaluating io_uring vs epoll for your workload
  • 在 Linux 5.1+ 上构建高吞吐量网络或磁盘服务器
  • 使用更少的系统调用替代 epoll + 线程池
  • 为连接密集型服务实现 multi-shot accept/recv
  • 使用
    IORING_OP_SEND_ZC
    实现 zero-copy 网络传输
  • 通过
    tokio-uring
    在 Rust 中集成异步 I/O
  • 针对你的工作负载评估 io_uring 与 epoll

Workflow

工作流程

1. SQ/CQ model

1. SQ/CQ 模型

Application                    Kernel
    │                            │
    ├── mmap SQ ring ───────────►│ submission queue (SQE)
    ├── mmap CQ ring ◄──────────│ completion queue (CQE)
    ├── io_uring_submit() ──────►│ processes SQEs
    └── io_uring_wait_cqe() ◄──│ posts CQEs
Each SQE describes one operation; each CQE reports result and user_data cookie.
Application                    Kernel
    │                            │
    ├── mmap SQ ring ───────────►│ submission queue (SQE)
    ├── mmap CQ ring ◄──────────│ completion queue (CQE)
    ├── io_uring_submit() ──────►│ processes SQEs
    └── io_uring_wait_cqe() ◄──│ posts CQEs
每个 SQE 描述一个操作;每个 CQE 报告操作结果和 user_data cookie。

2. Minimal liburing example

2. 最简 liburing 示例

c
#include <liburing.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>

int main(void) {
    struct io_uring ring;
    io_uring_queue_init(32, &ring, 0);

    int fd = open("test.txt", O_RDONLY);
    char buf[4096];

    struct io_uring_sqe *sqe = io_uring_get_sqe(&ring);
    io_uring_prep_read(sqe, fd, buf, sizeof(buf), 0);
    io_uring_sqe_set_data(sqe, (void *)1);

    io_uring_submit(&ring);

    struct io_uring_cqe *cqe;
    io_uring_wait_cqe(&ring, &cqe);
    if (cqe->res >= 0)
        printf("read %d bytes\n", cqe->res);
    else
        perror("read");

    io_uring_cqe_seen(&ring, cqe);
    io_uring_queue_exit(&ring);
    close(fd);
    return 0;
}
bash
gcc -o uring_read uring_read.c -luring
./uring_read
c
#include <liburing.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>

int main(void) {
    struct io_uring ring;
    io_uring_queue_init(32, &ring, 0);

    int fd = open("test.txt", O_RDONLY);
    char buf[4096];

    struct io_uring_sqe *sqe = io_uring_get_sqe(&ring);
    io_uring_prep_read(sqe, fd, buf, sizeof(buf), 0);
    io_uring_sqe_set_data(sqe, (void *)1);

    io_uring_submit(&ring);

    struct io_uring_cqe *cqe;
    io_uring_wait_cqe(&ring, &cqe);
    if (cqe->res >= 0)
        printf("read %d bytes\n", cqe->res);
    else
        perror("read");

    io_uring_cqe_seen(&ring, cqe);
    io_uring_queue_exit(&ring);
    close(fd);
    return 0;
}
bash
gcc -o uring_read uring_read.c -luring
./uring_read

3. Common prep operations

3. 常用 prep 操作

FunctionOperation
io_uring_prep_read
File read
io_uring_prep_write
File write
io_uring_prep_recv
Socket recv
io_uring_prep_send
Socket send
io_uring_prep_accept
Accept connection
io_uring_prep_connect
Outbound connect
io_uring_prep_poll_add
Poll fd
io_uring_prep_timeout
Timeout/link timeout
c
// Batch multiple SQEs before single submit
for (int i = 0; i < n; i++) {
    sqe = io_uring_get_sqe(&ring);
    io_uring_prep_read(sqe, fds[i], bufs[i], sizes[i], 0);
    io_uring_sqe_set_data(sqe, (void *)(intptr_t)i);
}
io_uring_submit(&ring);
函数操作
io_uring_prep_read
文件读取
io_uring_prep_write
文件写入
io_uring_prep_recv
Socket 接收
io_uring_prep_send
Socket 发送
io_uring_prep_accept
接受连接
io_uring_prep_connect
发起连接
io_uring_prep_poll_add
轮询 fd
io_uring_prep_timeout
超时或链接超时
c
// Batch multiple SQEs before single submit
for (int i = 0; i < n; i++) {
    sqe = io_uring_get_sqe(&ring);
    io_uring_prep_read(sqe, fds[i], bufs[i], sizes[i], 0);
    io_uring_sqe_set_data(sqe, (void *)(intptr_t)i);
}
io_uring_submit(&ring);

4. Multi-shot operations

4. Multi-shot 操作

c
// Multi-shot accept — one SQE handles many connections
sqe = io_uring_get_sqe(&ring);
io_uring_prep_multishot_accept(sqe, listen_fd, NULL, NULL, 0);
io_uring_sqe_set_data(sqe, (void *)ACCEPT_COOKIE);

// Loop on CQEs until CQE_FLAG_MORE is clear
while (1) {
    io_uring_wait_cqe(&ring, &cqe);
    if (cqe->user_data == ACCEPT_COOKIE) {
        int client_fd = cqe->res;
        if (client_fd >= 0)
            handle_client(client_fd);
        if (!(cqe->flags & IORING_CQE_F_MORE))
            break;  // re-arm accept SQE
    }
    io_uring_cqe_seen(&ring, cqe);
}
c
// Multi-shot accept — one SQE handles many connections
sqe = io_uring_get_sqe(&ring);
io_uring_prep_multishot_accept(sqe, listen_fd, NULL, NULL, 0);
io_uring_sqe_set_data(sqe, (void *)ACCEPT_COOKIE);

// Loop on CQEs until CQE_FLAG_MORE is clear
while (1) {
    io_uring_wait_cqe(&ring, &cqe);
    if (cqe->user_data == ACCEPT_COOKIE) {
        int client_fd = cqe->res;
        if (client_fd >= 0)
            handle_client(client_fd);
        if (!(cqe->flags & IORING_CQE_F_MORE))
            break;  // re-arm accept SQE
    }
    io_uring_cqe_seen(&ring, cqe);
}

5. Provided buffer rings

5. Provided buffer ring

c
#include <liburing.h>

#define BUF_GROUP 0
#define BUF_SIZE  4096
#define BUF_COUNT 64

struct io_uring_buf_ring *buf_ring;
char bufs[BUF_COUNT][BUF_SIZE];

// Setup buffer ring
buf_ring = io_uring_setup_buf_ring(&ring, BUF_COUNT, BUF_GROUP, 0, &ret);
for (int i = 0; i < BUF_COUNT; i++)
    io_uring_buf_ring_add(buf_ring, bufs[i], BUF_SIZE, i, BUF_COUNT - 1 - i, 0);
io_uring_buf_ring_advance(buf_ring, BUF_COUNT);

// Recv with provided buffers
sqe = io_uring_get_sqe(&ring);
io_uring_prep_recv_multishot(sqe, sock_fd, NULL, 0, 0);
sqe->buf_group = BUF_GROUP;
Kernel selects buffer from ring — eliminates per-recv buffer allocation.
c
#include <liburing.h>

#define BUF_GROUP 0
#define BUF_SIZE  4096
#define BUF_COUNT 64

struct io_uring_buf_ring *buf_ring;
char bufs[BUF_COUNT][BUF_SIZE];

// Setup buffer ring
buf_ring = io_uring_setup_buf_ring(&ring, BUF_COUNT, BUF_GROUP, 0, &ret);
for (int i = 0; i < BUF_COUNT; i++)
    io_uring_buf_ring_add(buf_ring, bufs[i], BUF_SIZE, i, BUF_COUNT - 1 - i, 0);
io_uring_buf_ring_advance(buf_ring, BUF_COUNT);

// Recv with provided buffers
sqe = io_uring_get_sqe(&ring);
io_uring_prep_recv_multishot(sqe, sock_fd, NULL, 0, 0);
sqe->buf_group = BUF_GROUP;
内核会从 ring 中选择 buffer,从而免除每次 recv 的 buffer 分配。

6. Fixed files and registered buffers

6. Fixed files 与 registered buffers

c
// Register files — avoids per-op fd table lookup
int fds[64];
io_uring_register_files(&ring, fds, 64);
io_uring_prep_read(sqe, 0, buf, size, offset);  // index 0, not fd

// Registered buffers — pinned, DMA-friendly
struct iovec iov = { .iov_base = buf, .iov_len = size };
io_uring_register_buffers(&ring, &iov, 1);
io_uring_prep_read_fixed(sqe, fd, buf, size, offset, 0);
c
// Register files — avoids per-op fd table lookup
int fds[64];
io_uring_register_files(&ring, fds, 64);
io_uring_prep_read(sqe, 0, buf, size, offset);  // index 0, not fd

// Registered buffers — pinned, DMA-friendly
struct iovec iov = { .iov_base = buf, .iov_len = size };
io_uring_register_buffers(&ring, &iov, 1);
io_uring_prep_read_fixed(sqe, fd, buf, size, offset, 0);

7. Zero-copy send

7. Zero-copy send

c
sqe = io_uring_get_sqe(&ring);
io_uring_prep_send_zc(sqe, sock_fd, data, len, 0, 0);
// May complete with IORING_CQE_F_MORE — wait for notification CQE
Requires kernel 6.0+, NIC/driver support. Falls back to copy mode if unavailable.
c
sqe = io_uring_get_sqe(&ring);
io_uring_prep_send_zc(sqe, sock_fd, data, len, 0, 0);
// May complete with IORING_CQE_F_MORE — wait for notification CQE
需要内核 6.0+,以及 NIC/驱动支持。如果不可用,则回退到 copy 模式。

8. tokio-uring (Rust)

8. tokio-uring(Rust)

rust
use tokio_uring::fs::File;

#[tokio_uring::main]
async fn main() -> std::io::Result<()> {
    let file = File::open("test.txt").await?;
    let buf = vec![0u8; 4096];
    let (res, buf) = file.read_at(buf, 0).await;
    println!("read {} bytes", res?);
    Ok(())
}
bash
cargo add tokio-uring
rust
use tokio_uring::fs::File;

#[tokio_uring::main]
async fn main() -> std::io::Result<()> {
    let file = File::open("test.txt").await?;
    let buf = vec![0u8; 4096];
    let (res, buf) = file.read_at(buf, 0).await;
    println!("read {} bytes", res?);
    Ok(())
}
bash
cargo add tokio-uring

9. io_uring vs epoll

9. io_uring 与 epoll 对比

Aspectepollio_uring
Syscallsepoll_wait + read/write per opBatch SQEs, single submit
Buffer mgmtApp allocatesProvided buffers
Zero-copysendfile, manualSEND_ZC built-in
ComplexityLowerHigher
Kernel version2.6+5.1+ (features vary)
Choose io_uring when
├── Syscall overhead is measurable bottleneck
├── High connection count with multi-shot
└── Kernel 6.x+ with needed op support

Stay on epoll when
├── Simple app, portability matters
└── Team familiarity outweighs gains
方面epollio_uring
系统调用epoll_wait + 每次操作的 read/write批量 SQE,单次 submit
Buffer 管理应用分配Provided buffer
Zero-copysendfile,手动实现内置 SEND_ZC
复杂度较低较高
内核版本2.6+5.1+(具体功能因版本而异)
选择 io_uring,当
├── 系统调用开销是可测量的瓶颈
├── 使用 multi-shot 且连接数较高
└── 内核为 6.x+,并支持所需操作

当满足以下情况时继续使用 epoll
├── 应用简单,且可移植性很重要
└── 团队熟悉度的价值高于性能收益

10. Security considerations

10. 安全注意事项

bash
undefined
bash
undefined

Restrict unprivileged io_uring (some distros disable by default)

Restrict unprivileged io_uring (some distros disable by default)

sysctl kernel.io_uring_disabled # 0=enabled, 1=disabled, 2=allow registered sysctl kernel.io_uring_group # gid allowed when disabled=2

io_uring has been an attack surface — keep kernels patched; sandbox untrusted code.
sysctl kernel.io_uring_disabled # 0=enabled, 1=disabled, 2=allow registered sysctl kernel.io_uring_group # gid allowed when disabled=2

io_uring 一直是一个攻击面——请及时修补内核,并对不受信任的代码使用沙箱隔离。

Common Problems

常见问题

SymptomCauseFix
-EINVAL
on prep
Op not supported on kernelCheck
io_uring_get_probe()
Hang on wait_cqeMissing submitCall
io_uring_submit
after SQEs
Buffer not returnedProvided buffer not re-added
buf_ring_add
after consuming
SEND_ZC incompleteNeed second CQEWait for
IORING_CQE_F_MORE
clear
Slower than epollSmall batch sizeBatch more SQEs; use SQPOLL
Permission deniedio_uring disabledsysctl or run with capability
症状原因修复方法
prep 时出现
-EINVAL
内核不支持该操作检查
io_uring_get_probe()
wait_cqe 挂起缺少 submit在 SQE 后调用
io_uring_submit
Buffer 未归还Provided buffer 未重新添加使用后调用
buf_ring_add
SEND_ZC 未完成需要第二个 CQE等待
IORING_CQE_F_MORE
被清除
比 epoll 更慢批量大小过小批处理更多 SQE;使用 SQPOLL
Permission deniedio_uring 已被禁用通过 sysctl 启用或使用相应 capability 运行

Related Skills

相关 Skill

  • skills/async-io/dpdk
    — kernel bypass alternative for networking
  • skills/async-io/af-xdp
    — XDP socket path for packet I/O
  • skills/observability/ebpf
    — trace io_uring events
  • skills/rust/rust-async-internals
    — async model behind tokio-uring
  • skills/profilers/strace-ltrace
    — syscall comparison vs epoll
  • skills/kernel/kernel-internals
    — kernel I/O subsystem context
  • skills/async-io/dpdk
    — 网络场景下的内核旁路替代方案
  • skills/async-io/af-xdp
    — 用于数据包 I/O 的 XDP socket 路径
  • skills/observability/ebpf
    — 跟踪 io_uring 事件
  • skills/rust/rust-async-internals
    — tokio-uring 背后的异步模型
  • skills/profilers/strace-ltrace
    — 与 epoll 进行系统调用对比
  • skills/kernel/kernel-internals
    — 内核 I/O 子系统背景