Back to Details

core-dumps

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Core Dumps

Core Dumps（核心转储）

Purpose

用途

Guide agents through enabling, collecting, and analysing core dumps for post-mortem crash investigation without rerunning the buggy program.
指导相关人员完成Core Dump的启用、收集和分析，无需重新运行出错程序即可进行事后崩溃调查。

Triggers

触发场景

  • "My program crashed in production — how do I analyse the core?"
  • "How do I enable core dumps on Linux?"
  • "I have a core file but no symbols / source"
  • "How do I use debuginfod to get symbols for a core?"
  • "coredumpctl show me the crash"
  • “我的程序在生产环境崩溃了——我该如何分析core文件？”
  • “如何在Linux上启用Core Dump？”
  • “我有core文件但没有符号/源码”
  • “如何使用debuginfod获取core文件的符号？”
  • “coredumpctl帮我查看崩溃信息”

Workflow

操作流程

1. Enable core dumps (Linux)

1. 在Linux上启用Core Dump

bash
undefined
bash
undefined

Per-session (lost on logout)

会话级生效（登出后失效）

ulimit -c unlimited
ulimit -c unlimited

Persistent (add to /etc/security/limits.conf)

永久生效（添加到/etc/security/limits.conf）

  • soft core unlimited
  • hard core unlimited
  • soft core unlimited
  • hard core unlimited

Check current limit

检查当前限制

ulimit -c
ulimit -c

Set core pattern (where and how cores are named)

设置core文件命名规则（存储位置和命名方式）

Default: 'core' in CWD — often not useful

默认：当前工作目录下的'core'文件——通常实用性不强

sudo sysctl -w kernel.core_pattern=/tmp/core-%e-%p-%t
sudo sysctl -w kernel.core_pattern=/tmp/core-%e-%p-%t

%e = executable, %p = PID, %t = timestamp

%e = 可执行文件名, %p = 进程ID, %t = 时间戳

Persistent (add to /etc/sysctl.d/99-core.conf)

永久生效（添加到/etc/sysctl.d/99-core.conf）

kernel.core_pattern=/tmp/core-%e-%p-%t kernel.core_uses_pid=1
undefined
kernel.core_pattern=/tmp/core-%e-%p-%t kernel.core_uses_pid=1
undefined

2. systemd/coredumpctl (modern Linux)

2. systemd/coredumpctl（现代Linux系统）

If systemd manages core dumps (common on Ubuntu 20+, Fedora, Arch):
bash
undefined
如果系统由systemd管理Core Dump（Ubuntu 20+、Fedora、Arch等系统常见）：
bash
undefined

List recent crashes

列出近期崩溃记录

coredumpctl list
coredumpctl list

Show details of the latest crash

显示最新崩溃的详细信息

coredumpctl info
coredumpctl info

Load latest crash in GDB

在GDB中加载最新崩溃的core文件

coredumpctl gdb
coredumpctl gdb

Load specific PID crash

加载指定进程ID的崩溃core文件

coredumpctl gdb 12345
coredumpctl gdb 12345

Export core file

导出core文件

coredumpctl dump -o myapp.core PID

Core storage location: `/var/lib/systemd/coredump/`.
coredumpctl dump -o myapp.core PID

Core文件存储位置：`/var/lib/systemd/coredump/`。

3. Enable core dumps (macOS)

3. 在macOS上启用Core Dump

bash
undefined
bash
undefined

macOS uses /cores by default (must be root-writable)

macOS默认存储在/cores目录（需具备root写入权限）

ulimit -c unlimited
ulimit -c unlimited

Check

检查core文件

ls /cores/
ls /cores/

launchd-launched services: set in plist

launchd启动的服务：需在plist文件中配置

<key>HardResourceLimits</key>

<key>HardResourceLimits</key>

<dict><key>Core</key><integer>9223372036854775807</integer></dict>

<dict><key>Core</key><integer>9223372036854775807</integer></dict>

undefined
undefined

4. Analyse a core with GDB

4. 使用GDB分析core文件

bash
undefined
bash
undefined

Load binary and core

加载二进制文件和core文件

gdb ./prog core.12345
gdb ./prog core.12345

If the binary was stripped, provide the unstripped copy

如果二进制文件已被剥离符号，提供未剥离符号的副本

gdb ./prog-with-symbols core.12345
gdb ./prog-with-symbols core.12345

Essential first commands

核心初始命令

(gdb) bt # call stack (gdb) bt full # stack + locals (gdb) info registers # CPU state at crash (gdb) frame 2 # jump to interesting frame (gdb) info locals # local variables in frame (gdb) print ptr # inspect a pointer
(gdb) bt # 调用栈 (gdb) bt full # 调用栈+局部变量 (gdb) info registers # 崩溃时的CPU状态 (gdb) frame 2 # 跳转到指定栈帧 (gdb) info locals # 当前栈帧的局部变量 (gdb) print ptr # 查看指针内容

All threads (multi-threaded crash)

所有线程信息（多线程崩溃场景）

(gdb) thread apply all bt full
undefined
(gdb) thread apply all bt full
undefined

5. Analyse a core with LLDB

5. 使用LLDB分析core文件

bash
lldb ./prog -c core.12345
bash
lldb ./prog -c core.12345

Or

或者

lldb (lldb) target create ./prog --core core.12345
lldb (lldb) target create ./prog --core core.12345

Commands

常用命令

(lldb) bt (lldb) thread backtrace all (lldb) frame select 2 (lldb) frame variable
undefined
(lldb) bt (lldb) thread backtrace all (lldb) frame select 2 (lldb) frame variable
undefined

6. Missing symbols: debuginfod

6. 缺失符号：使用debuginfod

debuginfod
serves debug symbols from a central server, mapping build IDs to DWARF data.
bash
undefined
debuginfod
从中央服务器提供调试符号，将构建ID映射到DWARF数据。
bash
undefined

Install client (Debian/Ubuntu)

安装客户端（Debian/Ubuntu）

sudo apt install debuginfod
sudo apt install debuginfod

Enable (add to ~/.bashrc or /etc/environment)

启用（添加到~/.bashrc或/etc/environment）

export DEBUGINFOD_URLS="https://debuginfod.ubuntu.com https://debuginfod.elfutils.org"
export DEBUGINFOD_URLS="https://debuginfod.ubuntu.com https://debuginfod.elfutils.org"

GDB auto-fetches symbols when DEBUGINFOD_URLS is set

设置DEBUGINFOD_URLS后，GDB会自动获取符号

gdb ./prog core
gdb ./prog core

Manually query

手动查询

debuginfod-find debuginfo <build-id> debuginfod-find source <build-id> /path/to/file.c
undefined
debuginfod-find debuginfo <build-id> debuginfod-find source <build-id> /path/to/file.c
undefined

7. Missing symbols: manual approach

7. 缺失符号：手动处理方式

bash
undefined
bash
undefined

Check if binary has a build ID

检查二进制文件是否包含构建ID

readelf -n ./prog | grep Build
readelf -n ./prog | grep Build

Find the correct debug package

查找对应的调试包

Debian: apt install prog-dbg or prog-dbgsym

Debian系统：apt install prog-dbg或prog-dbgsym

RPM: dnf install prog-debuginfo

RPM系统：dnf install prog-debuginfo

Point GDB to debug symbols directory

为GDB指定调试符号目录

(gdb) set debug-file-directory /usr/lib/debug
(gdb) set debug-file-directory /usr/lib/debug

Or use eu-readelf to dump build ID, then find .debug file

或者使用eu-readelf导出构建ID，然后查找.debug文件

eu-readelf -n ./prog find /usr/lib/debug -name "*.debug" | xargs eu-readelf -n 2>/dev/null | grep <build-id>
undefined
eu-readelf -n ./prog find /usr/lib/debug -name "*.debug" | xargs eu-readelf -n 2>/dev/null | grep <build-id>
undefined

8. Strip binaries and keep symbols

8. 剥离二进制文件符号并保留符号文件

Best practice: build with symbols, strip for distribution, keep an unstripped copy.
bash
undefined
最佳实践：编译时保留符号，剥离符号后用于分发，保留未剥离符号的副本。
bash
undefined

Build

编译

gcc -g -O2 -o prog main.c
gcc -g -O2 -o prog main.c

Separate debug info

分离调试信息

objcopy --only-keep-debug prog prog.debug objcopy --strip-debug prog prog.stripped
objcopy --only-keep-debug prog prog.debug objcopy --strip-debug prog prog.stripped

Add a debuglink so GDB finds the debug file automatically

添加debuglink，让GDB自动找到调试文件

objcopy --add-gnu-debuglink=prog.debug prog.stripped
objcopy --add-gnu-debuglink=prog.debug prog.stripped

Deploy prog.stripped; keep prog.debug in a symbols store indexed by build-id

部署prog.stripped；将prog.debug存储在按构建ID索引的符号仓库中

undefined
undefined

9. Quick triage from core without full debug session

9. 无需完整调试会话快速排查core文件

bash
undefined
bash
undefined

Print backtrace non-interactively

非交互式打印调用栈

gdb -batch -ex 'bt full' -ex 'thread apply all bt full' ./prog core 2>&1 | tee crash.txt
gdb -batch -ex 'bt full' -ex 'thread apply all bt full' ./prog core 2>&1 | tee crash.txt

Print registers

打印寄存器信息

gdb -batch -ex 'info registers' ./prog core
gdb -batch -ex 'info registers' ./prog core

Check signal that caused crash

检查导致崩溃的信号

gdb -batch -ex 'info signal' ./prog core

For a full cheatsheet covering core pattern tokens, coredumpctl, GDB/LLDB commands, debuginfod servers, and strip/symbol workflows, see [references/cheatsheet.md](references/cheatsheet.md).
gdb -batch -ex 'info signal' ./prog core

如需涵盖core文件命名规则、coredumpctl、GDB/LLDB命令、debuginfod服务器、符号剥离/保留流程的完整速查表，请查看[references/cheatsheet.md](references/cheatsheet.md)。

Related skills

相关技能

  • Use 
    skills/debuggers/gdb
    for full GDB session details
  • Use 
    skills/debuggers/lldb
    for LLDB-based analysis
  • Use 
    skills/runtimes/sanitizers
    to catch the bug before it reaches production
  • Use 
    skills/binaries/elf-inspection
    for 
    readelf
    , build IDs, and binary inspection
  • 使用
    skills/debuggers/gdb
    获取完整GDB会话细节
  • 使用
    skills/debuggers/lldb
    进行基于LLDB的分析
  • 使用
    skills/runtimes/sanitizers
    在问题进入生产环境前捕获bug
  • 使用
    skills/binaries/elf-inspection
    进行
    readelf
    、构建ID和二进制文件检查