analyzing-docker-container-forensics

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Analyzing Docker Container Forensics

Docker容器取证分析

When to Use

适用场景

  • When investigating a compromised Docker container or container host
  • For analyzing malicious Docker images pulled from registries
  • During incident response involving containerized application breaches
  • When examining container escape attempts or privilege escalation
  • For auditing container configurations and identifying misconfigurations
  • 调查被入侵的Docker容器或容器宿主机时
  • 分析从镜像仓库拉取的恶意Docker镜像时
  • 涉及容器化应用 breach 的事件响应过程中
  • 检查容器逃逸尝试或权限提升行为时
  • 审计容器配置并识别配置错误时

Prerequisites

前置条件

  • Docker CLI access on the forensic workstation
  • Access to the Docker host file system (forensic image or live)
  • Understanding of Docker layered file system (overlay2, aufs)
  • dive, docker-explorer, or container-diff for image analysis
  • Knowledge of Docker daemon configuration and socket security
  • Trivy or Grype for vulnerability scanning of container images
  • 取证工作站具备Docker CLI访问权限
  • 可访问Docker宿主机文件系统(取证镜像或实时系统)
  • 了解Docker分层文件系统(overlay2、aufs)
  • 用于镜像分析的dive、docker-explorer或container-diff工具
  • 了解Docker守护进程配置与套接字安全
  • 用于容器镜像漏洞扫描的Trivy或Grype工具

Workflow

工作流程

Step 1: Preserve Container State and Evidence

步骤1:保留容器状态与证据

bash
undefined
bash
undefined

List all containers (including stopped)

List all containers (including stopped)

docker ps -a --no-trunc > /cases/case-2024-001/docker/container_list.txt
docker ps -a --no-trunc > /cases/case-2024-001/docker/container_list.txt

Inspect the compromised container

Inspect the compromised container

CONTAINER_ID="abc123def456" docker inspect $CONTAINER_ID > /cases/case-2024-001/docker/container_inspect.json
CONTAINER_ID="abc123def456" docker inspect $CONTAINER_ID > /cases/case-2024-001/docker/container_inspect.json

Export container filesystem as tarball (preserves current state)

Export container filesystem as tarball (preserves current state)

docker export $CONTAINER_ID > /cases/case-2024-001/docker/container_export.tar
docker export $CONTAINER_ID > /cases/case-2024-001/docker/container_export.tar

Create an image from the container's current state

Create an image from the container's current state

docker commit $CONTAINER_ID forensic-evidence:case-2024-001 docker save forensic-evidence:case-2024-001 > /cases/case-2024-001/docker/container_image.tar
docker commit $CONTAINER_ID forensic-evidence:case-2024-001 docker save forensic-evidence:case-2024-001 > /cases/case-2024-001/docker/container_image.tar

Capture container logs

Capture container logs

docker logs $CONTAINER_ID --timestamps > /cases/case-2024-001/docker/container_logs.txt 2>&1
docker logs $CONTAINER_ID --timestamps > /cases/case-2024-001/docker/container_logs.txt 2>&1

Capture running processes (if container is still running)

Capture running processes (if container is still running)

docker top $CONTAINER_ID > /cases/case-2024-001/docker/container_processes.txt
docker top $CONTAINER_ID > /cases/case-2024-001/docker/container_processes.txt

Capture network connections

Capture network connections

docker exec $CONTAINER_ID netstat -tlnp 2>/dev/null > /cases/case-2024-001/docker/container_network.txt
docker exec $CONTAINER_ID netstat -tlnp 2>/dev/null > /cases/case-2024-001/docker/container_network.txt

Copy specific files from the container

Copy specific files from the container

docker cp $CONTAINER_ID:/var/log/ /cases/case-2024-001/docker/container_var_log/ docker cp $CONTAINER_ID:/tmp/ /cases/case-2024-001/docker/container_tmp/ docker cp $CONTAINER_ID:/etc/passwd /cases/case-2024-001/docker/container_passwd
docker cp $CONTAINER_ID:/var/log/ /cases/case-2024-001/docker/container_var_log/ docker cp $CONTAINER_ID:/tmp/ /cases/case-2024-001/docker/container_tmp/ docker cp $CONTAINER_ID:/etc/passwd /cases/case-2024-001/docker/container_passwd

Hash all exported evidence

Hash all exported evidence

sha256sum /cases/case-2024-001/docker/*.tar > /cases/case-2024-001/docker/evidence_hashes.txt
undefined
sha256sum /cases/case-2024-001/docker/*.tar > /cases/case-2024-001/docker/evidence_hashes.txt
undefined

Step 2: Analyze Container Image Layers

步骤2:分析容器镜像分层

bash
undefined
bash
undefined

Install dive for image layer analysis

Install dive for image layer analysis

Analyze image layers interactively

Analyze image layers interactively

dive forensic-evidence:case-2024-001
dive forensic-evidence:case-2024-001

Non-interactive layer analysis

Non-interactive layer analysis

dive forensic-evidence:case-2024-001 --ci --json /cases/case-2024-001/docker/dive_analysis.json
dive forensic-evidence:case-2024-001 --ci --json /cases/case-2024-001/docker/dive_analysis.json

Extract and examine individual layers

Extract and examine individual layers

mkdir -p /cases/case-2024-001/docker/layers/ tar -xf /cases/case-2024-001/docker/container_image.tar -C /cases/case-2024-001/docker/layers/
mkdir -p /cases/case-2024-001/docker/layers/ tar -xf /cases/case-2024-001/docker/container_image.tar -C /cases/case-2024-001/docker/layers/

List the image manifest and layer order

List the image manifest and layer order

cat /cases/case-2024-001/docker/layers/manifest.json | python3 -m json.tool
cat /cases/case-2024-001/docker/layers/manifest.json | python3 -m json.tool

Examine each layer for changes

Examine each layer for changes

for layer in /cases/case-2024-001/docker/layers/*/layer.tar; do echo "=== Layer: $(dirname $layer | xargs basename) ===" tar -tf "$layer" | head -20 echo "..." done
for layer in /cases/case-2024-001/docker/layers/*/layer.tar; do echo "=== Layer: $(dirname $layer | xargs basename) ===" tar -tf "$layer" | head -20 echo "..." done

Use container-diff to compare with original base image

Use container-diff to compare with original base image

Install container-diff

Install container-diff

Compare committed image with original

Compare committed image with original

./container-diff-linux-amd64 diff daemon://nginx:latest daemon://forensic-evidence:case-2024-001
--type=file --type=apt --type=history --json \
/cases/case-2024-001/docker/container_diff.json
undefined
./container-diff-linux-amd64 diff daemon://nginx:latest daemon://forensic-evidence:case-2024-001
--type=file --type=apt --type=history --json \
/cases/case-2024-001/docker/container_diff.json
undefined

Step 3: Examine Docker Host Artifacts

步骤3:检查Docker宿主机工件

bash
undefined
bash
undefined

Docker data directory (default: /var/lib/docker/)

Docker data directory (default: /var/lib/docker/)

DOCKER_ROOT="/mnt/evidence/var/lib/docker"
DOCKER_ROOT="/mnt/evidence/var/lib/docker"

Examine overlay2 filesystem layers

Examine overlay2 filesystem layers

ls -la $DOCKER_ROOT/overlay2/
ls -la $DOCKER_ROOT/overlay2/

Find the container's merged filesystem

Find the container's merged filesystem

CONTAINER_HASH=$(docker inspect $CONTAINER_ID --format '{{.GraphDriver.Data.MergedDir}}' 2>/dev/null)
CONTAINER_HASH=$(docker inspect $CONTAINER_ID --format '{{.GraphDriver.Data.MergedDir}}' 2>/dev/null)

Or manually from forensic image:

Or manually from forensic image:

Look in /var/lib/docker/containers/<container_id>/config.v2.json

Look in /var/lib/docker/containers/<container_id>/config.v2.json

Analyze container configuration files

Analyze container configuration files

cat $DOCKER_ROOT/containers/$CONTAINER_ID/config.v2.json | python3 -m json.tool \
/cases/case-2024-001/docker/container_config.json
cat $DOCKER_ROOT/containers/$CONTAINER_ID/config.v2.json | python3 -m json.tool \
/cases/case-2024-001/docker/container_config.json

Check Docker daemon configuration

Check Docker daemon configuration

cat /mnt/evidence/etc/docker/daemon.json 2>/dev/null > /cases/case-2024-001/docker/daemon_config.json
cat /mnt/evidence/etc/docker/daemon.json 2>/dev/null > /cases/case-2024-001/docker/daemon_config.json

Examine Docker events log

Examine Docker events log

cat $DOCKER_ROOT/containers/$CONTAINER_ID/*.log > /cases/case-2024-001/docker/container_json_logs.txt
cat $DOCKER_ROOT/containers/$CONTAINER_ID/*.log > /cases/case-2024-001/docker/container_json_logs.txt

Check for volume mounts (potential host filesystem access)

Check for volume mounts (potential host filesystem access)

python3 << 'PYEOF' import json
with open('/cases/case-2024-001/docker/container_inspect.json') as f: data = json.load(f)
inspect = data[0] if isinstance(data, list) else data
print("=== CONTAINER SECURITY ANALYSIS ===\n")
python3 << 'PYEOF' import json
with open('/cases/case-2024-001/docker/container_inspect.json') as f: data = json.load(f)
inspect = data[0] if isinstance(data, list) else data
print("=== CONTAINER SECURITY ANALYSIS ===\n")

Check mounts

Check mounts

print("Volume Mounts:") for mount in inspect.get('Mounts', []): rw = "READ-WRITE" if mount.get('RW') else "READ-ONLY" print(f" {mount.get('Source', 'N/A')} -> {mount.get('Destination', 'N/A')} ({rw})") if mount.get('Source') in ('/', '/etc', '/var', '/root') and mount.get('RW'): print(f" WARNING: Sensitive host path mounted read-write!")
print("Volume Mounts:") for mount in inspect.get('Mounts', []): rw = "READ-WRITE" if mount.get('RW') else "READ-ONLY" print(f" {mount.get('Source', 'N/A')} -> {mount.get('Destination', 'N/A')} ({rw})") if mount.get('Source') in ('/', '/etc', '/var', '/root') and mount.get('RW'): print(f" WARNING: Sensitive host path mounted read-write!")

Check privileged mode

Check privileged mode

host_config = inspect.get('HostConfig', {}) if host_config.get('Privileged'): print("\nWARNING: Container was running in PRIVILEGED mode!")
host_config = inspect.get('HostConfig', {}) if host_config.get('Privileged'): print("\nWARNING: Container was running in PRIVILEGED mode!")

Check capabilities

Check capabilities

cap_add = host_config.get('CapAdd', []) if cap_add: print(f"\nAdded Capabilities: {cap_add}") dangerous_caps = ['SYS_ADMIN', 'SYS_PTRACE', 'NET_ADMIN', 'SYS_MODULE'] for cap in cap_add: if cap in dangerous_caps: print(f" WARNING: Dangerous capability: {cap}")
cap_add = host_config.get('CapAdd', []) if cap_add: print(f"\nAdded Capabilities: {cap_add}") dangerous_caps = ['SYS_ADMIN', 'SYS_PTRACE', 'NET_ADMIN', 'SYS_MODULE'] for cap in cap_add: if cap in dangerous_caps: print(f" WARNING: Dangerous capability: {cap}")

Check PID namespace

Check PID namespace

if host_config.get('PidMode') == 'host': print("\nWARNING: Container shares host PID namespace!")
if host_config.get('PidMode') == 'host': print("\nWARNING: Container shares host PID namespace!")

Check network mode

Check network mode

if host_config.get('NetworkMode') == 'host': print("\nWARNING: Container shares host network namespace!")
if host_config.get('NetworkMode') == 'host': print("\nWARNING: Container shares host network namespace!")

Check user

Check user

user = inspect.get('Config', {}).get('User', 'root (default)') print(f"\nRunning as user: {user}")
user = inspect.get('Config', {}).get('User', 'root (default)') print(f"\nRunning as user: {user}")

Check environment variables for secrets

Check environment variables for secrets

env_vars = inspect.get('Config', {}).get('Env', []) print(f"\nEnvironment Variables: {len(env_vars)}") for env in env_vars: key = env.split('=')[0] if any(s in key.upper() for s in ['PASSWORD', 'SECRET', 'KEY', 'TOKEN', 'CREDENTIAL']): print(f" SENSITIVE: {key}=REDACTED") PYEOF
undefined
env_vars = inspect.get('Config', {}).get('Env', []) print(f"\nEnvironment Variables: {len(env_vars)}") for env in env_vars: key = env.split('=')[0] if any(s in key.upper() for s in ['PASSWORD', 'SECRET', 'KEY', 'TOKEN', 'CREDENTIAL']): print(f" SENSITIVE: {key}=REDACTED") PYEOF
undefined

Step 4: Analyze Container File System Changes

步骤4:分析容器文件系统变更

bash
undefined
bash
undefined

Compare container filesystem to original image

Compare container filesystem to original image

docker diff $CONTAINER_ID > /cases/case-2024-001/docker/filesystem_changes.txt
docker diff $CONTAINER_ID > /cases/case-2024-001/docker/filesystem_changes.txt

A = Added, C = Changed, D = Deleted

A = Added, C = Changed, D = Deleted

Analyze changes

Analyze changes

python3 << 'PYEOF' added = [] changed = [] deleted = []
with open('/cases/case-2024-001/docker/filesystem_changes.txt') as f: for line in f: line = line.strip() if line.startswith('A '): added.append(line[2:]) elif line.startswith('C '): changed.append(line[2:]) elif line.startswith('D '): deleted.append(line[2:])
print(f"Files Added: {len(added)}") print(f"Files Changed: {len(changed)}") print(f"Files Deleted: {len(deleted)}")
python3 << 'PYEOF' added = [] changed = [] deleted = []
with open('/cases/case-2024-001/docker/filesystem_changes.txt') as f: for line in f: line = line.strip() if line.startswith('A '): added.append(line[2:]) elif line.startswith('C '): changed.append(line[2:]) elif line.startswith('D '): deleted.append(line[2:])
print(f"Files Added: {len(added)}") print(f"Files Changed: {len(changed)}") print(f"Files Deleted: {len(deleted)}")

Flag suspicious additions

Flag suspicious additions

suspicious = [f for f in added if any(s in f for s in ['/tmp/', '/dev/shm/', '/root/', '.sh', '.py', '.elf', 'reverse', 'shell', 'backdoor'])] if suspicious: print(f"\nSuspicious Added Files:") for f in suspicious: print(f" {f}")
suspicious = [f for f in added if any(s in f for s in ['/tmp/', '/dev/shm/', '/root/', '.sh', '.py', '.elf', 'reverse', 'shell', 'backdoor'])] if suspicious: print(f"\nSuspicious Added Files:") for f in suspicious: print(f" {f}")

Flag suspicious changes

Flag suspicious changes

sus_changed = [f for f in changed if any(s in f for s in ['/etc/passwd', '/etc/shadow', '/etc/crontab', '/etc/ssh', '.bashrc'])] if sus_changed: print(f"\nSuspicious Changed Files:") for f in sus_changed: print(f" {f}") PYEOF
sus_changed = [f for f in changed if any(s in f for s in ['/etc/passwd', '/etc/shadow', '/etc/crontab', '/etc/ssh', '.bashrc'])] if sus_changed: print(f"\nSuspicious Changed Files:") for f in sus_changed: print(f" {f}") PYEOF

Extract and examine the container export

Extract and examine the container export

mkdir -p /cases/case-2024-001/docker/container_fs/ tar -xf /cases/case-2024-001/docker/container_export.tar -C /cases/case-2024-001/docker/container_fs/
mkdir -p /cases/case-2024-001/docker/container_fs/ tar -xf /cases/case-2024-001/docker/container_export.tar -C /cases/case-2024-001/docker/container_fs/

Scan for webshells and malicious files

Scan for webshells and malicious files

find /cases/case-2024-001/docker/container_fs/tmp/ -type f -exec file {} ; find /cases/case-2024-001/docker/container_fs/ -name "*.php" -newer /cases/case-2024-001/docker/container_fs/etc/hostname
undefined
find /cases/case-2024-001/docker/container_fs/tmp/ -type f -exec file {} ; find /cases/case-2024-001/docker/container_fs/ -name "*.php" -newer /cases/case-2024-001/docker/container_fs/etc/hostname
undefined

Step 5: Scan for Vulnerabilities and Generate Report

步骤5:漏洞扫描与报告生成

bash
undefined
bash
undefined

Scan the image for known vulnerabilities

Scan the image for known vulnerabilities

trivy image forensic-evidence:case-2024-001
--format json
--output /cases/case-2024-001/docker/vulnerability_scan.json
trivy image forensic-evidence:case-2024-001
--format json
--output /cases/case-2024-001/docker/vulnerability_scan.json

Scan the exported filesystem

Scan the exported filesystem

trivy fs /cases/case-2024-001/docker/container_fs/
--format table
--output /cases/case-2024-001/docker/fs_vulnerabilities.txt
trivy fs /cases/case-2024-001/docker/container_fs/
--format table
--output /cases/case-2024-001/docker/fs_vulnerabilities.txt

Check for secrets in the image

Check for secrets in the image

trivy image forensic-evidence:case-2024-001
--scanners secret
--format json
--output /cases/case-2024-001/docker/secrets_scan.json
undefined
trivy image forensic-evidence:case-2024-001
--scanners secret
--format json
--output /cases/case-2024-001/docker/secrets_scan.json
undefined

Key Concepts

核心概念

ConceptDescription
Image layersRead-only filesystem layers stacked to form the container image
overlay2Default Docker storage driver using union filesystem for layers
Container diffComparison of runtime filesystem changes against the original image
Privileged modeContainer with full host capabilities (bypasses most isolation)
Docker socketUnix socket (/var/run/docker.sock) controlling the Docker daemon
Container escapeTechnique for breaking out of container isolation to the host
Volume mountsHost filesystem paths made accessible inside the container
Image historyRecord of Dockerfile instructions used to build each layer
概念描述
镜像分层堆叠形成容器镜像的只读文件系统分层
overlay2Docker默认存储驱动,使用联合文件系统实现分层
容器差异对比对比运行时文件系统与原始镜像的变更
特权模式拥有宿主机全部权限的容器模式(绕过大部分隔离机制)
Docker套接字控制Docker守护进程的Unix套接字(/var/run/docker.sock)
容器逃逸突破容器隔离机制,获取宿主机访问权限的技术
卷挂载宿主机文件系统路径映射到容器内部的机制
镜像历史记录构建每个分层所用Dockerfile指令的日志

Tools & Systems

工具与系统

ToolPurpose
docker inspectDetailed container configuration and state information
docker diffShow filesystem changes made in a running/stopped container
diveInteractive Docker image layer analysis tool
container-diffGoogle tool for comparing container image contents
TrivyVulnerability scanner for container images and filesystems
docker-explorerForensic tool for offline Docker artifact analysis
SysdigContainer runtime security monitoring and forensics
FalcoRuntime threat detection for containers and Kubernetes
工具用途
docker inspect获取容器详细配置与状态信息
docker diff显示运行中/已停止容器的文件系统变更
dive交互式Docker镜像分层分析工具
container-diffGoogle推出的容器镜像内容对比工具
Trivy容器镜像与文件系统漏洞扫描工具
docker-explorer用于离线Docker工件分析的取证工具
Sysdig容器运行时安全监控与取证工具
Falco容器与Kubernetes运行时威胁检测工具

Common Scenarios

常见场景

Scenario 1: Web Application Container Compromise Export the container filesystem, identify webshells in web root, analyze access logs for exploitation attempts, check for added files and modified configurations, examine network connections for C2 communication, review container capabilities for escalation paths.
Scenario 2: Supply Chain Attack via Malicious Image Analyze image layers with dive to identify which layer added malicious content, compare with the official base image using container-diff, check image history for suspicious RUN commands, scan for embedded backdoors and cryptocurrency miners, trace the image pull from registry logs.
Scenario 3: Container Escape Investigation Check if container ran privileged or with dangerous capabilities, examine host filesystem mount points for unauthorized access, review Docker socket mount enabling Docker-in-Docker abuse, analyze host system logs for container escape indicators, check for kernel exploit artifacts.
Scenario 4: Cryptojacking in Container Environment Identify high-CPU containers, export and analyze the container image for mining binaries, check for unauthorized images in the registry, review container creation events for rogue deployments, examine network connections for mining pool communications.
场景1:Web应用容器被入侵 导出容器文件系统,识别网站根目录中的webshell,分析访问日志以查找攻击尝试,检查新增文件与修改的配置,排查用于C2通信的网络连接,查看容器权限以寻找提权路径。
场景2:恶意镜像导致的供应链攻击 使用dive分析镜像分层,识别添加恶意内容的分层,通过container-diff与官方基础镜像对比,检查镜像历史中的可疑RUN命令,扫描嵌入的后门与加密货币矿工,从仓库日志追踪镜像拉取记录。
场景3:容器逃逸调查 检查容器是否以特权模式运行或拥有危险权限,排查宿主机文件系统挂载点是否存在未授权访问,查看是否挂载Docker套接字以实现Docker-in-Docker滥用,分析宿主机系统日志中的容器逃逸迹象,检查内核漏洞利用痕迹。
场景4:容器环境中的挖矿攻击 识别高CPU占用的容器,导出并分析容器镜像中的挖矿二进制文件,检查镜像仓库中的未授权镜像,查看容器创建事件中的恶意部署记录,排查用于矿池通信的网络连接。

Output Format

输出格式

Docker Container Forensics Summary:
  Container: abc123def456 (nginx-app)
  Image: company/web-app:v2.1
  Status: Running (started 2024-01-10 09:00 UTC)
  Host: docker-host-01.corp.local

  Security Configuration:
    Privileged: No
    Capabilities Added: NET_ADMIN (WARNING)
    Volume Mounts: /var/log -> /host-logs (RW)
    Network Mode: bridge
    User: root (WARNING)

  Filesystem Changes:
    Added: 23 files (5 suspicious)
    Changed: 12 files (2 suspicious)
    Deleted: 0 files

  Suspicious Findings:
    /tmp/reverse.sh - Reverse shell script (Added)
    /var/www/html/.hidden/shell.php - PHP webshell (Added)
    /etc/crontab - Modified (persistence cron entry added)
    /root/.ssh/authorized_keys - Modified (unauthorized key added)

  Vulnerability Scan:
    Critical: 3 (CVE-2024-xxxx in base image)
    High: 12
    Medium: 34

  Evidence: /cases/case-2024-001/docker/
Docker Container Forensics Summary:
  Container: abc123def456 (nginx-app)
  Image: company/web-app:v2.1
  Status: Running (started 2024-01-10 09:00 UTC)
  Host: docker-host-01.corp.local

  Security Configuration:
    Privileged: No
    Capabilities Added: NET_ADMIN (WARNING)
    Volume Mounts: /var/log -> /host-logs (RW)
    Network Mode: bridge
    User: root (WARNING)

  Filesystem Changes:
    Added: 23 files (5 suspicious)
    Changed: 12 files (2 suspicious)
    Deleted: 0 files

  Suspicious Findings:
    /tmp/reverse.sh - Reverse shell script (Added)
    /var/www/html/.hidden/shell.php - PHP webshell (Added)
    /etc/crontab - Modified (persistence cron entry added)
    /root/.ssh/authorized_keys - Modified (unauthorized key added)

  Vulnerability Scan:
    Critical: 3 (CVE-2024-xxxx in base image)
    High: 12
    Medium: 34

  Evidence: /cases/case-2024-001/docker/