Back to Details

analyzing-security-logs-with-splunk

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Analyzing Security Logs with Splunk

使用Splunk分析安全日志

When to Use

适用场景

  • Investigating a security incident that requires correlation across multiple log sources
  • Hunting for adversary activity using known TTPs and IOCs
  • Building detection rules for specific attack patterns
  • Reconstructing an incident timeline from disparate log sources
  • Analyzing authentication anomalies, lateral movement, or data exfiltration patterns
Do not use for real-time packet-level analysis; use Wireshark or Zeek for full packet capture analysis.
  • 调查需要跨多个日志源关联分析的安全事件
  • 利用已知TTP和IOC追踪攻击者活动
  • 针对特定攻击模式构建检测规则
  • 从分散的日志源重建事件时间线
  • 分析认证异常、横向移动或数据泄露模式
请勿用于实时数据包级分析；如需完整数据包捕获分析，请使用Wireshark或Zeek。

Prerequisites

前提条件

  • Splunk Enterprise or Splunk Cloud with Enterprise Security (ES) app installed
  • Log sources ingested: Windows Event Logs (via Splunk Universal Forwarder or WEF), firewall, proxy, DNS, EDR, email gateway
  • Splunk CIM (Common Information Model) data models configured for normalized field names
  • SPL proficiency at intermediate level or higher
  • Role-based access with 
    search
    and 
    accelerate_search
    capabilities in Splunk
  • 已安装Enterprise Security (ES)应用的Splunk Enterprise或Splunk Cloud
  • 已采集的日志源：Windows事件日志（通过Splunk Universal Forwarder或WEF）、防火墙、代理、DNS、EDR、邮件网关
  • 已配置Splunk CIM（Common Information Model，通用信息模型）数据模型以实现字段标准化
  • 具备中级或更高水平的SPL使用能力
  • 在Splunk中拥有具备
    search
    和
    accelerate_search
    权限的角色

Workflow

工作流程

Step 1: Scope the Investigation in Splunk

步骤1：在Splunk中确定调查范围

Define search parameters based on incident triage data:
spl
| Set initial investigation scope
index=windows OR index=firewall OR index=proxy
  earliest="2025-11-14T00:00:00" latest="2025-11-16T00:00:00"
  (host="WKSTN-042" OR src_ip="10.1.5.42" OR user="jsmith")
| stats count by index, sourcetype, host
| sort -count
This query establishes which log sources contain relevant data for the investigation timeframe and affected assets.
根据事件分诊数据定义搜索参数：
spl
| Set initial investigation scope
index=windows OR index=firewall OR index=proxy
  earliest="2025-11-14T00:00:00" latest="2025-11-16T00:00:00"
  (host="WKSTN-042" OR src_ip="10.1.5.42" OR user="jsmith")
| stats count by index, sourcetype, host
| sort -count
该查询确定了哪些日志源包含调查时间范围和受影响资产的相关数据。

Step 2: Analyze Authentication Events

步骤2：分析认证事件

Investigate suspicious authentication patterns using Windows Security Event Logs:
spl
| Detect brute force and credential stuffing
index=windows sourcetype="WinEventLog:Security" EventCode=4625
  earliest=-24h
| stats count as failed_attempts, values(src_ip) as source_ips,
  dc(src_ip) as unique_sources by TargetUserName
| where failed_attempts > 10
| sort -failed_attempts

| Detect pass-the-hash (Logon Type 9 - NewCredentials)
index=windows sourcetype="WinEventLog:Security" EventCode=4624
  Logon_Type=9
| table _time, host, TargetUserName, src_ip, LogonProcessName

| Detect lateral movement via RDP
index=windows sourcetype="WinEventLog:Security" EventCode=4624
  Logon_Type=10
| stats count, values(host) as targets by TargetUserName, src_ip
| where count > 3
| sort -count
使用Windows安全事件日志调查可疑认证模式：
spl
| Detect brute force and credential stuffing
index=windows sourcetype="WinEventLog:Security" EventCode=4625
  earliest=-24h
| stats count as failed_attempts, values(src_ip) as source_ips,
  dc(src_ip) as unique_sources by TargetUserName
| where failed_attempts > 10
| sort -failed_attempts

| Detect pass-the-hash (Logon Type 9 - NewCredentials)
index=windows sourcetype="WinEventLog:Security" EventCode=4624
  Logon_Type=9
| table _time, host, TargetUserName, src_ip, LogonProcessName

| Detect lateral movement via RDP
index=windows sourcetype="WinEventLog:Security" EventCode=4624
  Logon_Type=10
| stats count, values(host) as targets by TargetUserName, src_ip
| where count > 3
| sort -count

Step 3: Trace Process Execution

步骤3：追踪进程执行

Use Sysmon logs to reconstruct process execution chains:
spl
| Process creation with parent chain (Sysmon Event ID 1)
index=sysmon EventCode=1 host="WKSTN-042"
  earliest="2025-11-15T14:00:00" latest="2025-11-15T15:00:00"
| table _time, ParentImage, ParentCommandLine, Image, CommandLine, User, Hashes
| sort _time

| Detect suspicious PowerShell execution
index=sysmon EventCode=1 Image="*\\powershell.exe"
  (CommandLine="*-enc*" OR CommandLine="*-encodedcommand*"
   OR CommandLine="*downloadstring*" OR CommandLine="*iex*")
| table _time, host, User, ParentImage, CommandLine
| sort _time

| Detect LSASS credential dumping
index=sysmon EventCode=10 TargetImage="*\\lsass.exe"
  GrantedAccess=0x1010
| table _time, host, SourceImage, SourceUser, GrantedAccess
使用Sysmon日志重建进程执行链：
spl
| Process creation with parent chain (Sysmon Event ID 1)
index=sysmon EventCode=1 host="WKSTN-042"
  earliest="2025-11-15T14:00:00" latest="2025-11-15T15:00:00"
| table _time, ParentImage, ParentCommandLine, Image, CommandLine, User, Hashes
| sort _time

| Detect suspicious PowerShell execution
index=sysmon EventCode=1 Image="*\\powershell.exe"
  (CommandLine="*-enc*" OR CommandLine="*-encodedcommand*"
   OR CommandLine="*downloadstring*" OR CommandLine="*iex*")
| table _time, host, User, ParentImage, CommandLine
| sort _time

| Detect LSASS credential dumping
index=sysmon EventCode=10 TargetImage="*\\lsass.exe"
  GrantedAccess=0x1010
| table _time, host, SourceImage, SourceUser, GrantedAccess

Step 4: Analyze Network Activity

步骤4：分析网络活动

Correlate network logs with endpoint events:
spl
| Detect C2 beaconing pattern
index=proxy OR index=firewall dest_ip="185.220.101.42"
| timechart span=1m count by src_ip
| where count > 0

| Detect DNS tunneling (high query volume to single domain)
index=dns
| rex field=query "(?<subdomain>[^\.]+)\.(?<domain>[^\.]+\.[^\.]+)$"
| stats count, avg(len(query)) as avg_query_len by domain, src_ip
| where count > 500 AND avg_query_len > 40
| sort -count

| Detect large data transfers (potential exfiltration)
index=proxy action=allowed
| stats sum(bytes_out) as total_bytes by src_ip, dest_ip, dest_host
| eval total_MB=round(total_bytes/1024/1024,2)
| where total_MB > 100
| sort -total_MB
将网络日志与终端事件关联：
spl
| Detect C2 beaconing pattern
index=proxy OR index=firewall dest_ip="185.220.101.42"
| timechart span=1m count by src_ip
| where count > 0

| Detect DNS tunneling (high query volume to single domain)
index=dns
| rex field=query "(?<subdomain>[^\.]+)\.(?<domain>[^\.]+\.[^\.]+)$"
| stats count, avg(len(query)) as avg_query_len by domain, src_ip
| where count > 500 AND avg_query_len > 40
| sort -count

| Detect large data transfers (potential exfiltration)
index=proxy action=allowed
| stats sum(bytes_out) as total_bytes by src_ip, dest_ip, dest_host
| eval total_MB=round(total_bytes/1024/1024,2)
| where total_MB > 100
| sort -total_MB

Step 5: Build the Incident Timeline

步骤5：构建事件时间线

Reconstruct a unified timeline across all log sources:
spl
| Unified incident timeline
index=windows OR index=sysmon OR index=proxy OR index=firewall
  (host="WKSTN-042" OR src_ip="10.1.5.42" OR user="jsmith")
  earliest="2025-11-15T14:00:00" latest="2025-11-15T16:00:00"
| eval event_summary=case(
    sourcetype=="WinEventLog:Security" AND EventCode==4624, "Logon: ".TargetUserName." from ".src_ip,
    sourcetype=="WinEventLog:Security" AND EventCode==4625, "Failed logon: ".TargetUserName,
    sourcetype=="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational" AND EventCode==1,
      "Process: ".Image." by ".User,
    sourcetype=="proxy", "Web: ".http_method." ".url,
    1==1, sourcetype.": ".EventCode)
| table _time, sourcetype, host, event_summary
| sort _time
跨所有日志源重建统一时间线：
spl
| Unified incident timeline
index=windows OR index=sysmon OR index=proxy OR index=firewall
  (host="WKSTN-042" OR src_ip="10.1.5.42" OR user="jsmith")
  earliest="2025-11-15T14:00:00" latest="2025-11-15T16:00:00"
| eval event_summary=case(
    sourcetype=="WinEventLog:Security" AND EventCode==4624, "登录: ".TargetUserName." 来自 ".src_ip,
    sourcetype=="WinEventLog:Security" AND EventCode==4625, "登录失败: ".TargetUserName,
    sourcetype=="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational" AND EventCode==1,
      "进程: ".Image." 由 ".User" 启动",
    sourcetype=="proxy", "Web访问: ".http_method." ".url,
    1==1, sourcetype.": ".EventCode)
| table _time, sourcetype, host, event_summary
| sort _time

Step 6: Create Detection Rules

步骤6：创建检测规则

Convert investigation findings into persistent Splunk correlation searches:
spl
| Correlation search: PowerShell spawned by Office applications
index=sysmon EventCode=1
  Image="*\\powershell.exe"
  (ParentImage="*\\winword.exe" OR ParentImage="*\\excel.exe"
   OR ParentImage="*\\outlook.exe")
| eval severity="high"
| eval mitre_technique="T1059.001"
| collect index=notable_events
将调查结果转换为持久化的Splunk关联搜索：
spl
| Correlation search: PowerShell spawned by Office applications
index=sysmon EventCode=1
  Image="*\\powershell.exe"
  (ParentImage="*\\winword.exe" OR ParentImage="*\\excel.exe"
   OR ParentImage="*\\outlook.exe")
| eval severity="high"
| eval mitre_technique="T1059.001"
| collect index=notable_events

Key Concepts

核心概念

TermDefinition
SPL (Search Processing Language)Splunk's query language for searching, filtering, transforming, and visualizing machine data
CIM (Common Information Model)Splunk's field normalization standard that maps vendor-specific field names to common names for cross-source queries
Notable EventAn event in Splunk Enterprise Security flagged for analyst review based on a correlation search match
Data ModelStructured representation of indexed data in Splunk enabling accelerated searches and pivot-based analysis
SourcetypeClassification label in Splunk that defines the format and parsing rules for a specific log type
Correlation SearchScheduled Splunk search that runs continuously and generates notable events when conditions are met
TimechartSPL command that creates time-series visualizations for identifying patterns, anomalies, and trends
术语定义
SPL (Search Processing Language)Splunk的查询语言，用于搜索、过滤、转换和可视化机器数据
CIM (Common Information Model)Splunk的字段标准化标准，将厂商特定的字段名映射为通用名称，以支持跨源查询
Notable EventSplunk Enterprise Security中，基于关联搜索匹配结果标记为需要分析师审核的事件
Data ModelSplunk中索引数据的结构化表示，支持加速搜索和基于透视的分析
SourcetypeSplunk中的分类标签，定义特定日志类型的格式和解析规则
Correlation SearchSplunk中持续运行的定时搜索，当满足条件时生成Notable Event
TimechartSPL命令，用于创建时间序列可视化，以识别模式、异常和趋势

Tools & Systems

工具与系统

  • Splunk Enterprise Security (ES): Premium SIEM application providing correlation searches, risk-based alerting, and investigation workbench
  • Splunk SOAR: Orchestration platform integrated with Splunk ES for automated response playbooks
  • Sysmon: Microsoft system monitoring tool providing detailed process, network, and file change telemetry ingested into Splunk
  • Splunk Attack Analyzer: Automated threat analysis that detonates suspicious files and URLs, feeding results into Splunk
  • BOSS of the SOC (BOTS): SANS/Splunk training dataset for practicing incident investigation SPL queries
  • Splunk Enterprise Security (ES)：高级SIEM应用，提供关联搜索、基于风险的告警和调查工作台
  • Splunk SOAR：与Splunk ES集成的编排平台，用于自动化响应剧本
  • Sysmon：微软系统监控工具，提供详细的进程、网络和文件变更遥测数据，并采集到Splunk中
  • Splunk Attack Analyzer：自动化威胁分析工具，可检测可疑文件和URL，并将结果导入Splunk
  • BOSS of the SOC (BOTS)：SANS/Splunk训练数据集，用于练习事件调查的SPL查询

Common Scenarios

常见场景

Scenario: Investigating Credential Stuffing Leading to Account Takeover

场景：调查导致账户接管的凭证填充攻击

Context: Security operations receives an alert for multiple successful logins to a single account from geographically dispersed IP addresses within a 30-minute window.
Approach:
  1. Query Event ID 4624 for the affected account to map all login sources and times
  2. Correlate login IPs against threat intelligence feeds using a Splunk lookup table
  3. Check proxy logs for suspicious activity from the authenticated sessions
  4. Search for lateral movement from the compromised account (Event ID 4624 Type 3 to other hosts)
  5. Build a timeline showing credential stuffing attempts, successful login, and post-compromise activity
  6. Create a correlation search to detect similar patterns on other accounts
Pitfalls:
  • Searching only the last 24 hours when the credential stuffing may have occurred over weeks
  • Not checking for VPN logs that may show the same account authenticating from impossible travel distances
  • Failing to normalize timestamps across log sources in different time zones
背景：安全运营团队收到告警，显示单个账户在30分钟内从地理位置分散的多个IP地址成功登录多次。
方法：
  1. 查询受影响账户的Event ID 4624，映射所有登录源和时间
  2. 使用Splunk查找表将登录IP与威胁情报源关联
  3. 检查代理日志中来自已认证会话的可疑活动
  4. 搜索受 compromise 账户的横向移动（Event ID 4624 Type 3到其他主机）
  5. 构建显示凭证填充尝试、成功登录和妥协后活动的时间线
  6. 创建关联搜索以检测其他账户上的类似模式
注意事项：
  • 仅搜索过去24小时的数据，而凭证填充攻击可能持续数周
  • 未检查VPN日志，这些日志可能显示同一账户从不可能的旅行距离进行认证
  • 未对不同时区日志源的时间戳进行标准化

Output Format

输出格式

SPLUNK INVESTIGATION REPORT
============================
Incident:        INC-2025-1547
Analyst:         [Name]
Investigation Period: 2025-11-14 00:00 UTC - 2025-11-16 00:00 UTC

SEARCH SCOPE
Indexes:         windows, sysmon, proxy, firewall, dns
Hosts:           WKSTN-042, SRV-FILE01
Users:           jsmith, svc-backup
Source IPs:      10.1.5.42, 10.1.10.15

KEY FINDINGS
1. [timestamp] - Initial compromise via phishing (Sysmon Event 1)
2. [timestamp] - C2 established (proxy logs, beacon pattern detected)
3. [timestamp] - Credential theft (Sysmon Event 10, LSASS access)
4. [timestamp] - Lateral movement to SRV-FILE01 (Event 4624 Type 3)
5. [timestamp] - Data staging and exfiltration (proxy bytes_out anomaly)

SPL QUERIES USED
[numbered list of key queries with descriptions]

DETECTION GAPS IDENTIFIED
- No Sysmon deployed on SRV-FILE01 (blind spot)
- Proxy logs missing SSL inspection for C2 domain
- PowerShell ScriptBlock logging not enabled

RECOMMENDED DETECTIONS
1. Correlation search for Office-spawned PowerShell
2. Threshold alert for LSASS access patterns
3. Behavioral rule for beacon-interval network traffic
SPLUNK INVESTIGATION REPORT
============================
事件编号:        INC-2025-1547
分析师:         [姓名]
调查时间段: 2025-11-14 00:00 UTC - 2025-11-16 00:00 UTC

搜索范围
索引:         windows, sysmon, proxy, firewall, dns
主机:           WKSTN-042, SRV-FILE01
用户:           jsmith, svc-backup
源IP:      10.1.5.42, 10.1.10.15

关键发现
1. [时间戳] - 初始妥协通过钓鱼攻击（Sysmon Event 1）
2. [时间戳] - 建立C2连接（代理日志，检测到信标模式）
3. [时间戳] - 凭证窃取（Sysmon Event 10，LSASS访问）
4. [时间戳] - 横向移动至SRV-FILE01（Event 4624 Type 3）
5. [时间戳] - 数据暂存与泄露（代理日志bytes_out异常）

使用的SPL查询
[带描述的关键查询编号列表]

识别出的检测缺口
- SRV-FILE01上未部署Sysmon（监控盲区）
- 代理日志缺少对C2域名的SSL检查
- 未启用PowerShell ScriptBlock日志

推荐的检测规则
1. Office应用启动PowerShell的关联搜索
2. LSASS访问模式的阈值告警
3. 信标间隔网络流量的行为规则