analyzing-browser-forensics-with-hindsight

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Analyzing Browser Forensics with Hindsight

使用Hindsight进行浏览器取证分析

Overview

概述

Hindsight is an open-source browser forensics tool designed to parse artifacts from Google Chrome and other Chromium-based browsers (Microsoft Edge, Brave, Opera, Vivaldi). It extracts and correlates data from multiple browser database files to create a unified timeline of web activity. Hindsight can parse URLs, download history, cache records, bookmarks, autofill records, saved passwords, preferences, browser extensions, HTTP cookies, Local Storage (HTML5 cookies), login data, and session/tab information. The tool produces chronological timelines in multiple output formats (XLSX, JSON, SQLite) that enable investigators to reconstruct user web activity for incident response, insider threat investigations, and criminal cases.
Hindsight是一款开源浏览器取证工具,用于解析Google Chrome及其他基于Chromium的浏览器(Microsoft Edge、Brave、Opera、Vivaldi)的工件。它从多个浏览器数据库文件中提取并关联数据,生成统一的网页活动时序时间线。Hindsight可以解析URL、下载历史、缓存记录、书签、自动填充记录、保存的密码、偏好设置、浏览器扩展、HTTP Cookie、Local Storage(HTML5 Cookie)、登录数据以及会话/标签页信息。该工具支持生成多种输出格式(XLSX、JSON、SQLite)的时序时间线,帮助调查人员在事件响应、内部威胁调查和刑事案件中重建用户的网页活动。

When to Use

使用场景

  • When investigating security incidents that require analyzing browser forensics with hindsight
  • When building detection rules or threat hunting queries for this domain
  • When SOC analysts need structured procedures for this analysis type
  • When validating security monitoring coverage for related attack techniques
  • 当需要使用Hindsight分析浏览器取证数据来调查安全事件时
  • 当为此领域构建检测规则或威胁狩猎查询时
  • 当SOC分析师需要此类分析的结构化流程时
  • 当验证相关攻击技术的安全监控覆盖范围时

Prerequisites

前提条件

  • Python 3.8+ with Hindsight installed (
    pip install pyhindsight
    )
  • Access to browser profile directories from forensic image
  • Browser profile data (not encrypted with OS-level encryption)
  • Timeline Explorer or spreadsheet application for analysis
  • Python 3.8及以上版本,且已安装Hindsight(
    pip install pyhindsight
  • 能够访问取证镜像中的浏览器配置文件目录
  • 浏览器配置文件数据(未经过操作系统级加密)
  • 用于分析的Timeline Explorer或电子表格应用

Browser Profile Locations

浏览器配置文件位置

BrowserWindows Profile Path
Chrome%LOCALAPPDATA%\Google\Chrome\User Data\Default\
Edge%LOCALAPPDATA%\Microsoft\Edge\User Data\Default\
Brave%LOCALAPPDATA%\BraveSoftware\Brave-Browser\User Data\Default\
Opera%APPDATA%\Opera Software\Opera Stable\
Vivaldi%LOCALAPPDATA%\Vivaldi\User Data\Default\
Chrome (macOS)~/Library/Application Support/Google/Chrome/Default/
Chrome (Linux)~/.config/google-chrome/Default/
浏览器Windows配置文件路径
Chrome%LOCALAPPDATA%\Google\Chrome\User Data\Default\
Edge%LOCALAPPDATA%\Microsoft\Edge\User Data\Default\
Brave%LOCALAPPDATA%\BraveSoftware\Brave-Browser\User Data\Default\
Opera%APPDATA%\Opera Software\Opera Stable\
Vivaldi%LOCALAPPDATA%\Vivaldi\User Data\Default\
Chrome (macOS)~/Library/Application Support/Google/Chrome/Default/
Chrome (Linux)~/.config/google-chrome/Default/

Key Artifact Files

关键工件文件

FileContents
HistoryURL visits, downloads, keyword searches
CookiesHTTP cookies with domain, expiry, values
Web DataAutofill entries, saved credit cards
Login DataSaved usernames/passwords (encrypted)
BookmarksJSON bookmark tree
PreferencesBrowser configuration and extensions
Local Storage/HTML5 Local Storage per domain
Session Storage/Session-specific storage per domain
Network Action PredictorPreviously typed URLs
ShortcutsOmnibox shortcuts and predictions
Top SitesFrequently visited sites
文件内容
HistoryURL访问记录、下载记录、关键词搜索记录
Cookies包含域名、过期时间、值的HTTP Cookie
Web Data自动填充条目、保存的信用卡信息
Login Data保存的用户名/密码(已加密)
BookmarksJSON格式的书签树
Preferences浏览器配置和扩展信息
Local Storage/按域名划分的HTML5本地存储
Session Storage/按会话划分的特定存储
Network Action Predictor之前输入过的URL
Shortcuts地址栏快捷方式和预测内容
Top Sites频繁访问的网站

Running Hindsight

运行Hindsight

Command Line

命令行

bash
undefined
bash
undefined

Basic analysis of a Chrome profile

Basic analysis of a Chrome profile

hindsight.exe -i "C:\Evidence\Users\suspect\AppData\Local\Google\Chrome\User Data\Default" -o C:\Output\chrome_analysis
hindsight.exe -i "C:\Evidence\Users\suspect\AppData\Local\Google\Chrome\User Data\Default" -o C:\Output\chrome_analysis

Specify browser type

Specify browser type

hindsight.exe -i "/path/to/profile" -o /output/analysis -b Chrome
hindsight.exe -i "/path/to/profile" -o /output/analysis -b Chrome

JSON output format

JSON output format

hindsight.exe -i "C:\Evidence\Chrome\Default" -o C:\Output\chrome --format jsonl
hindsight.exe -i "C:\Evidence\Chrome\Default" -o C:\Output\chrome --format jsonl

With cache parsing (slower but more complete)

With cache parsing (slower but more complete)

hindsight.exe -i "C:\Evidence\Chrome\Default" -o C:\Output\chrome --cache
undefined
hindsight.exe -i "C:\Evidence\Chrome\Default" -o C:\Output\chrome --cache
undefined

Web UI

Web UI

bash
undefined
bash
undefined

Start Hindsight web interface

Start Hindsight web interface

hindsight_gui.exe
hindsight_gui.exe

Upload or point to browser profile directory

Upload or point to browser profile directory

Configure output format and analysis options

Configure output format and analysis options

Generate and download report

Generate and download report

undefined
undefined

Artifact Analysis Details

工件分析详情

URL History and Visits

URL历史与访问记录

sql
-- Chrome History database schema (key tables)
-- urls table: id, url, title, visit_count, typed_count, last_visit_time
-- visits table: id, url, visit_time, from_visit, transition, segment_id

-- Timestamps are Chrome/WebKit format: microseconds since 1601-01-01
-- Convert: datetime((visit_time/1000000)-11644473600, 'unixepoch')
sql
-- Chrome History database schema (key tables)
-- urls table: id, url, title, visit_count, typed_count, last_visit_time
-- visits table: id, url, visit_time, from_visit, transition, segment_id

-- Timestamps are Chrome/WebKit format: microseconds since 1601-01-01
-- Convert: datetime((visit_time/1000000)-11644473600, 'unixepoch')

Download History

下载历史

sql
-- downloads table: id, current_path, target_path, start_time, end_time,
--   received_bytes, total_bytes, state, danger_type, interrupt_reason,
--   url, referrer, tab_url, mime_type, original_mime_type
sql
-- downloads table: id, current_path, target_path, start_time, end_time,
--   received_bytes, total_bytes, state, danger_type, interrupt_reason,
--   url, referrer, tab_url, mime_type, original_mime_type

Cookie Analysis

Cookie分析

sql
-- cookies table: creation_utc, host_key, name, value, encrypted_value,
--   path, expires_utc, is_secure, is_httponly, last_access_utc,
--   has_expires, is_persistent, priority, samesite
sql
-- cookies table: creation_utc, host_key, name, value, encrypted_value,
--   path, expires_utc, is_secure, is_httponly, last_access_utc,
--   has_expires, is_persistent, priority, samesite

Python Analysis Script

Python分析脚本

python
import sqlite3
import os
import json
import sys
from datetime import datetime, timedelta


CHROME_EPOCH = datetime(1601, 1, 1)


def chrome_time_to_datetime(chrome_ts: int):
    """Convert Chrome timestamp to datetime."""
    if chrome_ts == 0:
        return None
    try:
        return CHROME_EPOCH + timedelta(microseconds=chrome_ts)
    except (OverflowError, OSError):
        return None


def analyze_chrome_history(profile_path: str, output_dir: str) -> dict:
    """Analyze Chrome History database for forensic evidence."""
    history_db = os.path.join(profile_path, "History")
    if not os.path.exists(history_db):
        return {"error": "History database not found"}

    os.makedirs(output_dir, exist_ok=True)
    conn = sqlite3.connect(f"file:{history_db}?mode=ro", uri=True)

    # URL visits with timestamps
    cursor = conn.cursor()
    cursor.execute("""
        SELECT u.url, u.title, v.visit_time, u.visit_count,
               v.transition & 0xFF as transition_type
        FROM visits v JOIN urls u ON v.url = u.id
        ORDER BY v.visit_time DESC LIMIT 5000
    """)
    visits = [{
        "url": r[0], "title": r[1],
        "visit_time": str(chrome_time_to_datetime(r[2])),
        "total_visits": r[3], "transition": r[4]
    } for r in cursor.fetchall()]

    # Downloads
    cursor.execute("""
        SELECT target_path, tab_url, start_time, end_time,
               received_bytes, total_bytes, mime_type, state
        FROM downloads ORDER BY start_time DESC LIMIT 1000
    """)
    downloads = [{
        "path": r[0], "source_url": r[1],
        "start_time": str(chrome_time_to_datetime(r[2])),
        "end_time": str(chrome_time_to_datetime(r[3])),
        "received_bytes": r[4], "total_bytes": r[5],
        "mime_type": r[6], "state": r[7]
    } for r in cursor.fetchall()]

    # Keyword searches
    cursor.execute("""
        SELECT k.term, u.url, k.url_id
        FROM keyword_search_terms k JOIN urls u ON k.url_id = u.id
        ORDER BY u.last_visit_time DESC LIMIT 1000
    """)
    searches = [{"term": r[0], "url": r[1]} for r in cursor.fetchall()]

    conn.close()

    report = {
        "analysis_timestamp": datetime.now().isoformat(),
        "profile_path": profile_path,
        "total_visits": len(visits),
        "total_downloads": len(downloads),
        "total_searches": len(searches),
        "visits": visits,
        "downloads": downloads,
        "searches": searches
    }

    report_path = os.path.join(output_dir, "browser_forensics.json")
    with open(report_path, "w") as f:
        json.dump(report, f, indent=2)

    return report


def main():
    if len(sys.argv) < 3:
        print("Usage: python process.py <chrome_profile_path> <output_dir>")
        sys.exit(1)
    analyze_chrome_history(sys.argv[1], sys.argv[2])


if __name__ == "__main__":
    main()
python
import sqlite3
import os
import json
import sys
from datetime import datetime, timedelta


CHROME_EPOCH = datetime(1601, 1, 1)


def chrome_time_to_datetime(chrome_ts: int):
    """Convert Chrome timestamp to datetime."""
    if chrome_ts == 0:
        return None
    try:
        return CHROME_EPOCH + timedelta(microseconds=chrome_ts)
    except (OverflowError, OSError):
        return None


def analyze_chrome_history(profile_path: str, output_dir: str) -> dict:
    """Analyze Chrome History database for forensic evidence."""
    history_db = os.path.join(profile_path, "History")
    if not os.path.exists(history_db):
        return {"error": "History database not found"}

    os.makedirs(output_dir, exist_ok=True)
    conn = sqlite3.connect(f"file:{history_db}?mode=ro", uri=True)

    # URL visits with timestamps
    cursor = conn.cursor()
    cursor.execute("""
        SELECT u.url, u.title, v.visit_time, u.visit_count,
               v.transition & 0xFF as transition_type
        FROM visits v JOIN urls u ON v.url = u.id
        ORDER BY v.visit_time DESC LIMIT 5000
    """)
    visits = [{
        "url": r[0], "title": r[1],
        "visit_time": str(chrome_time_to_datetime(r[2])),
        "total_visits": r[3], "transition": r[4]
    } for r in cursor.fetchall()]

    # Downloads
    cursor.execute("""
        SELECT target_path, tab_url, start_time, end_time,
               received_bytes, total_bytes, mime_type, state
        FROM downloads ORDER BY start_time DESC LIMIT 1000
    """)
    downloads = [{
        "path": r[0], "source_url": r[1],
        "start_time": str(chrome_time_to_datetime(r[2])),
        "end_time": str(chrome_time_to_datetime(r[3])),
        "received_bytes": r[4], "total_bytes": r[5],
        "mime_type": r[6], "state": r[7]
    } for r in cursor.fetchall()]

    # Keyword searches
    cursor.execute("""
        SELECT k.term, u.url, k.url_id
        FROM keyword_search_terms k JOIN urls u ON k.url_id = u.id
        ORDER BY u.last_visit_time DESC LIMIT 1000
    """)
    searches = [{"term": r[0], "url": r[1]} for r in cursor.fetchall()]

    conn.close()

    report = {
        "analysis_timestamp": datetime.now().isoformat(),
        "profile_path": profile_path,
        "total_visits": len(visits),
        "total_downloads": len(downloads),
        "total_searches": len(searches),
        "visits": visits,
        "downloads": downloads,
        "searches": searches
    }

    report_path = os.path.join(output_dir, "browser_forensics.json")
    with open(report_path, "w") as f:
        json.dump(report, f, indent=2)

    return report


def main():
    if len(sys.argv) < 3:
        print("Usage: python process.py <chrome_profile_path> <output_dir>")
        sys.exit(1)
    analyze_chrome_history(sys.argv[1], sys.argv[2])


if __name__ == "__main__":
    main()

References

参考资料

Example Output

示例输出

text
$ python hindsight.py -i /evidence/chrome-profile -o /analysis/hindsight_output

Hindsight v2024.01 - Chrome/Chromium Browser Forensic Analysis
================================================================

Profile: /evidence/chrome-profile (Chrome 120.0.6099.130)
OS: Windows 10

[+] Parsing History database...
    URL records:          12,456
    Download records:     234
    Search terms:         567

[+] Parsing Cookies database...
    Cookie records:       8,923
    Encrypted cookies:    6,712

[+] Parsing Web Data (Autofill)...
    Autofill entries:     1,234
    Credit card entries:  2 (encrypted)

[+] Parsing Login Data...
    Saved credentials:    45 (encrypted)

[+] Parsing Bookmarks...
    Bookmark entries:     189

--- Browsing History (Last 10 Entries) ---
Timestamp (UTC)          | URL                                          | Title                        | Visit Count
2024-01-15 14:32:05.123  | https://mail.corporate.com/inbox             | Corporate Mail                | 45
2024-01-15 14:33:12.456  | https://drive.google.com/file/d/1aBcDe...    | Q4_Financial_Report.xlsx     | 1
2024-01-15 14:35:44.789  | https://mega.nz/folder/xYz123               | MEGA - Secure Cloud          | 3
2024-01-15 14:36:01.234  | https://mega.nz/folder/xYz123#upload        | MEGA - Upload                | 8
2024-01-15 14:42:15.567  | https://pastebin.com/raw/kL9mN2pQ           | Pastebin (raw)               | 1
2024-01-15 15:01:33.890  | https://192.168.1.50:8443/admin              | Admin Panel                  | 12
2024-01-15 15:15:22.111  | https://transfer.sh/upload                  | transfer.sh                  | 2
2024-01-15 15:30:45.222  | https://vpn-gateway.corporate.com            | VPN Login                    | 5
2024-01-15 16:00:00.333  | https://whatismyipaddress.com                 | What Is My IP                | 1
2024-01-15 16:05:12.444  | https://protonmail.com/inbox                 | ProtonMail                   | 3

--- Downloads (Suspicious) ---
Timestamp (UTC)          | Filename                    | URL Source                               | Size
2024-01-15 14:33:15.000  | Q4_Financial_Report.xlsm   | https://phish-domain.com/docs/report     | 245 KB
2024-01-15 14:34:02.000  | update_client.exe          | https://cdn.evil-updates.com/client.exe  | 1.2 MB

--- Cookies (Session Tokens) ---
Domain                   | Name              | Expires            | Secure | HttpOnly
.corporate.com           | SESSION_ID        | 2024-01-16 14:32   | Yes    | Yes
.mega.nz                 | session           | Session            | Yes    | Yes
.protonmail.com          | AUTH-TOKEN        | 2024-02-15 00:00   | Yes    | Yes

Report saved to: /analysis/hindsight_output/Hindsight_Report.xlsx
text
$ python hindsight.py -i /evidence/chrome-profile -o /analysis/hindsight_output

Hindsight v2024.01 - Chrome/Chromium Browser Forensic Analysis
================================================================

Profile: /evidence/chrome-profile (Chrome 120.0.6099.130)
OS: Windows 10

[+] Parsing History database...
    URL records:          12,456
    Download records:     234
    Search terms:         567

[+] Parsing Cookies database...
    Cookie records:       8,923
    Encrypted cookies:    6,712

[+] Parsing Web Data (Autofill)...
    Autofill entries:     1,234
    Credit card entries:  2 (encrypted)

[+] Parsing Login Data...
    Saved credentials:    45 (encrypted)

[+] Parsing Bookmarks...
    Bookmark entries:     189

--- Browsing History (Last 10 Entries) ---
Timestamp (UTC)          | URL                                          | Title                        | Visit Count
2024-01-15 14:32:05.123  | https://mail.corporate.com/inbox             | Corporate Mail                | 45
2024-01-15 14:33:12.456  | https://drive.google.com/file/d/1aBcDe...    | Q4_Financial_Report.xlsx     | 1
2024-01-15 14:35:44.789  | https://mega.nz/folder/xYz123               | MEGA - Secure Cloud          | 3
2024-01-15 14:36:01.234  | https://mega.nz/folder/xYz123#upload        | MEGA - Upload                | 8
2024-01-15 14:42:15.567  | https://pastebin.com/raw/kL9mN2pQ           | Pastebin (raw)               | 1
2024-01-15 15:01:33.890  | https://192.168.1.50:8443/admin              | Admin Panel                  | 12
2024-01-15 15:15:22.111  | https://transfer.sh/upload                  | transfer.sh                  | 2
2024-01-15 15:30:45.222  | https://vpn-gateway.corporate.com            | VPN Login                    | 5
2024-01-15 16:00:00.333  | https://whatismyipaddress.com                 | What Is My IP                | 1
2024-01-15 16:05:12.444  | https://protonmail.com/inbox                 | ProtonMail                   | 3

--- Downloads (Suspicious) ---
Timestamp (UTC)          | Filename                    | URL Source                               | Size
2024-01-15 14:33:15.000  | Q4_Financial_Report.xlsm   | https://phish-domain.com/docs/report     | 245 KB
2024-01-15 14:34:02.000  | update_client.exe          | https://cdn.evil-updates.com/client.exe  | 1.2 MB

--- Cookies (Session Tokens) ---
Domain                   | Name              | Expires            | Secure | HttpOnly
.corporate.com           | SESSION_ID        | 2024-01-16 14:32   | Yes    | Yes
.mega.nz                 | session           | Session            | Yes    | Yes
.protonmail.com          | AUTH-TOKEN        | 2024-02-15 00:00   | Yes    | Yes

Report saved to: /analysis/hindsight_output/Hindsight_Report.xlsx