testing-for-sensitive-data-exposure

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Testing for Sensitive Data Exposure

敏感数据暴露测试

When to Use

适用场景

  • During authorized penetration tests when assessing data protection controls
  • When evaluating applications for GDPR, PCI DSS, HIPAA, or other data protection compliance
  • For identifying leaked API keys, credentials, tokens, and secrets in application responses
  • When testing whether sensitive data is properly encrypted in transit and at rest
  • During security assessments of APIs that handle PII, financial data, or health records
  • 在授权的渗透测试中评估数据保护控制措施时
  • 评估应用是否符合GDPR、PCI DSS、HIPAA或其他数据保护合规要求时
  • 识别应用响应中泄露的API密钥、凭证、令牌和机密信息时
  • 测试敏感数据在传输和存储过程中是否已正确加密时
  • 处理个人可识别信息(PII)、财务数据或健康记录的API安全评估时

Prerequisites

前置条件

  • Authorization: Written penetration testing agreement with data handling scope
  • Burp Suite Professional: For intercepting and analyzing responses for sensitive data
  • trufflehog: Secret scanning tool (
    pip install trufflehog
    )
  • gitleaks: Git repository secret scanner (
    go install github.com/gitleaks/gitleaks/v8@latest
    )
  • curl/httpie: For manual endpoint testing
  • Browser DevTools: For examining local storage, session storage, and cached data
  • testssl.sh: TLS configuration testing tool
  • 授权:包含数据处理范围的书面渗透测试协议
  • Burp Suite Professional:用于拦截和分析包含敏感数据的响应
  • trufflehog:密钥扫描工具(
    pip install trufflehog
  • gitleaks:Git仓库密钥扫描工具(
    go install github.com/gitleaks/gitleaks/v8@latest
  • curl/httpie:用于手动端点测试
  • Browser DevTools:用于检查本地存储、会话存储和缓存数据
  • testssl.sh:TLS配置测试工具

Workflow

工作流程

Step 1: Scan for Secrets in Client-Side Code

步骤1:扫描客户端代码中的密钥

Search JavaScript files, HTML source, and other client-side resources for exposed secrets.
bash
undefined
在JavaScript文件、HTML源码和其他客户端资源中搜索暴露的密钥。
bash
undefined

Download and search JavaScript files for secrets

Download and search JavaScript files for secrets

curl -s "https://target.example.com/" |
grep -oP 'src="[^"].js[^"]"' |
grep -oP '"[^"]"' | tr -d '"' | while read js; do echo "=== Scanning: $js ===" # Handle relative URLs if [[ "$js" == / ]]; then curl -s "https://target.example.com$js" else curl -s "$js" fi | grep -inE
"(api[-]?key|apikey|api[-]?secret|aws[-]?access|aws[-]?secret|private[_-]?key|password|secret|token|auth|credential|AKIA[0-9A-Z]{16})"
| head -20 done
curl -s "https://target.example.com/" |
grep -oP 'src="[^"].js[^"]"' |
grep -oP '"[^"]"' | tr -d '"' | while read js; do echo "=== Scanning: $js ===" # Handle relative URLs if [[ "$js" == / ]]; then curl -s "https://target.example.com$js" else curl -s "$js" fi | grep -inE
"(api[-]?key|apikey|api[-]?secret|aws[-]?access|aws[-]?secret|private[_-]?key|password|secret|token|auth|credential|AKIA[0-9A-Z]{16})"
| head -20 done

Search for common secret patterns

Search for common secret patterns

curl -s "https://target.example.com/static/app.js" | grep -nP
"(AIza[0-9A-Za-z-]{35}|AKIA[0-9A-Z]{16}|sk-[a-zA-Z0-9]{48}|ghp[a-zA-Z0-9]{36}|xox[bpsa]-[0-9a-zA-Z-]{10,})"
curl -s "https://target.example.com/static/app.js" | grep -nP
"(AIza[0-9A-Za-z-]{35}|AKIA[0-9A-Z]{16}|sk-[a-zA-Z0-9]{48}|ghp[a-zA-Z0-9]{36}|xox[bpsa]-[0-9a-zA-Z-]{10,})"

Check source maps for exposed source code

Check source maps for exposed source code

Source maps may contain original source code with embedded secrets

Source maps may contain original source code with embedded secrets

Search HTML source for exposed data

Search HTML source for exposed data

curl -s "https://target.example.com/" | grep -inE
"(api_key|secret|password|token|private_key|database_url|smtp_password)" | head -20
curl -s "https://target.example.com/" | grep -inE
"(api_key|secret|password|token|private_key|database_url|smtp_password)" | head -20

Check for exposed .env or configuration files

Check for exposed .env or configuration files

for file in .env .env.local .env.production config.json settings.json
.aws/credentials .docker/config.json; do status=$(curl -s -o /dev/null -w "%{http_code}"
"https://target.example.com/$file") if [ "$status" == "200" ]; then echo "FOUND: $file ($status)" fi done
undefined
for file in .env .env.local .env.production config.json settings.json
.aws/credentials .docker/config.json; do status=$(curl -s -o /dev/null -w "%{http_code}"
"https://target.example.com/$file") if [ "$status" == "200" ]; then echo "FOUND: $file ($status)" fi done
undefined

Step 2: Analyze API Responses for Data Over-Exposure

步骤2:分析API响应中的数据过度暴露问题

Check if API endpoints return more data than necessary.
bash
undefined
检查API端点是否返回了超出必要范围的数据。
bash
undefined

Fetch user profile and examine response fields

Fetch user profile and examine response fields

curl -s -H "Authorization: Bearer $TOKEN"
"https://target.example.com/api/users/me" | jq .
curl -s -H "Authorization: Bearer $TOKEN"
"https://target.example.com/api/users/me" | jq .

Look for sensitive fields that should not be exposed:

Look for sensitive fields that should not be exposed:

- password, password_hash, password_salt

- password, password_hash, password_salt

- ssn, social_security_number, national_id

- ssn, social_security_number, national_id

- credit_card_number, card_cvv, card_expiry

- credit_card_number, card_cvv, card_expiry

- api_key, secret_key, access_token, refresh_token

- api_key, secret_key, access_token, refresh_token

- internal_id, database_id

- internal_id, database_id

- ip_address, session_id

- ip_address, session_id

- date_of_birth, drivers_license

- date_of_birth, drivers_license

Check list endpoints for excessive data

Check list endpoints for excessive data

curl -s -H "Authorization: Bearer $TOKEN"
"https://target.example.com/api/users" | jq '.[0] | keys'
curl -s -H "Authorization: Bearer $TOKEN"
"https://target.example.com/api/users" | jq '.[0] | keys'

Compare public vs authenticated responses

Compare public vs authenticated responses

echo "=== Public ===" curl -s "https://target.example.com/api/users/1" | jq 'keys' echo "=== Authenticated ===" curl -s -H "Authorization: Bearer $TOKEN"
"https://target.example.com/api/users/1" | jq 'keys'
echo "=== Public ===" curl -s "https://target.example.com/api/users/1" | jq 'keys' echo "=== Authenticated ===" curl -s -H "Authorization: Bearer $TOKEN"
"https://target.example.com/api/users/1" | jq 'keys'

Check error responses for information leakage

Check error responses for information leakage

curl -s -X POST
-H "Content-Type: application/json"
-d '{"invalid": "data"}'
"https://target.example.com/api/users" | jq .
curl -s -X POST
-H "Content-Type: application/json"
-d '{"invalid": "data"}'
"https://target.example.com/api/users" | jq .

Look for: stack traces, database queries, internal paths, version info

Look for: stack traces, database queries, internal paths, version info

Test for PII in search/autocomplete responses

Test for PII in search/autocomplete responses

curl -s -H "Authorization: Bearer $TOKEN"
"https://target.example.com/api/search?q=john" | jq .
curl -s -H "Authorization: Bearer $TOKEN"
"https://target.example.com/api/search?q=john" | jq .

May return full user records instead of just names

May return full user records instead of just names

undefined
undefined

Step 3: Test Data Transmission Security

步骤3:测试数据传输安全性

Verify that sensitive data is encrypted during transmission.
bash
undefined
验证敏感数据在传输过程中是否已加密。
bash
undefined

Check TLS configuration

Check TLS configuration

Using testssl.sh

Using testssl.sh

Quick TLS checks with curl

Quick TLS checks with curl

curl -s -v "https://target.example.com/" 2>&1 | grep -E "(SSL|TLS|cipher|subject)"
curl -s -v "https://target.example.com/" 2>&1 | grep -E "(SSL|TLS|cipher|subject)"

Check for HTTP (non-HTTPS) endpoints

Check for HTTP (non-HTTPS) endpoints

curl -s -I "http://target.example.com/" | head -5
curl -s -I "http://target.example.com/" | head -5

Should redirect to HTTPS

Should redirect to HTTPS

Check for mixed content (HTTP resources on HTTPS pages)

Check for mixed content (HTTP resources on HTTPS pages)

curl -s "https://target.example.com/" | grep -oP "http://[^"'> ]+" | head -20
curl -s "https://target.example.com/" | grep -oP "http://[^"'> ]+" | head -20

Check if sensitive forms submit over HTTPS

Check if sensitive forms submit over HTTPS

curl -s "https://target.example.com/login" | grep -oP 'action="[^"]*"'
curl -s "https://target.example.com/login" | grep -oP 'action="[^"]*"'

Form action should use HTTPS

Form action should use HTTPS

Check for sensitive data in URL parameters (query string)

Check for sensitive data in URL parameters (query string)

URLs are logged in browser history, server logs, proxy logs, Referer headers

URLs are logged in browser history, server logs, proxy logs, Referer headers

Look for: /login?username=admin&password=secret

Look for: /login?username=admin&password=secret

/api/data?ssn=123-45-6789

/api/data?ssn=123-45-6789

/search?credit_card=4111111111111111

/search?credit_card=4111111111111111

Check WebSocket encryption

Check WebSocket encryption

curl -s "https://target.example.com/" | grep -oP "(ws|wss)://[^"'> ]+"
curl -s "https://target.example.com/" | grep -oP "(ws|wss)://[^"'> ]+"

ws:// is unencrypted; should only use wss://

ws:// is unencrypted; should only use wss://

undefined
undefined

Step 4: Examine Browser Storage for Sensitive Data

步骤4:检查浏览器存储中的敏感数据

Check local storage, session storage, cookies, and cached responses.
bash
undefined
检查本地存储、会话存储、Cookie和缓存响应。
bash
undefined

Check what cookies are set and their security attributes

Check what cookies are set and their security attributes

curl -s -I "https://target.example.com/login" | grep -i "set-cookie"
curl -s -I "https://target.example.com/login" | grep -i "set-cookie"

In browser DevTools (Application tab):

In browser DevTools (Application tab):

1. Local Storage: Check for stored tokens, PII, credentials

1. Local Storage: Check for stored tokens, PII, credentials

2. Session Storage: Check for temporary sensitive data

2. Session Storage: Check for temporary sensitive data

3. IndexedDB: Check for cached application data

3. IndexedDB: Check for cached application data

4. Cache Storage: Check for cached API responses containing PII

4. Cache Storage: Check for cached API responses containing PII

5. Cookies: Check for sensitive data in cookie values

5. Cookies: Check for sensitive data in cookie values

Common insecure storage patterns:

Common insecure storage patterns:

localStorage.setItem('access_token', 'eyJ...'); // XSS can steal

localStorage.setItem('access_token', 'eyJ...'); // XSS can steal

localStorage.setItem('user', JSON.stringify({email: '...', ssn: '...'}));

localStorage.setItem('user', JSON.stringify({email: '...', ssn: '...'}));

sessionStorage.setItem('credit_card', '4111...');

sessionStorage.setItem('credit_card', '4111...');

Check for autocomplete on sensitive forms

Check for autocomplete on sensitive forms

curl -s "https://target.example.com/login" |
grep -oP '<input[^>](password|credit|ssn|card)[^>]>' |
grep -v 'autocomplete="off"'
curl -s "https://target.example.com/login" |
grep -oP '<input[^>](password|credit|ssn|card)[^>]>' |
grep -v 'autocomplete="off"'

Password and credit card fields should have autocomplete="off"

Password and credit card fields should have autocomplete="off"

Check Cache-Control headers on sensitive pages

Check Cache-Control headers on sensitive pages

for page in /account/profile /api/users/me /transactions /billing; do echo -n "$page: " curl -s -I "https://target.example.com$page"
-H "Authorization: Bearer $TOKEN" |
grep -i "cache-control" | tr -d '\r' echo done
for page in /account/profile /api/users/me /transactions /billing; do echo -n "$page: " curl -s -I "https://target.example.com$page"
-H "Authorization: Bearer $TOKEN" |
grep -i "cache-control" | tr -d '\r' echo done

Sensitive pages should have: Cache-Control: no-store

Sensitive pages should have: Cache-Control: no-store

undefined
undefined

Step 5: Scan Git Repositories and Source Code for Secrets

步骤5:扫描Git仓库和源代码中的密钥

Search for accidentally committed secrets in version control.
bash
undefined
搜索版本控制中意外提交的密钥。
bash
undefined

Check for exposed .git directory

Check for exposed .git directory

If .git is exposed, use git-dumper to download

If .git is exposed, use git-dumper to download

pip install git-dumper

pip install git-dumper

git-dumper https://target.example.com/.git /tmp/target-repo
git-dumper https://target.example.com/.git /tmp/target-repo

Scan downloaded repository with trufflehog

Scan downloaded repository with trufflehog

trufflehog filesystem /tmp/target-repo
trufflehog filesystem /tmp/target-repo

Scan with gitleaks

Scan with gitleaks

gitleaks detect --source /tmp/target-repo -v
gitleaks detect --source /tmp/target-repo -v

If GitHub/GitLab repository is available (authorized scope)

If GitHub/GitLab repository is available (authorized scope)

trufflehog github --org target-organization --token $GITHUB_TOKEN gitleaks detect --source https://github.com/org/repo -v
trufflehog github --org target-organization --token $GITHUB_TOKEN gitleaks detect --source https://github.com/org/repo -v

Common secrets found in repositories:

Common secrets found in repositories:

- AWS access keys (AKIA...)

- AWS access keys (AKIA...)

- Database connection strings

- Database connection strings

- API keys (Google, Stripe, Twilio, SendGrid)

- API keys (Google, Stripe, Twilio, SendGrid)

- Private SSH keys

- Private SSH keys

- JWT signing secrets

- JWT signing secrets

- OAuth client secrets

- OAuth client secrets

- SMTP credentials

- SMTP credentials

Search for secrets in Docker images

Search for secrets in Docker images

docker save target-image:latest | tar x -C /tmp/docker-layers

docker save target-image:latest | tar x -C /tmp/docker-layers

Search each layer for credentials

Search each layer for credentials

undefined
undefined

Step 6: Test Data Masking and Redaction

步骤6:测试数据掩码与脱敏

Verify that sensitive data is properly masked in the application.
bash
undefined
验证应用中敏感数据是否已正确掩码。
bash
undefined

Check if credit card numbers are fully displayed

Check if credit card numbers are fully displayed

curl -s -H "Authorization: Bearer $TOKEN"
"https://target.example.com/api/payment-methods" | jq .
curl -s -H "Authorization: Bearer $TOKEN"
"https://target.example.com/api/payment-methods" | jq .

Should show: **** **** **** 4242, not full number

Should show: **** **** **** 4242, not full number

Check if SSN/national ID is masked

Check if SSN/national ID is masked

curl -s -H "Authorization: Bearer $TOKEN"
"https://target.example.com/api/users/me" | jq '.ssn'
curl -s -H "Authorization: Bearer $TOKEN"
"https://target.example.com/api/users/me" | jq '.ssn'

Should show: *--6789, not full SSN

Should show: *--6789, not full SSN

Check API responses for password hashes

Check API responses for password hashes

curl -s -H "Authorization: Bearer $TOKEN"
"https://target.example.com/api/users" | jq '.[].password // empty'
curl -s -H "Authorization: Bearer $TOKEN"
"https://target.example.com/api/users" | jq '.[].password // empty'

Should return nothing; password hashes should never be in API responses

Should return nothing; password hashes should never be in API responses

Check export/download features for unmasked data

Check export/download features for unmasked data

curl -s -H "Authorization: Bearer $TOKEN"
"https://target.example.com/api/users/export?format=csv" | head -5
curl -s -H "Authorization: Bearer $TOKEN"
"https://target.example.com/api/users/export?format=csv" | head -5

CSV exports often contain unmasked PII

CSV exports often contain unmasked PII

Check logging endpoints for sensitive data

Check logging endpoints for sensitive data

curl -s -H "Authorization: Bearer $TOKEN"
"https://target.example.com/api/admin/logs" |
grep -iE "(password|token|secret|credit_card|ssn)" | head -10
curl -s -H "Authorization: Bearer $TOKEN"
"https://target.example.com/api/admin/logs" |
grep -iE "(password|token|secret|credit_card|ssn)" | head -10

Logs should not contain sensitive data in plaintext

Logs should not contain sensitive data in plaintext

Test for sensitive data in error messages

Test for sensitive data in error messages

curl -s -X POST
-H "Content-Type: application/json"
-d '{"email":"duplicate@test.com"}'
"https://target.example.com/api/register"
curl -s -X POST
-H "Content-Type: application/json"
-d '{"email":"duplicate@test.com"}'
"https://target.example.com/api/register"

Should not reveal: "User with email duplicate@test.com already exists"

Should not reveal: "User with email duplicate@test.com already exists"

Should show: "Registration failed" (generic)

Should show: "Registration failed" (generic)

undefined
undefined

Key Concepts

核心概念

ConceptDescription
Sensitive Data ExposureUnintended disclosure of PII, credentials, financial data, or health records
Data Over-ExposureAPI returning more data fields than the client needs
Secret LeakageAPI keys, tokens, or credentials exposed in client-side code or logs
Data at RestSensitive data stored in databases, files, or backups without encryption
Data in TransitSensitive data transmitted over network without TLS encryption
Data MaskingReplacing sensitive data with redacted values (e.g., showing last 4 digits of credit card)
PIIPersonally Identifiable Information - data that can identify an individual
Information LeakageExcessive error messages, stack traces, or debug information in responses
概念描述
敏感数据暴露意外泄露个人可识别信息(PII)、凭证、财务数据或健康记录
数据过度暴露API返回了客户端不需要的额外数据字段
密钥泄露API密钥、令牌或凭证在客户端代码或日志中暴露
静态数据存储在数据库、文件或备份中且未加密的敏感数据
传输中数据通过网络传输且未使用TLS加密的敏感数据
数据掩码用脱敏值替换敏感数据(例如,仅显示信用卡的最后4位数字)
PII个人可识别信息——可用于识别个人身份的数据
信息泄露响应中包含过多错误信息、堆栈跟踪或调试信息

Tools & Systems

工具与系统

ToolPurpose
Burp Suite ProfessionalResponse analysis and regex-based sensitive data scanning
trufflehogSecret detection across git repos, filesystems, and cloud storage
gitleaksGit repository scanning for hardcoded secrets
testssl.shTLS/SSL configuration assessment
git-dumperDownloading exposed .git directories from web servers
SecretFinderJavaScript file analysis for exposed API keys and tokens
Retire.jsDetecting JavaScript libraries with known vulnerabilities
工具用途
Burp Suite Professional响应分析和基于正则表达式的敏感数据扫描
trufflehog在Git仓库、文件系统和云存储中检测密钥
gitleaks扫描Git仓库中的硬编码密钥
testssl.shTLS/SSL配置评估
git-dumper从Web服务器下载暴露的.git目录
SecretFinder分析JavaScript文件以查找暴露的API密钥和令牌
Retire.js检测存在已知漏洞的JavaScript库

Common Scenarios

常见场景

Scenario 1: API Key in JavaScript Bundle

场景1:JavaScript包中的API密钥

The application's JavaScript bundle contains a hardcoded Google Maps API key and a Stripe publishable key. The Stripe key has overly broad permissions, allowing the attacker to create charges.
应用的JavaScript包中包含硬编码的Google Maps API密钥和Stripe发布密钥。该Stripe密钥权限过宽,允许攻击者创建收费订单。

Scenario 2: User API Returns Password Hashes

场景2:用户API返回密码哈希

The
/api/users
endpoint returns complete user objects including bcrypt password hashes. Attackers can extract hashes and attempt offline cracking.
/api/users
端点返回完整的用户对象,包括bcrypt密码哈希。攻击者可以提取哈希值并尝试离线破解。

Scenario 3: PII in Cached API Responses

场景3:缓存API响应中的PII

The user profile API endpoint returns full SSN and credit card numbers without masking. The endpoint does not set
Cache-Control: no-store
, so responses are cached in the browser and proxy caches.
用户资料API端点返回完整的社保号码(SSN)和信用卡号,且未进行掩码。该端点未设置
Cache-Control: no-store
,因此响应会被浏览器和代理缓存。

Scenario 4: Git Repository with Database Credentials

场景4:包含数据库凭证的Git仓库

The
.git
directory is accessible on the production server. Using git-dumper, the attacker downloads the repository history, finding database credentials committed in an early commit that were later "removed" but remain in git history.
生产服务器上的.git目录可被访问。攻击者使用git-dumper下载仓库历史记录,发现在早期提交中包含已被“删除”但仍保留在Git历史中的数据库凭证。

Output Format

输出格式

undefined
undefined

Sensitive Data Exposure Assessment Report

敏感数据暴露评估报告

Target: target.example.com Assessment Date: 2024-01-15 OWASP Category: A02:2021 - Cryptographic Failures
目标: target.example.com 评估日期: 2024-01-15 OWASP分类: A02:2021 - 加密失败

Findings Summary

发现摘要

FindingSeverityData Type
API keys in JavaScript sourceHighCredentials
Password hashes in API responseCriticalAuthentication
Unmasked SSN in user profileCriticalPII
Credit card number in exportHighFinancial
.git directory exposedCriticalSource code + secrets
Missing TLS on API endpointHighAll data in transit
Sensitive data in error messagesMediumTechnical info
发现项严重程度数据类型
JavaScript源码中的API密钥凭证
API响应中的密码哈希关键身份验证
用户资料中的未掩码SSN关键PII
导出文件中的信用卡号财务数据
暴露的.git目录关键源代码+密钥
API端点未启用TLS所有传输中数据
错误消息中的信息泄露技术信息

Critical: Exposed Secrets

关键问题:暴露的密钥

Secret TypeLocationRisk
AWS Access Key (AKIA...)/static/app.js line 342AWS resource access
Stripe Secret Key (sk_live_...).env (via .git exposure)Payment processing
Database URL with credentials.git history commit abc123Database access
JWT Signing Secretconfig.json (via .git)Token forgery
密钥类型位置风险
AWS访问密钥(AKIA...)/static/app.js 第342行AWS资源访问权限
Stripe密钥(sk_live_...).env文件(通过暴露的.git目录获取)支付处理权限
包含凭证的数据库URLGit历史提交abc123数据库访问权限
JWT签名密钥config.json(通过暴露的.git目录获取)令牌伪造

Data Over-Exposure in APIs

API中的数据过度暴露

EndpointUnnecessary Fields Returned
GET /api/userspassword_hash, internal_id, created_ip
GET /api/users/{id}ssn, credit_card_full, date_of_birth
GET /api/orderscustomer_phone, customer_address
端点返回的不必要字段
GET /api/userspassword_hash, internal_id, created_ip
GET /api/users/{id}ssn, credit_card_full, date_of_birth
GET /api/orderscustomer_phone, customer_address

Recommendation

建议

  1. Remove all hardcoded secrets from client-side code; use backend proxies
  2. Rotate all exposed credentials immediately
  3. Remove .git directory from production web root
  4. Implement response field filtering; return only required fields
  5. Mask sensitive data (SSN, credit card) in all API responses
  6. Add Cache-Control: no-store to all sensitive endpoints
  7. Enable TLS 1.2+ on all endpoints; redirect HTTP to HTTPS
  8. Implement secret scanning in CI/CD pipeline (trufflehog/gitleaks)
undefined
  1. 从客户端代码中移除所有硬编码密钥;使用后端代理
  2. 立即轮换所有暴露的凭证
  3. 从生产Web根目录中删除.git目录
  4. 实现响应字段过滤;仅返回必要字段
  5. 在所有API响应中对敏感数据(SSN、信用卡)进行掩码处理
  6. 为所有敏感端点添加Cache-Control: no-store头
  7. 在所有端点启用TLS 1.2+;将HTTP重定向到HTTPS
  8. 在CI/CD流水线中实现密钥扫描(使用trufflehog/gitleaks)
undefined