secrets-in-git-history

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Secrets in git history

Git 历史中的 Secrets

A repository is three intelligence products: a list of humans with real email addresses, a map of the organisation's infrastructure, and a credential store nobody meant to publish. The costly mistake is scanning the working tree. Secrets deleted from HEAD stay in history forever, and commits deleted from a branch stay reachable through the fork network — scan only what is checked out and you are searching the one place the secret was definitely removed from.

一个 repository 本身就是三项情报产品：一份带真实 email address 的人员名单、一张组织 infrastructure 地图，以及一个无人打算公开的 credential store。代价高昂的错误是只扫描 working tree。已从 HEAD 删除的 secrets 会永远留在 history 中，而从 branch 删除的 commits 仍可通过 fork network 访问——如果你只扫描已 checkout 的内容，你搜索的恰恰是 secret 确定已被移除的那个地方。

What you're holding, and where to start

你掌握了什么，以及从哪里开始

You have	Start with	Why
An organisation name	Public org members, then their personal repos	Corporate repos are reviewed; personal ones are not
A developer's name or handle	Commit emails across their repos	Yields an email nothing else gives you
A commit email	Reverse-search across hosts; resolve any noreply ID	Links accounts across orgs and platforms
A suspected credential leak	Both scanners over a mirror clone, all refs	HEAD-only scanning misses the point entirely
A specific string — hostname, key prefix	`git log -S` pickaxe across all refs	Tells you when it entered and when it left
A live domain	Check for an exposed `.git` — see the legal note	Full history from a web server, if authorized
Nothing but a company website	Code search the org's internal domain names	Finds repos with no obvious link to the org

你掌握的信息	从哪里开始	原因
组织名称	公开的组织成员，再查他们的个人 repos	公司 repos 会接受审查，个人 repos 不会
开发者姓名或 handle	其各个 repo 中的 commit emails	获取其他方式无法得到的 email
commit email	跨 hosts 反向搜索；解析任何 noreply ID	将各组织和平台上的账号关联起来
疑似 credential 泄露	在 mirror clone 上运行两种 scanners，并扫描所有 refs	只扫描 HEAD 完全抓不到重点
特定字符串——hostname、key prefix	在所有 refs 上使用 `git log -S` pickaxe	确定它何时进入以及何时离开
在线 domain	检查是否暴露了 `.git` ——见法律说明	获得授权后，可从 web server 获取完整 history
只有公司网站	对该组织的内部 domain 名称进行 code search	找到与该组织没有明显关联的 repos

The commit email leak

Commit email 泄露

Git records an author identity on every commit, self-asserted rather than verified. That cuts both ways: emails are trivially forged, and they are also the most reliable public link between a code-host identity and a real mailbox.

bash

git clone --mirror https://github.com/example-org/example-repo.git repo.git
git --git-dir=repo.git log --all --format='%an <%ae>' | sort -u

GitHub's web UI does not display commit emails, which leads people to assume they are hidden. They are not. Append

.patch

to any commit URL for a mail-format patch whose

From:

header carries the author name and email; the API returns the same under

commit.author.email

bash

curl -s https://github.com/example-org/example-repo/commit/<sha>.patch | head -5
curl -s https://api.github.com/repos/example-org/example-repo/commits/<sha> | jq .commit.author

The noreply pattern is the part worth understanding properly. Users with email privacy enabled commit as

username@users.noreply.github.com

, or in the newer form

12345678+username@users.noreply.github.com

. That second form leaks the account's numeric user ID, which is immutable and survives every rename the account ever makes, so an old commit resolves to whoever holds it today:

bash

curl -s https://api.github.com/user/12345678 | jq '.login, .created_at'

That defeats a rename: an old commit under a long-abandoned handle points at the person's current identity. The complementary failure on the subject's side is enabling email privacy after years of committing — every commit before that date carries the real address, so pull the author list across all history.

Git 会记录每次 commit 的作者身份；这个身份是自我声明的，而非经过验证。这具有两面性：emails 很容易伪造，但它们也是代码托管平台身份与真实 mailbox 之间最可靠的公开关联。

bash

git clone --mirror https://github.com/example-org/example-repo.git repo.git
git --git-dir=repo.git log --all --format='%an <%ae>' | sort -u

GitHub 的 web UI 不显示 commit emails，这会让人误以为它们被隐藏了。事实并非如此。在任意 commit URL 后追加

.patch

，即可得到 mail-format patch，其

From:

header 会包含作者姓名和 email；API 返回的同一信息位于

commit.author.email

中。

bash

curl -s https://github.com/example-org/example-repo/commit/<sha>.patch | head -5
curl -s https://api.github.com/repos/example-org/example-repo/commits/<sha> | jq .commit.author

noreply 模式才是真正值得透彻理解的部分。 启用 email privacy 的用户会在 commit 中使用

username@users.noreply.github.com

，较新的形式则是

12345678+username@users.noreply.github.com

。第二种形式会泄露账号的 numeric user ID。该 ID 不可变，并且在账号经历任何更名后依然保留，因此旧 commit 可以解析到如今持有该账号的人：

bash

curl -s https://api.github.com/user/12345678 | jq '.login, .created_at'

这会让 rename 失去作用：旧 commit 中那个早已弃用的 handle，会指向该用户当前的 identity。反过来，调查对象也可能在多年 commit 之后才启用 email privacy——该日期之前的每次 commit 都带有真实 address，因此应拉取完整 history 中的作者列表。

Enumerating an organisation as a social graph

将组织枚举为一张 social graph

Corporate repositories are reviewed. Personal ones are not, and developers put the same infrastructure in both. The pivot is org to human to personal account.

Public org membership is opt-in and therefore incomplete, so triangulate:

https://api.github.com/orgs/<org>/public_members

gives the members who chose to be visible, while the author list across every org repository gives everyone who actually committed, including those who never went public and those who left.

From each human, pull repos, gists, followers, following, starred and watched repositories. Stars and follows are the underused ones: nobody curates them for privacy, and they map interests, employers, side projects and professional relationships more honestly than any profile field. A cluster of developers all starring the same obscure internal-looking tool are colleagues.

Public activity events cover a short recent window, but the public event stream is also published as a historical archive dataset, extending the analysis back years. Commit timestamps in aggregate give working hours and a timezone.

公司 repos 会经过审查，个人 repos 不会，而开发者往往会在两者中使用相同的基础设施。转换路径是 org → human → personal account。

Public org membership 采用 opt-in，因此并不完整，需要通过三角验证来补全：

https://api.github.com/orgs/<org>/public_members

会列出选择公开可见的 members，而每个 org repo 的 author list 则能给出所有实际提交过代码的人，包括从未公开 membership 的人以及已经离开的人。

对每个人，拉取 repos、gists、followers、following、starred 和 watched repositories。Stars 和 follows 是最少被利用的两类信息：没人会为了隐私特意整理这些内容，而且与任何 profile field 相比，它们更诚实地映射了人们的兴趣、雇主、副业项目和职业关系。一组开发者都 star 同一个冷门、看起来像内部工具的项目，他们很可能就是同事。

公开活动 events 只覆盖近期较短的时间窗口，但 public event stream 也以 historical archive dataset 的形式发布，可以将分析扩展到过去数年。汇总 commit timestamps 还可以推断工作时段和 timezone。

Code search and its real limits

Code search 及其真正的限制

Host code search finds strings across public repositories. Search what only this organisation would write: internal hostnames, private package names, cloud account IDs, the email domain. The company name finds marketing copy; the internal domain finds infrastructure.

The limits matter more than the syntax, because each is a silent gap. Code search covers the default branch only — no history, no other branches — so the deleted secret is invisible to it by construction. Not every repository is indexed: very large ones, and forks with no independent activity, are commonly excluded. Authentication is generally required and rate limits are tight. And matching is token-based, so a substring inside a longer token may not match the way you expect.

Treat code search as a discovery tool that tells you which repositories to clone. The actual work happens locally.

平台 code search 可以在 public repositories 中查找字符串。应搜索只有该组织才会写出的内容：internal hostnames、private package names、cloud account IDs、email domain。公司名称会找到营销文案；内部 domain 才能定位到 infrastructure。

这些限制比语法更重要，因为每一项都会造成无声缺口。Code search 仅覆盖 default branch——不包含 history，也不包含其他 branches——因此从机制上就看不到已删除的 secret。并非每个 repository 都会被索引：非常大的 repos，以及没有独立活动的 forks，通常会被排除。通常需要 authentication，rate limits 也很严格。而且匹配以 token 为基础，因此较长 token 中的 substring 未必会按预期匹配。

把 code search 当作发现工具，用来确定哪些 repositories 需要 clone。真正的工作发生在本地。

Everything that survives deletion

删除后仍会残留的一切

Secrets removed from HEAD remain in history. Deleting a file is a commit, not an erasure; rewriting history does not reach clones, forks or caches; and almost nobody rotates afterwards, because deleting felt like fixing.
Commits in a fork network stay reachable by SHA, even after a branch is deleted, a PR is closed unmerged, or the fork is removed. Making a repository private does not retract what reached forks, and force-pushed commits are dangling rather than gone.
Gists are unlisted, not private — anyone with the URL reads a "secret" gist, and URLs leak into logs, chats and tickets.
CI logs are public on public repositories. Masking of known secret values fails whenever a value is base64-encoded, concatenated, URL-encoded, or printed by a subprocess. Issue and PR attachments are likewise served by opaque URL and stay retrievable after the issue or repository is gone.

从 HEAD 移除的 secrets 仍留在 history 中。 删除 file 只是一次 commit，并不是擦除；重写 history 也触及不到已经存在的 clones、forks 或 caches；而且几乎没人会在事后 rotate，因为删除让人以为已经修复。
Fork network 中的 commits 仍可通过 SHA 访问，即便 branch 已删除、PR 已关闭且未合并，或 fork 已被移除。把 repository 设为 private 不会撤回已经进入 forks 的内容；force-pushed commits 只会变成 dangling，而不是彻底消失。
Gists 是 unlisted，而不是 private——任何拿到 URL 的人都能读取所谓的“secret” gist，而 URL 会泄漏进 logs、chats 和 tickets。
CI logs 在 public repositories 上是公开的。 对已知 secret values 的 masking，只要 value 经过 base64 编码、拼接、URL 编码，或由 subprocess 打印出来，就会失效。Issue 和 PR attachments 同样通过 opaque URL 提供，即使 issue 或 repository 已经消失，仍可检索到。

Scanning properly

正确执行扫描

Mirror-clone, then run both scanners — their detector sets and false-positive profiles differ and neither is a superset. Then pickaxe the strings they have no detector for: internal domains, cloud account IDs, hostnames from config files.

bash

trufflehog git file://./repo.git
gitleaks detect --source=. -v
git log --all --source -S'internal.example.com' --oneline
git log --all --diff-filter=D --name-only --format='%H %aI %an'

One warning about verification. TruffleHog can confirm a candidate is live by calling the provider's API with it. That is an interaction with a third party using someone else's credential, and it writes to their logs. Inside an authorized assessment it is normally expected. Outside one, do not.

Commands in reference/git-archaeology.md; what to grep for and how to triage a hit in reference/high-signal-indicators.md.

先做 mirror-clone，再运行两种 scanners——它们的 detector sets 和 false-positive profiles 各不相同，任何一方都不是另一方的超集。然后用 pickaxe 搜索它们没有 detector 可覆盖的 strings：内部 domains、cloud account IDs、config files 中的 hostnames。

bash

trufflehog git file://./repo.git
gitleaks detect --source=. -v
git log --all --source -S'internal.example.com' --oneline
git log --all --diff-filter=D --name-only --format='%H %aI %an'

关于 verification 有一条警告。 TruffleHog 可以调用 provider 的 API，确认候选 credential 是否仍然有效。这相当于使用他人的 credential 与第三方进行交互，并会在对方 logs 中留下记录。在获得授权的 assessment 内，这通常属于预期行为；在授权范围之外，则不要这样做。

相关命令见 reference/git-archaeology.md；要 grep 哪些内容以及如何对命中结果进行分诊，请参阅 reference/high-signal-indicators.md。

Exposed .git directories

暴露的 .git directories

A deployment that copies a working tree to a web root publishes

.git

with it, and from

.git/config

.git/HEAD

and the object store the entire history is reconstructible — source, credentials, author identities, everything.

Observing that such a path exists, through a search index or an existing scan result, is passive. Retrieving it is not. Downloading a

.git

directory is accessing a system and taking data from it, and no misconfiguration constitutes permission. Only inside a written authorization; otherwise, report it.

某次 deployment 把 working tree 复制到 web root 时，也会连同

.git

一起发布；从

.git/config

、

.git/HEAD

和 object store 中，可以重建完整 history——source、credentials、author identities，所有内容都在内。

通过 search index 或已有 scan result 观察到这类路径存在，属于 passive 行为。获取它则不是。 下载

.git

directory 意味着访问一个系统并从中获取数据，任何 misconfiguration 都不构成 permission。只有在书面授权范围内才可以执行；否则应进行 report。

Where this goes wrong

容易出错的地方

Author identity is self-asserted. Anyone can commit as anyone; an email in a commit is a claim, not proof of authorship. Signed commits are the exception and the signature is what you verify, not the string.
Bots dominate commit counts. CI, dependency bots and merge automation produce huge volumes of commits under synthetic identities. Filter them before concluding anything about people.
Rebases and squash merges rewrite authorship. The committer becomes whoever performed the merge, and squashing collapses many authors into one. Absence from the log is not absence from the project.
Timezone inference is soft. Laptops travel, offsets get set by hand, CI commits at UTC. Corroboration only.
Test fixtures and documentation examples look exactly like credentials, and provider examples propagate into thousands of repos. Search first.
Public-by-design keys are not leaks. Publishable and referrer-locked browser keys are meant to ship; reporting one as a breach costs you credibility on the findings that are real.
A namesake org or typosquat is not your target, and a history-only finding is often more live than a current one, because removal without rotation is the norm.

作者身份是自我声明的。 任何人都可以以任何人的身份 commit；commit 中的 email 是一项声明，而非作者身份证明。Signed commits 是例外，而应验证的是 signature，而不是字符串。
Bots 主导 commit 数量。 CI、dependency bots 和 merge automation 会以 synthetic identities 产生大量 commits。在对人员得出任何结论前，应先过滤它们。
Rebases 和 squash merges 会重写 authorship。 Committer 会变成执行 merge 的人，而 squash 会将许多 authors 合并为一个。日志中没有某人，并不表示此人没有参与项目。
Timezone 推断只能作为弱证据。 Laptops 会 travels，offsets 可以手动设置，CI commits 使用 UTC。需要 corroboration。
Test fixtures 和 documentation examples 看起来与 credentials 完全一样，而 provider examples 会传播到数千个 repos。应先搜索确认。
按设计公开的 keys 不是 leaks。 可发布以及受 referrer 限制的 browser keys 本来就要随产品发布；将其报告为 breach 会损害你在真实 findings 上的可信度。
名称相同的 org 或 typosquat 不是你的目标，而仅存在于 history 中的 finding 往往比当前 finding 更可能仍然有效，因为删除后不 rotate 是常态。

Confidence grading

置信度评级

Confirmed — an identity link where a noreply numeric ID resolves to the account, or the same commit email appears under multiple accounts already tied to the person. A credential exposure whose format is valid, whose scope the surrounding context establishes, with a commit SHA and date.
Probable — a distinctive email or handle recurring across repositories with consistent commit timing, or a credential-shaped string in a real config file outside a test path.
Unconfirmed — a single unsigned commit's author string; a common handle; a scanner hit in a fixture or documentation example; a credential of unknown scope.
Rejected — a documented placeholder, a publishable key, or an expired temporary credential presented as a live exposure.

Record repository, commit SHA, date and file path for every finding. A SHA is an immutable citation; a line number is not.

已确认（Confirmed） — 身份关联方面，noreply numeric ID 能解析到对应账号，或同一个 commit email 出现在多个已与该人关联的账号下；credential exposure 方面，格式有效、周边 context 能确定 scope，并且有 commit SHA 和 date。
较可能（Probable） — 具有辨识度的 email 或 handle 持续出现在多个 repos 中且 commit timing 一致，或者真实 config file（不在 test path 中）里存在形似 credential 的 string。
未确认（Unconfirmed） — 单条 unsigned commit 的 author string；常见 handle；scanner 在 fixture 或 documentation example 中的命中；scope 未知的 credential。
已排除（Rejected） — 有文档记录的 placeholder、可发布的 key，或被当作 live exposure 提交的 expired temporary credential。

为每项 finding 记录 repository、commit SHA、date 和 file path。SHA 是不可变的 citation；line number 不是。

Worked example

实例

Objective: map the engineering team of a company whose site names only executives.

Code search on the internal domain

svc.example-corp.net

returns three repositories, none carrying the company's name. Mirror-cloning them gives fourteen distinct author emails, eleven corporate and three personal.

One author commits as

48211903+dhaverford@users.noreply.github.com

. Resolving the numeric ID returns a different current username — the account was renamed, and the old handle links to conference talks and a personal blog the current one does not.

The dead end: gitleaks flags an AWS-shaped key in a test fixture, and searching the string finds it verbatim in AWS's own documentation. Discarded, and recorded as discarded so nobody re-raises it.

The real finding is a

terraform.tfstate

added and deleted in the same week two years ago, recovered from history. No live credential, but four internal hostnames and a cloud account ID — infrastructure for

find-hidden-subdomains

Grade: team roster probable, since commit authorship is self-asserted and uncorroborated. The rename link confirmed — the numeric ID is immutable.

目标：绘制一家公司的 engineering team 名单，而其官网只列出 executives。

对 internal domain

svc.example-corp.net

进行 code search 后返回三个 repositories，其中都不带公司的名称。Mirror-cloning 它们后得到 14 个不同的 author emails，其中 11 个 corporate，3 个 personal。

其中一位 author 使用

48211903+dhaverford@users.noreply.github.com

提交。解析 numeric ID 后得到的是另一个 current username——该 account 已 renamed，而 old handle 能关联到 current one 无法关联的 conference talks 和 personal blog。

死胡同：gitleaks 在 test fixture 中标记了一个 AWS-shaped key，而搜索该 string 会在 AWS 自己的 documentation 中找到逐字相同的内容。该项被丢弃，并记录为已丢弃，以免有人再次提出。

真正的 finding 是一个

terraform.tfstate

：两年前的同一周内被添加又删除，并从 history 中恢复出来。它没有 live credential，却包含四个 internal hostnames 和一个 cloud account ID——这些是供

find-hidden-subdomains

使用的基础设施。

评级：team roster 为 较可能（Probable），因为 commit authorship 是自我声明且未获 corroboration。Rename link 为 已确认（Confirmed），因为 numeric ID 不可变。

Pivots

转换方向

New selector	Goes to
Commit author emails	`what-an-email-reveals` , `what-leaked-about-you`
Handles, renamed accounts, numeric IDs	`hunt-a-handle`
Real names from commits and CODEOWNERS	`find-anyone`
Internal hostnames from config	`find-hidden-subdomains` , `who-owns-this-domain`
Cloud account IDs, buckets, IPs	`find-exposed-servers` , `google-like-a-spy`
Committed documents and images	`secrets-in-file-metadata`
Org structure and vendor stack	`x-ray-a-company`
Deleted repos and removed pages	`read-deleted-pages`
Credential appearing in a public dump	`find-leaks-in-the-wild`
The contributor network	`graph-the-network`

新选择器	指向
Commit author emails	`what-an-email-reveals` , `what-leaked-about-you`
Handles、renamed accounts、numeric IDs	`hunt-a-handle`
Commits 和 CODEOWNERS 中的真实姓名	`find-anyone`
Config 中的 internal hostnames	`find-hidden-subdomains` , `who-owns-this-domain`
Cloud account IDs、buckets、IPs	`find-exposed-servers` , `google-like-a-spy`
已 commit 的 documents 和 images	`secrets-in-file-metadata`
Org structure 和 vendor stack	`x-ray-a-company`
已删除的 repos 和已移除的 pages	`read-deleted-pages`
出现在 public dump 中的 credential	`find-leaks-in-the-wild`
Contributor network	`graph-the-network`

Legal, disclosure, and the one hard rule

法律、披露与唯一硬性规则

Never authenticate with a credential you find. Not to check whether it works, not read-only, not once. It is unauthorized access under computer-misuse law in most jurisdictions, and publication does not confer permission. Establish validity from format, context and metadata and nothing else. See ../../ETHICS.md.

A live-looking credential creates an obligation to disclose. Report promptly through a security contact, a

security.txt

route, or the host's vulnerability channel; give the repository, commit SHA and file path; state explicitly that history rewriting is not sufficient and the credential must be rotated; keep only a redacted reference; publish nothing. Hosts running automated secret-scanning partnerships may have revoked the token already, which does not remove the obligation to tell someone.

Cloning public repositories is normal use; bulk cloning and high-rate API querying are rate-limited and can breach terms. Commit emails are personal data under GDPR and equivalent regimes whatever the repository's status — collect the minimum and set a retention period.

绝不要用你找到的 credential 进行 authentication。 不是为了测试它是否有效，不是只读操作，也不只是做一次。在大多数司法辖区，这都属于 computer-misuse law 下的 unauthorized access；将其公开也不会赋予你权限。只能根据 format、context 和 metadata 判断其有效性，除此以外不要做其他验证。见 ../../ETHICS.md。

看起来仍然有效的 credential 会带来披露义务。应尽快通过 security contact、

security.txt

路径或 host 的 vulnerability channel 报告；提供 repository、commit SHA 和 file path；明确说明仅重写 history 并不足够，credential 必须被 rotate；只保留经过脱敏的 reference；不要公开任何内容。运行 automated secret-scanning partnerships 的 hosts 可能已经 revoke 了该 token，但这并不能免除通知相关人员的义务。

Clone public repositories 属于正常使用；批量 clone 和高频 API querying 会受到 rate limits 限制，也可能违反服务条款。无论 repository 处于何种状态，commit emails 都是 GDPR 及同等制度下的 personal data——只收集最低限度的数据，并设定 retention period。