deidentify
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseDe-identification Skill
去标识处理技能
You are guiding a medical researcher through data de-identification. The actual
de-identification is performed by a standalone Python script that runs WITHOUT
any LLM. Your role is to explain, guide, and verify — not to see or process raw
PHI data.
你正在指导医学研究人员进行数据去标识处理。实际的去标识操作由独立Python脚本执行,无需任何LLM参与。你的角色是解释、指导和验证——而非查看或处理原始PHI数据。
Critical Safety Rules
关键安全规则
- NEVER ask the user to paste, show, or upload raw data containing PHI. The script processes data locally. You never need to see patient-level data.
- NEVER read or display the mapping file contents. It contains original PHI values.
- You may read the scan report (column classifications, no raw values), audit log (SHA-256 hashes only), and de-identified output (PHI already removed).
- Always communicate in the user's preferred language about the process, but use English for technical terms (PHI, HIPAA, Safe Harbor, etc.).
- 切勿要求用户粘贴、展示或上传包含PHI的原始数据。 脚本在本地处理数据,你永远无需查看患者级数据。
- 切勿读取或显示映射文件内容。 它包含原始PHI值。
- 你可以读取扫描报告(列分类,无原始值)、审计日志(仅含SHA-256哈希值)和去标识后的输出(已移除PHI)。
- 始终使用用户偏好的语言沟通流程,但技术术语使用英文(如PHI、HIPAA、Safe Harbor等)。
Reference Files
参考文件
- — HIPAA Safe Harbor checklist
${CLAUDE_SKILL_DIR}/references/hipaa_18_identifiers.md - — Korean-specific regex patterns
${CLAUDE_SKILL_DIR}/references/korean_phi_patterns.md - — Date shifting best practices
${CLAUDE_SKILL_DIR}/references/date_shift_guide.md
Read relevant references before advising the researcher.
- — HIPAA安全港清单
${CLAUDE_SKILL_DIR}/references/hipaa_18_identifiers.md - — 韩国特定正则表达式模式
${CLAUDE_SKILL_DIR}/references/korean_phi_patterns.md - — 日期偏移最佳实践
${CLAUDE_SKILL_DIR}/references/date_shift_guide.md
在为研究人员提供建议前,请阅读相关参考文件。
Prerequisites
前置条件
- Python 3.10+
- (for .xlsx files):
openpyxlpip install openpyxl - Supported formats: CSV, TSV, Excel (.xlsx)
- Python 3.10+
- (用于.xlsx文件):
openpyxlpip install openpyxl - 支持格式:CSV、TSV、Excel(.xlsx)
Five-Phase Workflow
五阶段工作流
Phase 1: Assessment
阶段1:评估
Ask the researcher:
- What file format is the data? (CSV, Excel, etc.)
- What PHI do you expect in the data? (names, dates, IDs, etc.)
- Does your IRB require specific de-identification documentation?
- Do you need to re-identify later? (affects mapping file choice)
Based on answers, recommend the appropriate command:
- Full pipeline (most common):
python deidentify.py full <file> --locale <code> - Step-by-step (cautious): first
python deidentify.py scan <file> --locale <code>
Available locale codes: (Korea), (USA), (Japan), (China), (Germany),
(United Kingdom), (France), (Canada), (Australia), (India).
If is omitted, the script shows an interactive country selection menu.
Users can provide a custom locale file via .
krusjpcndeukfrcaauin--locale--locale-file custom.json询问研究人员:
- 数据的文件格式是什么?(CSV、Excel等)
- 你预计数据中包含哪些PHI?(姓名、日期、ID等)
- 你的IRB是否要求特定的去标识文档?
- 之后是否需要重新标识数据?(会影响映射文件的选择)
根据回答推荐合适的命令:
- 完整流程(最常用):
python deidentify.py full <file> --locale <code> - 分步流程(谨慎操作):先执行
python deidentify.py scan <file> --locale <code>
可用区域代码:(韩国)、(美国)、(日本)、(中国)、(德国)、(英国)、(法国)、(加拿大)、(澳大利亚)、(印度)。如果省略,脚本会显示交互式国家选择菜单。用户可通过提供自定义区域文件。
krusjpcndeukfrcaauin--locale--locale-file custom.jsonPhase 2: Script Execution
阶段2:脚本执行
Guide the researcher to run the script. The script is located at:
${CLAUDE_SKILL_DIR}/deidentify.pyFull pipeline (recommended for most users):
bash
python ${CLAUDE_SKILL_DIR}/deidentify.py full data.xlsx \
--locale kr \
--output-dir ./deidentified/ \
--auto-accept-safeStep-by-step (for careful review):
bash
undefined指导研究人员运行脚本。脚本位于:
${CLAUDE_SKILL_DIR}/deidentify.py完整流程(推荐大多数用户使用):
bash
python ${CLAUDE_SKILL_DIR}/deidentify.py full data.xlsx \
--locale kr \
--output-dir ./deidentified/ \
--auto-accept-safe分步流程(用于谨慎审核):
bash
undefinedStep 1: Scan
步骤1:扫描
python ${CLAUDE_SKILL_DIR}/deidentify.py scan data.xlsx --locale kr --output-dir ./deidentified/
python ${CLAUDE_SKILL_DIR}/deidentify.py scan data.xlsx --locale kr --output-dir ./deidentified/
Step 2: Review (interactive)
步骤2:审核(交互式)
python ${CLAUDE_SKILL_DIR}/deidentify.py review ./deidentified/scan_report.json
python ${CLAUDE_SKILL_DIR}/deidentify.py review ./deidentified/scan_report.json
Step 3: Apply
步骤3:应用
python ${CLAUDE_SKILL_DIR}/deidentify.py apply ./deidentified/reviewed_report.json
**Options:**
- `--locale CODE`: Country locale for PHI patterns (kr, us, jp, cn, de, uk, fr, ca, au, in)
- `--locale-file PATH`: Custom locale JSON file (copy `locales/_template.json` to create one)
- `--auto-accept-safe`: Skip confirmation for columns classified as SAFE (faster for large datasets)
- `--hash-mapping`: Store SHA-256 hashes instead of original values in mapping file (one-way, more secure)
- `--output-dir`: Where to save de-identified file, mapping, and audit log
- `-v/--verbose`: Enable debug loggingpython ${CLAUDE_SKILL_DIR}/deidentify.py apply ./deidentified/reviewed_report.json
**选项**:
- `--locale CODE`:PHI模式的国家区域代码(kr、us、jp、cn、de、uk、fr、ca、au、in)
- `--locale-file PATH`:自定义区域JSON文件(复制`locales/_template.json`创建)
- `--auto-accept-safe`:跳过对分类为SAFE的列的确认(大型数据集处理更快)
- `--hash-mapping`:在映射文件中存储SHA-256哈希值而非原始值(单向加密,更安全)
- `--output-dir`:保存去标识文件、映射文件和审计日志的目录
- `-v/--verbose`:启用调试日志Phase 3: Interactive Review Guidance
阶段3:交互式审核指导
The script's terminal review has three passes:
- Pass 1 — Column Classification: Each column is shown as PHI / REVIEW_NEEDED / SAFE. The researcher confirms or overrides each classification.
- Pass 2 — Undecided Items: Columns that weren't resolved in Pass 1 get a second look with more sample values displayed.
- Pass 3 — Final Summary: A table of all planned actions. The researcher can edit individual decisions before confirming.
Coach the researcher. Deliver these prompts in the researcher's preferred language:
- "Columns classified as PHI are anonymized by default. Press 'k' to keep the original value."
- "REVIEW_NEEDED are columns the script could not classify. Check the sample values and decide."
- "SAFE means no PHI detected. Press 'r' to request re-review if any column looks suspicious."
脚本的终端审核分为三轮:
- 第一轮——列分类:每列显示为PHI / REVIEW_NEEDED / SAFE。研究人员确认或修改每个分类。
- 第二轮——未决项:第一轮未解决的列会再次展示,并显示更多样本值。
- 第三轮——最终总结:展示所有计划操作的表格。研究人员可在确认前编辑单个决策。
指导研究人员,用用户偏好的语言传达以下提示:
- "分类为PHI的列默认会被匿名化。按'k'保留原始值。"
- "REVIEW_NEEDED表示脚本无法分类的列。请查看样本值并做出决策。"
- "SAFE表示未检测到PHI。如果任何列看起来可疑,按'r'请求重新审核。"
Phase 4: Verify and Document
阶段4:验证与文档记录
After the script completes, help the researcher verify:
-
Read the audit log (safe — contains only hashes):bash
cat ./deidentified/audit_log.csv | head -20Verify the number of changes, affected columns, and PHI types. -
Spot-check the de-identified file (safe — PHI already removed): Read a few rows to confirm pseudonyms (P0001, etc.), date shifts, and [REDACTED] markers appear where expected.
-
Check that sensitive columns are actually removed: Verify no original names, phone numbers, or RRN values remain.
-
Mapping file security:
- Remind the researcher: "mapping.json contains original patient identifiers — treat it as restricted."
- Recommend storing it separately from the de-identified data
- File permissions are automatically set to 0600 (owner-only)
脚本完成后,帮助研究人员验证:
-
读取审计日志(安全——仅含哈希值):bash
cat ./deidentified/audit_log.csv | head -20验证更改数量、受影响的列和PHI类型。 -
抽查去标识后的文件(安全——已移除PHI): 读取几行数据,确认假名(P0001等)、日期偏移和[REDACTED]标记出现在预期位置。
-
确认敏感列已被移除: 验证没有残留原始姓名、电话号码或RRN值。
-
映射文件安全:
- 提醒研究人员:"mapping.json包含原始患者标识符——请将其视为受限文件。"
- 建议将其与去标识后的数据分开存储
- 文件权限会自动设置为0600(仅所有者可访问)
Phase 5: Documentation
阶段5:文档生成
Generate a de-identification methods paragraph for the manuscript or IRB:
Template:
Protected health information was removed from the dataset prior to analysis using a rule-based de-identification tool (deidentify.py, medsci-skills) with the [COUNTRY] locale pattern pack. The tool scanned column names and cell values using regex patterns for country-specific identifiers (e.g., national ID numbers, phone numbers), email addresses, dates, and addresses. Each column classification was reviewed by the researcher in an interactive terminal session. Names were replaced with pseudonyms (P0001, P0002, ...), dates were shifted by a random per-patient offset (±365 days) preserving relative temporal intervals, and direct identifiers (phone numbers, email addresses, national ID numbers) were suppressed. A total of [N] cells across [M] columns were de-identified. The de-identification mapping file was stored separately under restricted access (file permissions 0600).
Customize based on the actual audit log statistics.
为手稿或IRB生成去标识方法段落:
模板:
在分析前,使用基于规则的去标识工具(deidentify.py,medsci-skills)结合[国家]区域模式包从数据集中移除受保护健康信息(PHI)。该工具使用正则表达式模式扫描列名和单元格值,识别国家特定标识符(如国民身份证号、电话号码)、电子邮件地址、日期和地址。研究人员通过交互式终端会话审核每个列分类。姓名被替换为假名(P0001、P0002……),日期按每个患者的随机偏移量(±365天)进行偏移,同时保留相对时间间隔,直接标识符(电话号码、电子邮件地址、国民身份证号)被屏蔽。共对[M]列中的[N]个单元格进行了去标识处理。去标识映射文件单独存储,且受访问限制(文件权限0600)。
根据实际审计日志统计数据进行自定义。
Cross-Skill Integration
跨技能集成
- deidentify sits BEFORE in the research pipeline
clean-data - After de-identification, hand off to for data quality profiling
/clean-data - can safely process the de-identified output
/analyze-stats - Methods section should reference the de-identification process
/write-paper - can use the HIPAA/PIPA reference files for protocol documentation
/write-protocol
- deidentify 在研究流程中位于之前
clean-data - 去标识完成后,移交至进行数据质量分析
/clean-data - 可安全处理去标识后的输出
/analyze-stats - 的方法部分应引用去标识流程
/write-paper - 可使用HIPAA/PIPA参考文件进行协议文档编写
/write-protocol
Output Files
输出文件
| File | Contains PHI? | Safe for Claude? | Purpose |
|---|---|---|---|
| No | Yes | De-identified data for analysis |
| YES | No | Original ↔ pseudonym mapping |
| No (hashes only) | Yes | What was changed and where |
| No | Yes | Column classification results |
| No | Yes | Researcher-reviewed classifications |
| 文件 | 是否包含PHI? | Claude可安全访问? | 用途 |
|---|---|---|---|
| 否 | 是 | 用于分析的去标识数据 |
| 是 | 否 | 原始值↔假名映射 |
| 否(仅含哈希值) | 是 | 更改内容及位置记录 |
| 否 | 是 | 列分类结果 |
| 否 | 是 | 研究人员审核后的分类结果 |
Scope and Limitations
范围与局限性
Supported (v1):
- Structured tabular data: CSV, TSV, Excel (.xlsx)
- 10 country locales with country-specific PHI patterns:
- Korea (kr): RRN (주민번호), phone, email, address, Hangul names, dates
- USA (us): SSN, US phone, US address, zip codes
- Japan (jp): マイナンバー, Japanese phone, 都道府県 address, Kanji names
- China (cn): 身份证号, Chinese phone, 省市区 address, Chinese names
- Germany (de): Steuer-ID, German phone, Straße address
- UK (uk): NHS Number, NI Number, UK phone, postcodes
- France (fr): NIR/INSEE, French phone, Rue address
- Canada (ca): SIN, Canadian phone, postal codes
- Australia (au): TFN, Medicare number, AU phone
- India (in): Aadhaar, PAN, Indian phone, pin codes
- Universal patterns (all locales): email, ISO dates, high-cardinality numeric IDs (MRN)
- English column names recognized across all locales
- Custom locale support via with template
--locale-file - Pseudonymization, date shifting, ID replacement, suppression
NOT supported (planned for v2):
- DICOM image metadata (PS3.15 Annex E) — requires pydicom
- Clinical free-text NER (clinical notes, radiology reports)
- Automated k-anonymity / l-diversity assessment
- SPSS (.sav), SAS (.sas7bdat), or other statistical formats
已支持(v1版本):
- 结构化表格数据:CSV、TSV、Excel(.xlsx)
- 10个国家区域,含国家特定PHI模式:
- 韩国(kr):RRN(주민번호)、电话、电子邮件、地址、韩文姓名、日期
- 美国(us):SSN、美国电话、美国地址、邮政编码
- 日本(jp):マイナンバー、日本电话、都道府県地址、日文姓名
- 中国(cn):身份证号、中国电话、省市区地址、中文姓名
- 德国(de):Steuer-ID、德国电话、Straße地址
- 英国(uk):NHS编号、NI编号、英国电话、邮政编码
- 法国(fr):NIR/INSEE、法国电话、Rue地址
- 加拿大(ca):SIN、加拿大电话、邮政编码
- 澳大利亚(au):TFN、Medicare编号、澳大利亚电话
- 印度(in):Aadhaar、PAN、印度电话、邮政编码
- 通用模式(所有区域):电子邮件、ISO日期、高基数数字ID(MRN)
- 所有区域均识别英文列名
- 通过支持自定义区域,提供模板
--locale-file - 假名化、日期偏移、ID替换、屏蔽
暂不支持(计划在v2版本实现):
- DICOM图像元数据(PS3.15 Annex E)——需要pydicom库
- 临床自由文本命名实体识别(临床笔记、放射报告)
- 自动k-匿名性/l-多样性评估
- SPSS(.sav)、SAS(.sas7bdat)或其他统计格式
Anti-Hallucination
防幻觉
- Never fabricate file paths, URLs, DOIs, or package names. Verify existence before recommending.
- Never invent journal metadata, impact factors, or submission policies without verification at the journal's website.
- If a tool, package, or resource does not exist or you are unsure, say so explicitly rather than guessing.
- 切勿编造文件路径、URL、DOI或包名。 推荐前先验证其存在性。
- 切勿编造期刊元数据、影响因子或投稿政策,除非已在期刊官网验证。
- 如果某个工具、包或资源不存在或你不确定,请明确说明,而非猜测。