autoskill

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

autoskill

autoskill

Requires a running screenpipe daemon. This skill has no alternate data source — it reads exclusively from the local screenpipe HTTP API (default
http://localhost:3030
). If the daemon isn't running,
run()
raises
ScreenpipeUnreachable
with install instructions.
Network access & environment variables. This skill makes authenticated HTTP requests to (a) the user's local screenpipe daemon on loopback, and (b) the user-configured LLM backend — one of
http://localhost:1234/v1
(LM Studio, default),
https://api.anthropic.com
(opt-in Claude), or a user-supplied BYOK Foundry gateway. The skill reads three environment variables —
SCREENPIPE_TOKEN
,
ANTHROPIC_API_KEY
,
FOUNDRY_API_KEY
— and uses each only to authenticate to the single endpoint its name implies. No other network destinations, no telemetry, no data egress to any third party.
需要运行中的screenpipe守护进程。 该技能无备用数据源——仅从本地screenpipe HTTP API(默认地址
http://localhost:3030
)读取数据。若守护进程未运行,
run()
方法会抛出
ScreenpipeUnreachable
异常并附带安装说明。
网络访问与环境变量。 该技能会向以下地址发起已认证的HTTP请求:(a) 用户本地回环地址上的screenpipe守护进程;(b) 用户配置的LLM后端——可选
http://localhost:1234/v1
(LM Studio,默认值)、
https://api.anthropic.com
(可选的Claude),或用户提供的BYOK Foundry网关。该技能会读取三个环境变量:
SCREENPIPE_TOKEN
ANTHROPIC_API_KEY
FOUNDRY_API_KEY
,且每个变量仅用于对其名称对应的单一端点进行认证。无其他网络目标,无遥测数据,无数据外传至任何第三方。

Overview

概述

Turn the user's own workflow history — captured passively by the local screenpipe daemon — into new skills. This skill is on-demand: the user invokes it with a time window, it queries screenpipe's local HTTP API, clusters repeated workflow patterns, compares each pattern against the existing skills in this repo, and produces a staged folder of proposals the user can review, edit, and promote.
将用户的工作流历史记录(由本地screenpipe守护进程被动捕获)转化为新技能。该技能为按需触发:用户指定时间窗口调用它,它会查询screenpipe的本地HTTP API,对重复的工作流模式进行聚类,将每个模式与本仓库中的现有技能进行对比,并生成一个包含建议内容的分级文件夹,供用户审核、编辑和推广使用。

When to Use This Skill

使用场景

Invoke this skill when the user asks to:
  • "Analyze my last 4 hours / day / week and propose new skills."
  • "Look at what I've been doing and tell me what's not covered yet."
  • "Draft a skill from my recent workflow."
  • "Find composition recipes for workflows I repeat."
Do not invoke it for one-off questions about screenpipe itself, for real-time screen queries, or without an explicit user request — the skill analyzes sensitive local content and must stay explicitly user-triggered.
当用户提出以下请求时调用该技能:
  • "分析我过去4小时/一天/一周的工作,提出新技能建议。"
  • "看看我一直在做什么,告诉我哪些内容尚未被覆盖。"
  • "根据我近期的工作流起草一个技能。"
  • "为我重复执行的工作流寻找组合方案。"
请勿在以下场景调用:关于screenpipe本身的一次性问题、实时屏幕查询,或无明确用户请求的情况——该技能会分析敏感的本地内容,必须由用户显式触发。

Privacy Posture

隐私保障

  • Screenpipe handles app/window filtering at capture time. Install a starter deny-list by copying
    references/screenpipe-config.yaml
    into the user's screenpipe config. Sensitive apps (password managers, messaging, banking) are never OCR'd in the first place.
  • Raw OCR never leaves the machine.
    scripts/fetch_window.py
    pulls data over localhost HTTP.
    scripts/cluster.py
    reduces the timeline to app/duration/title summaries.
    scripts/redact.py
    strips emails, API keys, bearer tokens, and phone numbers as defense-in-depth before any cluster summary reaches the LLM.
  • LLM backend defaults to
    local
    .
    The recommended setup is LM Studio running
    Gemma-4-31B-it
    — strong reasoning at a size that fits on most workstation GPUs, and no data ever leaves your machine. Cloud backends (
    claude
    ,
    foundry
    ) are opt-in and documented in
    config.yaml
    for users who explicitly want them. Detection and embeddings always run locally regardless of backend choice.
  • Dry-run mode (
    --plan
    ) prints the exact timeline that will be analyzed before any LLM call.
  • TLS for localhost (optional, for corporate policy): see
    references/https-proxy.md
    for the Caddy pattern.
  • screenpipe在捕获阶段处理应用/窗口过滤。 可通过将
    references/screenpipe-config.yaml
    复制到用户的screenpipe配置目录来安装初始拒绝列表。敏感应用(密码管理器、通讯软件、银行应用)从一开始就不会被OCR识别。
  • 原始OCR数据永远不会离开本地机器。
    scripts/fetch_window.py
    通过本地HTTP拉取数据。
    scripts/cluster.py
    将时间线简化为应用/时长/标题摘要。
    scripts/redact.py
    会在聚类摘要发送至LLM之前,通过深度防御机制去除邮箱、API密钥、Bearer令牌和电话号码。
  • LLM后端默认设置为
    local
    推荐配置为运行
    Gemma-4-31B-it
    LM Studio——具备强大推理能力,且大小适配大多数工作站GPU,数据永远不会离开你的机器。云端后端(
    claude
    foundry
    )为可选配置,在
    config.yaml
    中有文档说明,供明确需要的用户使用。无论选择哪种后端,检测和嵌入操作始终在本地执行。
  • 试运行模式
    --plan
    )会在调用LLM之前打印出将被分析的精确时间线。
  • 本地TLS加密(可选,符合企业政策):参考
    references/https-proxy.md
    中的Caddy配置方案。

Prerequisites

前置条件

1. Screenpipe daemon

1. Screenpipe守护进程

Either install the official release or build from source. Either way the daemon binds HTTP on
localhost:3030
by default.
From source (recommended if you want the CLI daemon without the desktop GUI):
bash
git clone --depth 1 https://github.com/mediar-ai/screenpipe.git
cd screenpipe
cargo build -p screenpipe-engine --release
可安装官方发布版本或从源码构建。无论哪种方式,守护进程默认会在
localhost:3030
绑定HTTP服务。
从源码构建(若你需要CLI守护进程而非桌面GUI,推荐此方式):
bash
git clone --depth 1 https://github.com/mediar-ai/screenpipe.git
cd screenpipe
cargo build -p screenpipe-engine --release

System deps (macOS): cmake + full Xcode.app (not just Command Line Tools).

系统依赖(macOS):cmake + 完整Xcode.app(而非仅命令行工具)。

brew install cmake

brew install cmake

# if xcodebuild plug-ins error: sudo xcodebuild -runFirstLaunch

# 若出现xcodebuild插件错误:sudo xcodebuild -runFirstLaunch

./target/release/screenpipe doctor # confirm permissions + ffmpeg ./target/release/screenpipe record --disable-audio --use-pii-removal

First run will prompt for macOS Screen Recording permission. Grant it and relaunch.
./target/release/screenpipe doctor # 确认权限和ffmpeg安装情况 ./target/release/screenpipe record --disable-audio --use-pii-removal

首次运行会提示获取macOS屏幕录制权限。授予权限后重新启动。

2. Screenpipe API token

2. Screenpipe API令牌

The local API now requires bearer auth. Retrieve your token and export it:
bash
export SCREENPIPE_TOKEN=$(screenpipe auth token)
(Or set
screenpipe.token
directly in
config.yaml
— env var is preferred since it keeps secrets out of version control.)
本地API现在需要Bearer认证。获取你的令牌并导出:
bash
export SCREENPIPE_TOKEN=$(screenpipe auth token)
(也可直接在
config.yaml
中设置
screenpipe.token
——推荐使用环境变量,因为它能避免将密钥存入版本控制系统。)

3. Python environment

3. Python环境

Via
pipenv
from the repo root:
bash
pipenv install httpx pyyaml sentence-transformers
The embedding model (
sentence-transformers/all-MiniLM-L6-v2
, ~80 MB) downloads on first run.
在仓库根目录通过
pipenv
安装:
bash
pipenv install httpx pyyaml sentence-transformers
嵌入模型(
sentence-transformers/all-MiniLM-L6-v2
,约80MB)会在首次运行时下载。

4. Local LLM (default path) — LM Studio

4. 本地LLM(默认方案)——LM Studio

  • Install LM Studio.
  • Download
    Gemma-4-31B-it
    (or another strong reasoning model; adjust
    local.model
    in
    config.yaml
    ).
  • Load it via the CLI for headless use (no GUI required):
bash
lms load gemma-4-31b-it --context-length 131072 --gpu max -y
lms status   # confirm server running on :1234
  • 安装LM Studio
  • 下载
    Gemma-4-31B-it
    (或其他推理能力较强的模型;可在
    config.yaml
    中调整
    local.model
    )。
  • 通过CLI加载模型以实现无头使用(无需GUI):
bash
lms load gemma-4-31b-it --context-length 131072 --gpu max -y
lms status   # 确认服务器在:1234端口运行

5. Cloud LLM backends (optional, opt-in)

5. 云端LLM后端(可选,需主动启用)

Only if you explicitly opt out of local:
  • claude
    : set
    ANTHROPIC_API_KEY
    , flip
    backend: claude
    in
    config.yaml
    .
  • foundry
    : set
    FOUNDRY_API_KEY
    , flip
    backend: foundry
    , set
    foundry.endpoint
    to your corporate gateway URL.
仅当你明确选择不使用本地LLM时才配置:
  • claude
    :设置
    ANTHROPIC_API_KEY
    ,在
    config.yaml
    中将
    backend: claude
  • foundry
    :设置
    FOUNDRY_API_KEY
    ,在
    config.yaml
    中将
    backend: foundry
    ,并将
    foundry.endpoint
    设置为你的企业网关URL。

Architecture

架构

screenpipe daemon (user-installed)
        │  HTTP on localhost:3030
scripts/fetch_window.py    → normalized timeline events
scripts/redact.py          → regex scrub (defense-in-depth)
scripts/cluster.py         → sessions + clusters (local only)
scripts/match_skills.py    → top-k vs existing 135 skills (local embeddings)
scripts/synthesize.py      → LLM judge: reuse / compose / novel
~/.autoskill/proposed/<timestamp>/        (default; override with --out)
  ├── report.md
  ├── composition-recipes/<name>/SKILL.md
  └── new-skills/<name>/SKILL.md

scripts/promote.py         → user-approved proposal → academic-skills/<name>/
screenpipe守护进程(用户自行安装)
        │  通过localhost:3030提供HTTP服务
scripts/fetch_window.py    → 标准化时间线事件
scripts/redact.py          → 正则表达式清理(深度防御)
scripts/cluster.py         → 会话与聚类(仅本地执行)
scripts/match_skills.py    → 与现有135个技能进行Top-K匹配(本地嵌入)
scripts/synthesize.py      → LLM判断:复用/组合/新建
~/.autoskill/proposed/<timestamp>/        (默认路径;可通过--out参数覆盖)
  ├── report.md
  ├── composition-recipes/<name>/SKILL.md
  └── new-skills/<name>/SKILL.md

scripts/promote.py         → 用户批准的建议 → academic-skills/<name>/

Workflow

工作流程

The skill ships a unified CLI at
scripts/autoskill.py
with three subcommands:
bash
python scripts/autoskill.py doctor   --config config.yaml --skills-dir ../
python scripts/autoskill.py run      --start ... --end ... --config config.yaml
python scripts/autoskill.py promote  --proposed ~/.autoskill/proposed/<ts> --skills-dir ../ --name <skill>
该技能在
scripts/autoskill.py
提供了统一的CLI,包含三个子命令:
bash
python scripts/autoskill.py doctor   --config config.yaml --skills-dir ../
python scripts/autoskill.py run      --start ... --end ... --config config.yaml
python scripts/autoskill.py promote  --proposed ~/.autoskill/proposed/<ts> --skills-dir ../ --name <skill>

0. Preflight with
doctor

0. 使用
doctor
进行预检

Before a full run, verify every dependency in one shot:
bash
python scripts/autoskill.py doctor \
  --config academic-skills/autoskill/config.yaml \
  --skills-dir academic-skills
The report covers
config
(backend choice valid),
skills_dir
(exists),
screenpipe
(reachable + authed), and
llm
(LM Studio serving or API key present). Non-zero exit on any failure, with the offending line marked
error
.
在完整运行前,一次性验证所有依赖项:
bash
python scripts/autoskill.py doctor \
  --config academic-skills/autoskill/config.yaml \
  --skills-dir academic-skills
报告内容涵盖
config
(后端选择有效)、
skills_dir
(目录存在)、
screenpipe
(可连接且已认证)和
llm
(LM Studio正在运行或API密钥已配置)。若任何检查失败,会返回非零退出码,并将问题行标记为
error

1. Run the pipeline

1. 运行流水线

bash
export SCREENPIPE_TOKEN=$(screenpipe auth token)
python scripts/autoskill.py run \
  --start "2026-04-17T00:00:00Z" \
  --end   "2026-04-17T23:59:59Z" \
  --config academic-skills/autoskill/config.yaml \
  --skills-dir academic-skills
Proposals land in
~/.autoskill/proposed/<timestamp>/
by default, keeping experimental output out of the skills repo. Pass
--out PATH
to override.
Internally:
  1. Fetch
    fetch_window
    paginates screenpipe's
    /search
    endpoint, normalizes events to
    {ts, app, window_title, text, content_type}
    .
  2. Redact
    redact
    scrubs emails, API keys, bearer tokens, phones from OCR text and window titles as defense-in-depth over screenpipe's own PII removal.
  3. Cluster
    segment_sessions
    splits on idle gaps (default 10 min) and drops short sessions;
    cluster_sessions
    groups sessions by app-signature and keeps clusters of size
    min_cluster_size
    (default 2).
  4. Match
    load_skill_descriptions
    reads frontmatter from every
    SKILL.md
    in
    academic-skills/
    ;
    top_k_matches
    ranks each cluster against all skills using local
    sentence-transformers
    embeddings (cosine similarity).
  5. Synthesize
    synthesize
    prompts the configured LLM backend to classify each cluster as
    reuse
    ,
    compose
    , or
    novel
    and emit a SKILL.md body where appropriate.
  6. Report — writes
    <out_dir>/<ts>/report.md
    , plus
    new-skills/<name>/SKILL.md
    or
    composition-recipes/<name>/SKILL.md
    for each proposal.
Add
--dry-run
to stop after clustering; this skips the LLM (and the sentence-transformers load), writing only
plan.md
for inspection.
bash
export SCREENPIPE_TOKEN=$(screenpipe auth token)
python scripts/autoskill.py run \
  --start "2026-04-17T00:00:00Z" \
  --end   "2026-04-17T23:59:59Z" \
  --config academic-skills/autoskill/config.yaml \
  --skills-dir academic-skills
建议内容默认会保存到
~/.autoskill/proposed/<timestamp>/
,避免实验性输出进入技能仓库。可通过
--out PATH
参数覆盖默认路径。
内部执行步骤:
  1. 获取数据
    fetch_window
    分页查询screenpipe的
    /search
    端点,将事件标准化为
    {ts, app, window_title, text, content_type}
    格式。
  2. 脱敏处理
    redact
    会从OCR文本和窗口标题中清理邮箱、API密钥、Bearer令牌和电话号码,作为screenpipe自身PII移除机制之外的深度防御措施。
  3. 聚类
    segment_sessions
    根据空闲间隔(默认10分钟)拆分会话并丢弃短会话;
    cluster_sessions
    根据应用特征对会话进行分组,保留大小达到
    min_cluster_size
    (默认2)的聚类。
  4. 匹配技能
    load_skill_descriptions
    读取
    academic-skills/
    中每个
    SKILL.md
    的前置内容;
    top_k_matches
    使用本地
    sentence-transformers
    嵌入(余弦相似度)将每个聚类与所有技能进行排名。
  5. 生成内容
    synthesize
    调用配置的LLM后端,将每个聚类分类为
    reuse
    (复用)、
    compose
    (组合)或
    novel
    (新建),并在合适情况下生成SKILL.md内容。
  6. 生成报告 — 写入
    <out_dir>/<ts>/report.md
    ,同时为每个建议生成
    new-skills/<name>/SKILL.md
    composition-recipes/<name>/SKILL.md
添加
--dry-run
参数可在聚类完成后停止执行;此参数会跳过LLM调用(以及sentence-transformers加载),仅生成
plan.md
供检查。

2. Review and promote

2. 审核与推广

Open
~/.autoskill/proposed/<ts>/report.md
, edit drafts in place, delete anything you don't want. Then:
bash
python scripts/autoskill.py promote \
  --proposed ~/.autoskill/proposed/2026-04-17T14-30-00 \
  --skills-dir academic-skills \
  --name zotero-pubmed-helper
promote
moves the directory into
academic-skills/<name>/
, refusing to overwrite an existing skill. Exits non-zero with a friendly error if the proposal isn't found or the target already exists.
打开
~/.autoskill/proposed/<ts>/report.md
,在原地编辑草稿,删除不需要的内容。然后执行:
bash
python scripts/autoskill.py promote \
  --proposed ~/.autoskill/proposed/2026-04-17T14-30-00 \
  --skills-dir academic-skills \
  --name zotero-pubmed-helper
promote
会将目录移动到
academic-skills/<name>/
,且不会覆盖已存在的技能。若未找到建议内容或目标技能已存在,会返回非零退出码并显示友好错误信息。

Configuration

配置

See
config.yaml
for the full shape. Default values (local-first):
yaml
backend: local
local:
  endpoint: http://localhost:1234/v1   # LM Studio's Developer server
  model: Gemma-4-31B-it

screenpipe:
  url: http://localhost:3030           # or https://screenpipe.local via Caddy

cluster:
  min_session_minutes: 5
  idle_gap_minutes: 10
  min_cluster_size: 2
To opt into a cloud backend:
yaml
backend: claude                         # or foundry
claude:
  model: claude-opus-4-7
完整配置格式请参考
config.yaml
。默认值(本地优先):
yaml
backend: local
local:
  endpoint: http://localhost:1234/v1   # LM Studio的开发者服务器
  model: Gemma-4-31B-it

screenpipe:
  url: http://localhost:3030           # 或通过Caddy配置为https://screenpipe.local

cluster:
  min_session_minutes: 5
  idle_gap_minutes: 10
  min_cluster_size: 2
若要启用云端后端:
yaml
backend: claude                         # 或foundry
claude:
  model: claude-opus-4-7

Composition recipes vs new skills

组合方案与新技能的区别

  • compose: the LLM judged that chaining existing skills covers the workflow. The emitted SKILL.md is intentionally thin — frontmatter + a "Workflow" section that invokes existing skills in order. The same agent runtime that discovered the skill can then invoke it end-to-end.
  • novel: no combination of existing skills covers it. A fuller SKILL.md is drafted, still following repo conventions (frontmatter, Overview, When to Use, Workflow). The user should always review new-skill drafts before promoting.
  • 组合(compose):LLM判定串联现有技能即可覆盖该工作流。生成的SKILL.md内容较为简洁——仅包含前置内容和一个调用现有技能的"工作流"章节。发现该技能的同一Agent运行时可直接端到端调用它。
  • 新建(novel):现有技能的组合无法覆盖该工作流。会生成更完整的SKILL.md草稿,仍遵循仓库规范(前置内容、概述、使用场景、工作流)。用户在推广前应始终审核新技能草稿。

Testing

测试

The skill is covered by a small pytest suite at
tests/
. Each script is unit-tested in isolation with dependency injection (mock HTTP transport, stub backend, stub embedder):
bash
cd academic-skills/autoskill
python -m pytest tests/ -v
该技能在
tests/
目录下有一个小型pytest测试套件。每个脚本都通过依赖注入(模拟HTTP传输、存根后端、存根嵌入器)进行独立单元测试:
bash
cd academic-skills/autoskill
python -m pytest tests/ -v

Composition with other skills in this repo

与本仓库中其他技能的组合

The autoskill's embedding index covers all 135 sibling skills. Workflows that look like scientific writing will match
scientific-writing
/
literature-review
/
citation-management
; figure work will match
scientific-schematics
/
generate-image
/
infographics
; slide prep matches
scientific-slides
/
pptx
; etc. When a cluster scores high against two or three sibling skills the emitted composition recipe names them explicitly, so the user's future agent invocations use the optimized paths already documented in this repo.
AutoSkill的嵌入索引涵盖了所有135个同级技能。类似科学写作的工作流会匹配
scientific-writing
/
literature-review
/
citation-management
;图表制作工作流会匹配
scientific-schematics
/
generate-image
/
infographics
;幻灯片制作会匹配
scientific-slides
/
pptx
等。当某个聚类与两三个同级技能的匹配度较高时,生成的组合方案会明确列出这些技能,以便用户未来的Agent调用使用本仓库中已记录的优化路径。