colab-operator

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Skill: Colab Session Operator

Skill: Colab Session Operator

Operate Google Colab environments via the
colab
CLI: provision GPU/TPU sessions, run Python/shell on the VM, sync files, and capture work as notebooks.
通过
colab
CLI操作Google Colab环境:创建GPU/TPU会话、在虚拟机上运行Python/Shell脚本、同步文件,并将工作内容保存为笔记本。

Installation

安装

If the user does not already have the
colab
tool installed, it can be acquired by running
uv tool install google-colab-cli
or
pip install google-colab-cli
.
如果用户尚未安装
colab
工具,可以通过运行
uv tool install google-colab-cli
pip install google-colab-cli
获取。

When to activate

激活场景

  • Creating or managing TPU/GPU sessions.
  • Running Python or shell on a remote Colab VM.
  • Syncing files between local and remote.
  • Automating environment setup (packages, auth, Drive).
  • Exporting session history as a Jupyter notebook.
  • 创建或管理TPU/GPU会话。
  • 在远程Colab虚拟机上运行Python或Shell脚本。
  • 在本地与远程之间同步文件。
  • 自动化环境配置(包、认证、云端硬盘)。
  • 将会话历史导出为Jupyter笔记本。

Mental model (read this first)

核心认知(请先阅读)

  • A session == a live Jupyter kernel on a rented VM.
    colab new
    allocates a billable VM;
    colab stop
    releases it. Nothing reclaims it automatically except a 24h keep-alive cap, so an unstopped session burns compute units indefinitely.
  • Kernel state PERSISTS across
    colab exec
    /
    colab repl
    calls in the same session.
    Each invocation reattaches to the same kernel (the kernel ID is cached in local state) and only closes the websocket on exit — it does not shut the kernel down. So imports, variables, and defined functions survive between separate
    colab exec
    commands. Build up state incrementally; don't re-import everything each call. (
    colab stop
    and
    colab restart-kernel
    are what actually reset it.)
  • Default working directory is
    /content
    .
    Every
    exec
    /
    repl
    /
    run
    cd
    s there first; prefer absolute paths (
    /content/...
    ) for file work. For
    colab ls/rm/upload/download
    , absolute
    /content/...
    paths work and the default
    ls
    path is
    content
    (VM root).
  • colab
    is fire-and-forget.
    Each command authenticates, does one thing, and exits. A detached background daemon (spawned by
    colab new
    ) handles keep-alive; you don't manage it.
  • 一个会话 == 租用虚拟机上的活跃Jupyter内核
    colab new
    会分配一个计费虚拟机;
    colab stop
    会释放它。除了24小时的存活上限外,没有自动回收机制,因此未停止的会话会持续消耗计算资源。
  • 内核状态在同一会话的
    colab exec
    /
    colab repl
    调用间保持持久化
    。每次调用都会重新连接到同一个内核(内核ID会缓存在本地状态中),仅在退出时关闭WebSocket——并不会关闭内核。因此,导入的模块、变量和定义的函数会在多次
    colab exec
    命令之间保留。可以逐步构建状态,无需每次调用都重新导入所有内容。(
    colab stop
    colab restart-kernel
    才会真正重置内核状态。)
  • 默认工作目录为
    /content
    。每次
    exec
    /
    repl
    /
    run
    都会先切换到该目录;处理文件时建议使用绝对路径(
    /content/...
    )。对于
    colab ls/rm/upload/download
    命令,绝对路径
    /content/...
    有效,
    ls
    命令的默认路径为
    content
    (虚拟机根目录)。
  • colab
    是“即发即弃”式工具
    。每个命令完成认证、执行操作后即退出。
    colab new
    会启动一个后台守护进程来维持会话存活,无需手动管理该进程。

Authentication (the #1 thing that blocks agents)

认证(最容易阻碍Agent的环节)

  • The global flag is
    --auth={adc,oauth2}
    and the default is
    adc
    (Application Default Credentials). It must come before the subcommand:
    colab --auth=adc new -s x
    .
  • ADC setup (most reliable for headless/agent use). The Colab backends need a specific scope set, so re-mint ADC with all four scopes:
    bash
    gcloud auth application-default login \
      --scopes=openid,\
    https://www.googleapis.com/auth/cloud-platform,\
    https://www.googleapis.com/auth/userinfo.email,\
    https://www.googleapis.com/auth/colaboratory
    Why all four:
    userinfo.email
    (session backend
    colab.research.google.com
    , else 401),
    colaboratory
    (RuntimeService
    colab.pa.googleapis.com
    keep-alive, else 403),
    openid
    +
    cloud-platform
    (mandated by gcloud itself; it rejects scope lists missing
    cloud-platform
    ).
  • oauth2 setup:
    colab --auth=oauth2 <anything>
    triggers a browser consent flow on first use (token cached at
    ~/.config/colab-cli/token.json
    ). Requires a client config at
    ~/.colab-cli-oauth-config.json
    (or
    -c PATH
    ). The browser step means it usually needs a human; prefer ADC for agents.
  • Verify auth in one shot:
    colab sessions
    (read-only, lists server assignments) or
    colab whoami
    (hidden debug command: prints the active email, scopes, audience, and expiry). When any call 403s against
    colab.pa.googleapis.com
    , the cause is almost always a missing scope —
    colab whoami
    shows it instantly.
  • colab new
    pre-flights the keep-alive RPC
    right after allocating. If your token lacks the
    colaboratory
    scope it unassigns the fresh VM (so you don't leak a billable assignment) and prints the exact remediation. Follow that message rather than retrying blindly.
  • Do NOT confuse
    colab auth
    with CLI authentication.
    colab auth
    injects VM-side GCP credentials into the running kernel (so notebook code can call BigQuery/GCS); it is orthogonal to how the CLI itself authenticates. Never suggest "run
    colab auth
    " to fix a CLI 401/403 — that's a scope/identity problem fixed via the
    gcloud
    command above.
  • 全局参数为
    --auth={adc,oauth2}
    默认值为
    adc
    (应用默认凭证)。该参数必须放在子命令之前:
    colab --auth=adc new -s x
  • ADC配置(无头/Agent场景下最可靠)。Colab后端需要特定的权限范围,因此需要重新生成包含以下四个权限范围的ADC:
    bash
    gcloud auth application-default login \\
      --scopes=openid,\\
    https://www.googleapis.com/auth/cloud-platform,\\
    https://www.googleapis.com/auth/userinfo.email,\\
    https://www.googleapis.com/auth/colaboratory
    为什么需要这四个权限:
    userinfo.email
    (会话后端
    colab.research.google.com
    需要,否则会返回401错误)、
    colaboratory
    (RuntimeService
    colab.pa.googleapis.com
    维持会话存活需要,否则会返回403错误)、
    openid
    +
    cloud-platform
    (gcloud强制要求;缺少
    cloud-platform
    的权限列表会被拒绝)。
  • oauth2配置
    colab --auth=oauth2 <任意命令>
    会在首次使用时触发浏览器授权流程(令牌会缓存到
    ~/.config/colab-cli/token.json
    )。需要在
    ~/.colab-cli-oauth-config.json
    (或通过
    -c PATH
    指定路径)放置客户端配置。浏览器步骤通常需要人工操作,因此Agent场景下优先使用ADC。
  • 一键验证认证状态
    colab sessions
    (只读,列出服务器分配情况)或
    colab whoami
    (隐藏调试命令:打印当前活跃邮箱、权限范围、受众和过期时间)。当调用
    colab.pa.googleapis.com
    返回403错误时,原因几乎都是缺少权限范围——
    colab whoami
    可以立即显示相关信息。
  • colab new
    会在分配虚拟机后预先检查存活RPC
    。如果令牌缺少
    colaboratory
    权限,它会释放刚分配的虚拟机(避免浪费计费资源)并打印具体的修复步骤。请按照提示操作,不要盲目重试。
  • 不要混淆
    colab auth
    与CLI认证
    colab auth
    是将GCP凭证注入运行中的内核(以便笔记本代码可以调用BigQuery/GCS);它与CLI自身的认证方式无关。永远不要建议“运行
    colab auth
    ”来修复CLI的401/403错误——这类问题需要通过上述
    gcloud
    命令修复权限/身份问题。

Workflow

工作流程

Provision

创建会话

  • colab new -s <name>
    (CPU). Add
    --gpu A100
    or
    --tpu v6e1
    for accelerators. Always pass
    -s <name>
    — an omitted name is auto-generated as a random 6-hex string, which makes later commands ambiguous.
  • Supported
    --gpu
    :
    T4
    ,
    L4
    ,
    G4
    ,
    H100
    ,
    A100
    . Supported
    --tpu
    :
    v5e1
    ,
    v6e1
    .
  • Gotcha: an unrecognized
    --gpu
    value silently falls back to A100 (which then usually fails the next step). A
    400
    on
    colab new
    with an accelerator means no quota/entitlement for it on this account — fall back to
    --gpu T4
    or omit the flag for CPU.
  • Accelerator availability is tier-gated; most accounts can only get CPU. Don't assume a GPU/TPU will allocate.
  • colab new -s <名称>
    (CPU版本)。添加
    --gpu A100
    --tpu v6e1
    以使用加速器。务必传入
    -s <名称>
    ——省略名称会自动生成随机6位十六进制字符串,导致后续命令产生歧义。
  • 支持的
    --gpu
    参数:
    T4
    L4
    G4
    H100
    A100
    。支持的
    --tpu
    参数:
    v5e1
    v6e1
  • 注意事项:无法识别的
    --gpu
    值会自动回退到A100(通常会导致下一步失败)。使用加速器的
    colab new
    命令返回400错误,意味着当前账户没有该加速器的配额/权限——可以回退到
    --gpu T4
    或省略该参数使用CPU。
  • 加速器的可用性有层级限制;大多数账户只能使用CPU。不要默认认为GPU/TPU可以成功分配。

Execute

执行代码

  • Preferred:
    colab exec -s <name> -f <script.py>
    runs a local script on the remote VM (read locally, sent to the kernel — no manual upload needed).
  • Piped code:
    echo "print(1)" | colab exec -s <name>
    or
    cat script.py | colab exec -s <name>
    .
  • Notebooks:
    colab exec -s <name> -f nb.ipynb
    runs each code cell and writes results to
    <basename>_output.ipynb
    next to the input. A
    # @title Foo
    first line labels the cell in progress output.
  • Plots/images: PNG/JPEG outputs are intercepted. Use
    --output-image <path>
    on
    exec
    /
    repl
    to save to a known location (otherwise a temp path is printed). Inline terminal-image escapes are auto-suppressed when stdout isn't a TTY, so piped/captured output stays clean.
  • Shell:
    echo "cmd" | colab console -s <name>
    for batch shell. Console wraps bash in tmux, so even piped output contains terminal-control bytes — filter with
    grep -a
    for a specific line.
    exec
    is faster when you don't need a real shell.
  • Never run
    colab repl
    ,
    colab console
    ,
    colab auth
    , or
    colab drivemount
    interactively from an agent
    — they expect a TTY and will hang.
    repl
    /
    console
    accept piped stdin and exit on EOF;
    auth
    /
    drivemount
    genuinely require a human at the terminal.
  • 推荐方式
    colab exec -s <名称> -f <script.py>
    在远程虚拟机上运行本地脚本(本地读取脚本内容,发送到内核——无需手动上传)。
  • 管道代码
    echo "print(1)" | colab exec -s <名称>
    cat script.py | colab exec -s <名称>
  • 笔记本文件
    colab exec -s <名称> -f nb.ipynb
    运行每个代码单元格,并将结果写入输入文件旁的
    <basename>_output.ipynb
    。单元格第一行的
    # @title Foo
    会在进度输出中作为单元格标签。
  • 图表/图片:PNG/JPEG输出会被拦截。在
    exec
    /
    repl
    命令中使用
    --output-image <路径>
    可以将图片保存到指定位置(否则会打印临时路径)。当标准输出不是TTY时,终端内联图片转义序列会被自动抑制,因此管道/捕获的输出会保持整洁。
  • Shell脚本
    echo "cmd" | colab console -s <名称>
    用于批量执行Shell命令。Console会将bash包裹在tmux中,因此即使是管道输出也包含终端控制字节——可以使用
    grep -a
    过滤特定行。当不需要完整Shell环境时,
    exec
    命令速度更快。
  • 永远不要让Agent交互式运行
    colab repl
    colab console
    colab auth
    colab drivemount
    ——这些命令需要TTY,会导致挂起。
    repl
    /
    console
    接受标准输入管道,在EOF时退出;
    auth
    /
    drivemount
    确实需要人工在终端操作。

Ephemeral one-shot jobs (
colab run
)

临时一次性任务(
colab run

  • colab run [--gpu T4] [--tpu v6e1] [--keep] [-s NAME] script.py [args...]
    =
    new
    +
    exec
    +
    stop
    in one command. It provisions a fresh VM, runs the script with
    sys.argv
    and
    __name__ == "__main__"
    set like native
    python script.py args
    , then tears the VM down (unless
    --keep
    ).
  • Exit codes propagate: an uncaught exception or
    sys.exit(N)
    in the script makes
    colab run
    exit non-zero (CPython semantics:
    sys.exit()
    /
    sys.exit(0)
    → 0,
    sys.exit(N)
    → N,
    sys.exit("msg")
    → 1).
  • Stream separation:
    colab run
    writes its own
    [colab] ...
    chatter to stderr and the script's output to stdout — so
    colab run job.py > out.txt
    captures only the script's stdout. (
    colab exec
    streams the script's stdout/stderr live to your stdout/stderr.)
  • Works as a shebang:
    #!/usr/bin/env -S colab run --gpu T4
    makes a
    chmod +x
    'd
    .py
    a self-contained "rent a GPU, run, clean up" script. After editing CLI behavior, reinstall before testing shebangs — they resolve
    colab
    via
    $PATH
    , not the editable install.
  • A nonexistent script path exits non-zero before allocating a VM (no wasted compute).
  • colab run [--gpu T4] [--tpu v6e1] [--keep] [-s NAME] script.py [args...]
    =
    new
    +
    exec
    +
    stop
    的组合命令。它会创建一个新虚拟机,以原生
    python script.py args
    的方式设置
    sys.argv
    __name__ == "__main__"
    并运行脚本,然后销毁虚拟机(除非使用
    --keep
    参数)。
  • 退出码会传递:脚本中的未捕获异常或
    sys.exit(N)
    会使
    colab run
    返回非零退出码(符合CPython语义:
    sys.exit()
    /
    sys.exit(0)
    → 0,
    sys.exit(N)
    → N,
    sys.exit("msg")
    → 1)。
  • 流分离
    colab run
    会将自身的
    [colab] ...
    日志写入标准错误输出,脚本的输出写入标准输出——因此
    colab run job.py > out.txt
    只会捕获脚本的标准输出。(
    colab exec
    会将脚本的标准输出/错误实时流到本地的标准输出/错误。)
  • 可作为Shebang使用:
    #!/usr/bin/env -S colab run --gpu T4
    可以让赋予执行权限的
    .py
    文件成为一个独立的“租用GPU、运行、清理”脚本。修改CLI行为后,重新安装后再测试Shebang——它们通过
    $PATH
    解析
    colab
    ,而不是通过可编辑安装路径。
  • 如果脚本路径不存在,
    colab run
    会在分配虚拟机之前返回非零退出码(不会浪费计算资源)。

Automate

自动化配置

  • colab auth -s <name>
    — VM-side GCP creds, needed before in-VM GCS/BigQuery calls (interactive; not agent-runnable).
  • colab drivemount -s <name> [PATH]
    — mounts Drive at
    /content/drive
    by default (interactive; not agent-runnable).
  • colab install -s <name> pkg1 pkg2
    — installs via
    uv pip install --system
    , falling back to
    pip
    . Also
    colab install -s <name> -r requirements.txt
    .
  • colab auth -s <名称>
    ——为虚拟机注入GCP凭证,在虚拟机内调用GCS/BigQuery之前需要执行(交互式;无法由Agent运行)。
  • colab drivemount -s <名称> [PATH]
    ——默认将云端硬盘挂载到
    /content/drive
    (交互式;无法由Agent运行)。
  • colab install -s <名称> pkg1 pkg2
    ——通过
    uv pip install --system
    安装包,回退使用
    pip
    。也支持
    colab install -s <名称> -r requirements.txt

Inspect & report

检查与报告

  • colab help
    (or
    colab help <cmd>
    ) lists/explains commands; the listing is alphabetical.
  • colab sessions
    lists server-side assignments and auto-prunes stale local entries. Orphans with no local record show as
    [?]
    .
  • colab status [-s <name>]
    shows hardware, IDLE/BUSY, and last execution.
  • colab log -s <name> [-n 20] [-t TYPE]
    shows recent structured events; invaluable when a task fails (keep-alive errors carry the raw
    response_body
    ).
  • colab log -s <name> -o summary.ipynb
    exports the session as a notebook (also
    .md
    ,
    .txt
    ,
    .jsonl
    by suffix).
  • colab url -s <name>
    prints a browser URL that attaches the Colab web UI to your existing CLI session instead of allocating a new VM (add
    --open
    to launch it).
  • colab skill
    /
    colab readme
    print this skill and the README (handy for self-discovery).
  • colab help
    (或
    colab help <命令>
    )列出/解释所有命令;命令列表按字母顺序排列。
  • colab sessions
    列出服务器端的会话分配情况,并自动清理本地过期条目。没有本地记录的孤立会话会显示为
    [?]
  • colab status [-s <名称>]
    显示硬件信息、IDLE/BUSY状态和最后执行时间。
  • colab log -s <名称> [-n 20] [-t TYPE]
    显示最近的结构化事件;当任务失败时非常有用(存活错误会包含原始
    response_body
    )。
  • colab log -s <名称> -o summary.ipynb
    将会话导出为笔记本(也支持通过后缀导出为
    .md
    .txt
    .jsonl
    格式)。
  • colab url -s <名称>
    打印一个浏览器URL,可将Colab网页UI连接到现有CLI会话,而无需分配新虚拟机(添加
    --open
    参数可直接打开浏览器)。
  • colab skill
    /
    colab readme
    打印本Skill文档和README(便于自我查阅)。

Safety

安全注意事项

  • Always
    colab stop -s <name>
    when done
    — idle VMs burn compute units.
    colab run
    (without
    --keep
    ) self-cleans even if the script errors.
  • Local state lives in
    ~/.config/colab-cli/sessions.json
    (settings in
    settings.json
    , history in
    history/*.jsonl
    ). Don't edit by hand.
  • Isolate parallel/agent runs with the global
    --config <path>
    flag to point session state at a scratch file (e.g.
    colab --config /tmp/agent.json new -s job
    ). The keep-alive daemon inherits
    --auth
    and
    --config
    automatically.
  • 使用完成后务必执行
    colab stop -s <名称>
    ——闲置虚拟机会持续消耗计算资源。
    colab run
    (不使用
    --keep
    参数)即使脚本出错也会自动清理资源。
  • 本地状态存储在
    ~/.config/colab-cli/sessions.json
    (设置在
    settings.json
    ,历史记录在
    history/*.jsonl
    )。不要手动编辑这些文件。
  • 并行/Agent运行时请隔离状态:使用全局
    --config <路径>
    参数将会话状态指向临时文件(例如
    colab --config /tmp/agent.json new -s job
    )。存活守护进程会自动继承
    --auth
    --config
    参数。

Recovery

故障恢复

  • "Session not found" / 404 / 401 on exec: the backend pruned the VM.
    colab exec
    /
    repl
    detect this and clean up local state automatically — run
    colab sessions
    and re-create with
    colab new
    .
  • Execution timeout or wedged kernel:
    colab restart-kernel -s <name>
    (keeps the VM, resets the kernel), or
    colab stop
    then
    colab new
    .
  • Keep-alive daemon died (
    colab log
    shows
    keep_alive_stopped reason=consecutive_4xx_errors
    ): almost always the missing
    colaboratory
    scope — re-auth per the Authentication section.
  • “会话未找到”/404/401错误(执行命令时):后端已清理虚拟机。
    colab exec
    /
    repl
    会自动检测到这种情况并清理本地状态——运行
    colab sessions
    并使用
    colab new
    重新创建会话。
  • 执行超时或内核卡住:
    colab restart-kernel -s <名称>
    (保留虚拟机,重置内核),或先执行
    colab stop
    再执行
    colab new
  • 存活守护进程终止(
    colab log
    显示
    keep_alive_stopped reason=consecutive_4xx_errors
    ):几乎都是因为缺少
    colaboratory
    权限范围——按照“认证”部分重新进行认证。",