colab-operator
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseSkill: Colab Session Operator
Skill: Colab Session Operator
Operate Google Colab environments via the CLI: provision GPU/TPU sessions, run Python/shell on the VM, sync files, and capture work as notebooks.
colab通过 CLI操作Google Colab环境:创建GPU/TPU会话、在虚拟机上运行Python/Shell脚本、同步文件,并将工作内容保存为笔记本。
colabInstallation
安装
If the user does not already have the tool installed, it can be acquired
by running or .
colabuv tool install google-colab-clipip install google-colab-cli如果用户尚未安装工具,可以通过运行或获取。
colabuv tool install google-colab-clipip install google-colab-cliWhen to activate
激活场景
- Creating or managing TPU/GPU sessions.
- Running Python or shell on a remote Colab VM.
- Syncing files between local and remote.
- Automating environment setup (packages, auth, Drive).
- Exporting session history as a Jupyter notebook.
- 创建或管理TPU/GPU会话。
- 在远程Colab虚拟机上运行Python或Shell脚本。
- 在本地与远程之间同步文件。
- 自动化环境配置(包、认证、云端硬盘)。
- 将会话历史导出为Jupyter笔记本。
Mental model (read this first)
核心认知(请先阅读)
- A session == a live Jupyter kernel on a rented VM. allocates a billable VM;
colab newreleases it. Nothing reclaims it automatically except a 24h keep-alive cap, so an unstopped session burns compute units indefinitely.colab stop - Kernel state PERSISTS across /
colab execcalls in the same session. Each invocation reattaches to the same kernel (the kernel ID is cached in local state) and only closes the websocket on exit — it does not shut the kernel down. So imports, variables, and defined functions survive between separatecolab replcommands. Build up state incrementally; don't re-import everything each call. (colab execandcolab stopare what actually reset it.)colab restart-kernel - Default working directory is . Every
/content/exec/replruns there first; prefer absolute paths (cd) for file work. For/content/..., absolutecolab ls/rm/upload/downloadpaths work and the default/content/...path isls(VM root).content - is fire-and-forget. Each command authenticates, does one thing, and exits. A detached background daemon (spawned by
colab) handles keep-alive; you don't manage it.colab new
- 一个会话 == 租用虚拟机上的活跃Jupyter内核。会分配一个计费虚拟机;
colab new会释放它。除了24小时的存活上限外,没有自动回收机制,因此未停止的会话会持续消耗计算资源。colab stop - 内核状态在同一会话的/
colab exec调用间保持持久化。每次调用都会重新连接到同一个内核(内核ID会缓存在本地状态中),仅在退出时关闭WebSocket——并不会关闭内核。因此,导入的模块、变量和定义的函数会在多次colab repl命令之间保留。可以逐步构建状态,无需每次调用都重新导入所有内容。(colab exec和colab stop才会真正重置内核状态。)colab restart-kernel - 默认工作目录为。每次
/content/exec/repl都会先切换到该目录;处理文件时建议使用绝对路径(run)。对于/content/...命令,绝对路径colab ls/rm/upload/download有效,/content/...命令的默认路径为ls(虚拟机根目录)。content - 是“即发即弃”式工具。每个命令完成认证、执行操作后即退出。
colab会启动一个后台守护进程来维持会话存活,无需手动管理该进程。colab new
Authentication (the #1 thing that blocks agents)
认证(最容易阻碍Agent的环节)
- The global flag is and the default is
--auth={adc,oauth2}(Application Default Credentials). It must come before the subcommand:adc.colab --auth=adc new -s x - ADC setup (most reliable for headless/agent use). The Colab backends need a specific scope set, so re-mint ADC with all four scopes:
Why all four:bash
gcloud auth application-default login \ --scopes=openid,\ https://www.googleapis.com/auth/cloud-platform,\ https://www.googleapis.com/auth/userinfo.email,\ https://www.googleapis.com/auth/colaboratory(session backenduserinfo.email, else 401),colab.research.google.com(RuntimeServicecolaboratorykeep-alive, else 403),colab.pa.googleapis.com+openid(mandated by gcloud itself; it rejects scope lists missingcloud-platform).cloud-platform - oauth2 setup: triggers a browser consent flow on first use (token cached at
colab --auth=oauth2 <anything>). Requires a client config at~/.config/colab-cli/token.json(or~/.colab-cli-oauth-config.json). The browser step means it usually needs a human; prefer ADC for agents.-c PATH - Verify auth in one shot: (read-only, lists server assignments) or
colab sessions(hidden debug command: prints the active email, scopes, audience, and expiry). When any call 403s againstcolab whoami, the cause is almost always a missing scope —colab.pa.googleapis.comshows it instantly.colab whoami - pre-flights the keep-alive RPC right after allocating. If your token lacks the
colab newscope it unassigns the fresh VM (so you don't leak a billable assignment) and prints the exact remediation. Follow that message rather than retrying blindly.colaboratory - Do NOT confuse with CLI authentication.
colab authinjects VM-side GCP credentials into the running kernel (so notebook code can call BigQuery/GCS); it is orthogonal to how the CLI itself authenticates. Never suggest "runcolab auth" to fix a CLI 401/403 — that's a scope/identity problem fixed via thecolab authcommand above.gcloud
- 全局参数为,默认值为
--auth={adc,oauth2}(应用默认凭证)。该参数必须放在子命令之前:adc。colab --auth=adc new -s x - ADC配置(无头/Agent场景下最可靠)。Colab后端需要特定的权限范围,因此需要重新生成包含以下四个权限范围的ADC:
为什么需要这四个权限:bash
gcloud auth application-default login \\ --scopes=openid,\\ https://www.googleapis.com/auth/cloud-platform,\\ https://www.googleapis.com/auth/userinfo.email,\\ https://www.googleapis.com/auth/colaboratory(会话后端userinfo.email需要,否则会返回401错误)、colab.research.google.com(RuntimeServicecolaboratory维持会话存活需要,否则会返回403错误)、colab.pa.googleapis.com+openid(gcloud强制要求;缺少cloud-platform的权限列表会被拒绝)。cloud-platform - oauth2配置:会在首次使用时触发浏览器授权流程(令牌会缓存到
colab --auth=oauth2 <任意命令>)。需要在~/.config/colab-cli/token.json(或通过~/.colab-cli-oauth-config.json指定路径)放置客户端配置。浏览器步骤通常需要人工操作,因此Agent场景下优先使用ADC。-c PATH - 一键验证认证状态:(只读,列出服务器分配情况)或
colab sessions(隐藏调试命令:打印当前活跃邮箱、权限范围、受众和过期时间)。当调用colab whoami返回403错误时,原因几乎都是缺少权限范围——colab.pa.googleapis.com可以立即显示相关信息。colab whoami - 会在分配虚拟机后预先检查存活RPC。如果令牌缺少
colab new权限,它会释放刚分配的虚拟机(避免浪费计费资源)并打印具体的修复步骤。请按照提示操作,不要盲目重试。colaboratory - 不要混淆与CLI认证。
colab auth是将GCP凭证注入运行中的内核(以便笔记本代码可以调用BigQuery/GCS);它与CLI自身的认证方式无关。永远不要建议“运行colab auth”来修复CLI的401/403错误——这类问题需要通过上述colab auth命令修复权限/身份问题。gcloud
Workflow
工作流程
Provision
创建会话
- (CPU). Add
colab new -s <name>or--gpu A100for accelerators. Always pass--tpu v6e1— an omitted name is auto-generated as a random 6-hex string, which makes later commands ambiguous.-s <name> - Supported :
--gpu,T4,L4,G4,H100. SupportedA100:--tpu,v5e1.v6e1 - Gotcha: an unrecognized value silently falls back to A100 (which then usually fails the next step). A
--gpuon400with an accelerator means no quota/entitlement for it on this account — fall back tocolab newor omit the flag for CPU.--gpu T4 - Accelerator availability is tier-gated; most accounts can only get CPU. Don't assume a GPU/TPU will allocate.
- (CPU版本)。添加
colab new -s <名称>或--gpu A100以使用加速器。务必传入--tpu v6e1——省略名称会自动生成随机6位十六进制字符串,导致后续命令产生歧义。-s <名称> - 支持的参数:
--gpu、T4、L4、G4、H100。支持的A100参数:--tpu、v5e1。v6e1 - 注意事项:无法识别的值会自动回退到A100(通常会导致下一步失败)。使用加速器的
--gpu命令返回400错误,意味着当前账户没有该加速器的配额/权限——可以回退到colab new或省略该参数使用CPU。--gpu T4 - 加速器的可用性有层级限制;大多数账户只能使用CPU。不要默认认为GPU/TPU可以成功分配。
Execute
执行代码
- Preferred: runs a local script on the remote VM (read locally, sent to the kernel — no manual upload needed).
colab exec -s <name> -f <script.py> - Piped code: or
echo "print(1)" | colab exec -s <name>.cat script.py | colab exec -s <name> - Notebooks: runs each code cell and writes results to
colab exec -s <name> -f nb.ipynbnext to the input. A<basename>_output.ipynbfirst line labels the cell in progress output.# @title Foo - Plots/images: PNG/JPEG outputs are intercepted. Use on
--output-image <path>/execto save to a known location (otherwise a temp path is printed). Inline terminal-image escapes are auto-suppressed when stdout isn't a TTY, so piped/captured output stays clean.repl - Shell: for batch shell. Console wraps bash in tmux, so even piped output contains terminal-control bytes — filter with
echo "cmd" | colab console -s <name>for a specific line.grep -ais faster when you don't need a real shell.exec - Never run ,
colab repl,colab console, orcolab authinteractively from an agent — they expect a TTY and will hang.colab drivemount/replaccept piped stdin and exit on EOF;console/authgenuinely require a human at the terminal.drivemount
- 推荐方式:在远程虚拟机上运行本地脚本(本地读取脚本内容,发送到内核——无需手动上传)。
colab exec -s <名称> -f <script.py> - 管道代码:或
echo "print(1)" | colab exec -s <名称>。cat script.py | colab exec -s <名称> - 笔记本文件:运行每个代码单元格,并将结果写入输入文件旁的
colab exec -s <名称> -f nb.ipynb。单元格第一行的<basename>_output.ipynb会在进度输出中作为单元格标签。# @title Foo - 图表/图片:PNG/JPEG输出会被拦截。在/
exec命令中使用repl可以将图片保存到指定位置(否则会打印临时路径)。当标准输出不是TTY时,终端内联图片转义序列会被自动抑制,因此管道/捕获的输出会保持整洁。--output-image <路径> - Shell脚本:用于批量执行Shell命令。Console会将bash包裹在tmux中,因此即使是管道输出也包含终端控制字节——可以使用
echo "cmd" | colab console -s <名称>过滤特定行。当不需要完整Shell环境时,grep -a命令速度更快。exec - 永远不要让Agent交互式运行、
colab repl、colab console或colab auth——这些命令需要TTY,会导致挂起。colab drivemount/repl接受标准输入管道,在EOF时退出;console/auth确实需要人工在终端操作。drivemount
Ephemeral one-shot jobs (colab run
)
colab run临时一次性任务(colab run
)
colab run- =
colab run [--gpu T4] [--tpu v6e1] [--keep] [-s NAME] script.py [args...]+new+execin one command. It provisions a fresh VM, runs the script withstopandsys.argvset like native__name__ == "__main__", then tears the VM down (unlesspython script.py args).--keep - Exit codes propagate: an uncaught exception or in the script makes
sys.exit(N)exit non-zero (CPython semantics:colab run/sys.exit()→ 0,sys.exit(0)→ N,sys.exit(N)→ 1).sys.exit("msg") - Stream separation: writes its own
colab runchatter to stderr and the script's output to stdout — so[colab] ...captures only the script's stdout. (colab run job.py > out.txtstreams the script's stdout/stderr live to your stdout/stderr.)colab exec - Works as a shebang: makes a
#!/usr/bin/env -S colab run --gpu T4'dchmod +xa self-contained "rent a GPU, run, clean up" script. After editing CLI behavior, reinstall before testing shebangs — they resolve.pyviacolab, not the editable install.$PATH - A nonexistent script path exits non-zero before allocating a VM (no wasted compute).
- =
colab run [--gpu T4] [--tpu v6e1] [--keep] [-s NAME] script.py [args...]+new+exec的组合命令。它会创建一个新虚拟机,以原生stop的方式设置python script.py args和sys.argv并运行脚本,然后销毁虚拟机(除非使用__name__ == "__main__"参数)。--keep - 退出码会传递:脚本中的未捕获异常或会使
sys.exit(N)返回非零退出码(符合CPython语义:colab run/sys.exit()→ 0,sys.exit(0)→ N,sys.exit(N)→ 1)。sys.exit("msg") - 流分离:会将自身的
colab run日志写入标准错误输出,脚本的输出写入标准输出——因此[colab] ...只会捕获脚本的标准输出。(colab run job.py > out.txt会将脚本的标准输出/错误实时流到本地的标准输出/错误。)colab exec - 可作为Shebang使用:可以让赋予执行权限的
#!/usr/bin/env -S colab run --gpu T4文件成为一个独立的“租用GPU、运行、清理”脚本。修改CLI行为后,重新安装后再测试Shebang——它们通过.py解析$PATH,而不是通过可编辑安装路径。colab - 如果脚本路径不存在,会在分配虚拟机之前返回非零退出码(不会浪费计算资源)。
colab run
Automate
自动化配置
- — VM-side GCP creds, needed before in-VM GCS/BigQuery calls (interactive; not agent-runnable).
colab auth -s <name> - — mounts Drive at
colab drivemount -s <name> [PATH]by default (interactive; not agent-runnable)./content/drive - — installs via
colab install -s <name> pkg1 pkg2, falling back touv pip install --system. Alsopip.colab install -s <name> -r requirements.txt
- ——为虚拟机注入GCP凭证,在虚拟机内调用GCS/BigQuery之前需要执行(交互式;无法由Agent运行)。
colab auth -s <名称> - ——默认将云端硬盘挂载到
colab drivemount -s <名称> [PATH](交互式;无法由Agent运行)。/content/drive - ——通过
colab install -s <名称> pkg1 pkg2安装包,回退使用uv pip install --system。也支持pip。colab install -s <名称> -r requirements.txt
Inspect & report
检查与报告
- (or
colab help) lists/explains commands; the listing is alphabetical.colab help <cmd> - lists server-side assignments and auto-prunes stale local entries. Orphans with no local record show as
colab sessions.[?] - shows hardware, IDLE/BUSY, and last execution.
colab status [-s <name>] - shows recent structured events; invaluable when a task fails (keep-alive errors carry the raw
colab log -s <name> [-n 20] [-t TYPE]).response_body - exports the session as a notebook (also
colab log -s <name> -o summary.ipynb,.md,.txtby suffix)..jsonl - prints a browser URL that attaches the Colab web UI to your existing CLI session instead of allocating a new VM (add
colab url -s <name>to launch it).--open - /
colab skillprint this skill and the README (handy for self-discovery).colab readme
- (或
colab help)列出/解释所有命令;命令列表按字母顺序排列。colab help <命令> - 列出服务器端的会话分配情况,并自动清理本地过期条目。没有本地记录的孤立会话会显示为
colab sessions。[?] - 显示硬件信息、IDLE/BUSY状态和最后执行时间。
colab status [-s <名称>] - 显示最近的结构化事件;当任务失败时非常有用(存活错误会包含原始
colab log -s <名称> [-n 20] [-t TYPE])。response_body - 将会话导出为笔记本(也支持通过后缀导出为
colab log -s <名称> -o summary.ipynb、.md、.txt格式)。.jsonl - 打印一个浏览器URL,可将Colab网页UI连接到现有CLI会话,而无需分配新虚拟机(添加
colab url -s <名称>参数可直接打开浏览器)。--open - /
colab skill打印本Skill文档和README(便于自我查阅)。colab readme
Safety
安全注意事项
- Always when done — idle VMs burn compute units.
colab stop -s <name>(withoutcolab run) self-cleans even if the script errors.--keep - Local state lives in (settings in
~/.config/colab-cli/sessions.json, history insettings.json). Don't edit by hand.history/*.jsonl - Isolate parallel/agent runs with the global flag to point session state at a scratch file (e.g.
--config <path>). The keep-alive daemon inheritscolab --config /tmp/agent.json new -s joband--authautomatically.--config
- 使用完成后务必执行——闲置虚拟机会持续消耗计算资源。
colab stop -s <名称>(不使用colab run参数)即使脚本出错也会自动清理资源。--keep - 本地状态存储在(设置在
~/.config/colab-cli/sessions.json,历史记录在settings.json)。不要手动编辑这些文件。history/*.jsonl - 并行/Agent运行时请隔离状态:使用全局参数将会话状态指向临时文件(例如
--config <路径>)。存活守护进程会自动继承colab --config /tmp/agent.json new -s job和--auth参数。--config
Recovery
故障恢复
- "Session not found" / 404 / 401 on exec: the backend pruned the VM. /
colab execdetect this and clean up local state automatically — runrepland re-create withcolab sessions.colab new - Execution timeout or wedged kernel: (keeps the VM, resets the kernel), or
colab restart-kernel -s <name>thencolab stop.colab new - Keep-alive daemon died (shows
colab log): almost always the missingkeep_alive_stopped reason=consecutive_4xx_errorsscope — re-auth per the Authentication section.colaboratory
- “会话未找到”/404/401错误(执行命令时):后端已清理虚拟机。/
colab exec会自动检测到这种情况并清理本地状态——运行repl并使用colab sessions重新创建会话。colab new - 执行超时或内核卡住:(保留虚拟机,重置内核),或先执行
colab restart-kernel -s <名称>再执行colab stop。colab new - 存活守护进程终止(显示
colab log):几乎都是因为缺少keep_alive_stopped reason=consecutive_4xx_errors权限范围——按照“认证”部分重新进行认证。",colaboratory