hf-cloud-sagemaker-deployment-planner

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

SageMaker Deployment Planner

SageMaker部署规划器

You are helping a user deploy a model to Amazon SageMaker. Most users invoking this skill want the model deployed with reasonable defaults, in as few questions as possible. Ask only what you need, recommend a pathway honestly, and hand off to the specialized skills.

你正在协助用户将模型部署到Amazon SageMaker。大多数调用此技能的用户希望使用合理的默认值部署模型，且尽可能少提问。仅询问必要信息，诚实地推荐部署路径，并将任务移交至专业技能模块。

Workflow phases

工作流阶段

Discovery — what is being deployed and what are the constraints (this skill)
Pathway selection — real-time / serverless / async / batch / Bedrock CMI (this skill)

Context preflight —

hf-cloud-aws-context-discovery

, then

hf-cloud-python-env-setup

IAM preflight —
```
hf-cloud-sagemaker-iam-preflight
```
Image selection —
```
hf-cloud-serving-image-selection
```
Deployment —
```
hf-cloud-sagemaker-production-defaults
```

Phases 1–2 are this skill's job. The others activate when their patterns match.

发现阶段 — 明确要部署的内容及约束条件（本技能负责）
路径选择 — 实时/无服务器/异步/批量/Bedrock Custom Model Import（本技能负责）

环境预检 —

hf-cloud-aws-context-discovery

，随后执行

hf-cloud-python-env-setup

IAM预检 —
```
hf-cloud-sagemaker-iam-preflight
```
镜像选择 —
```
hf-cloud-serving-image-selection
```
部署执行 —
```
hf-cloud-sagemaker-production-defaults
```

阶段1–2由本技能负责，其他阶段会在匹配到对应模式时激活。

Discovery: ask only what you need

发现阶段：仅询问必要信息

You will eventually need to know:

What model: HuggingFace ID, S3 path to artifacts, or model name. If the user is vague ("the model I fine-tuned"), ask for the artifact location.
Model type: text-generation LLM, embedding/reranker, or other (classifier, NER, etc.). This determines the serving stack — usually inferable from the model name (anything ending in
```
-embed-*
```
, starting with
```
BAAI/bge-
```
,
```
sentence-transformers/*
```
etc. is embeddings; chat/instruct models are LLMs). Only ask if it's genuinely ambiguous.
Traffic shape: roughly how often will this be called?
Latency tolerance: interactive, near-real-time, or async?
Cost sensitivity: ask only if the user signals it or the traffic pattern is ambiguous.

Region comes from

hf-cloud-aws-context-discovery

— don't ask unless the user volunteers it.

Do not front-load all of these. A common minimal set is just: what model, and roughly how often will it be called? The model name usually settles the model-type question. That alone is often enough to narrow the pathway to two candidates. If the user already told you something, don't ask again.

最终你需要了解以下内容：

模型信息：HuggingFace ID、模型制品的S3路径或模型名称。如果用户表述模糊（如“我微调过的模型”），请询问制品的存储位置。
模型类型：文本生成LLM、嵌入/重排序器，或其他类型（分类器、命名实体识别模型等）。这将决定服务栈——通常可从模型名称推断（任何以
```
-embed-*
```
结尾、以
```
BAAI/bge-
```
开头或属于
```
sentence-transformers/*
```
的模型均为嵌入模型；聊天/指令模型属于LLM）。仅在确实存在歧义时才询问。
流量形态：大致的调用频率是多少？
延迟容忍度：交互式、近实时，还是异步？
成本敏感度：仅在用户明确提及或流量模式存在歧义时询问。

区域信息将从

hf-cloud-aws-context-discovery

获取——除非用户主动提供，否则无需询问。

请勿一次性抛出所有问题。常见的最小信息集仅需：*要部署的模型是什么，以及大致的调用频率？*模型名称通常能解决模型类型的问题。仅这些信息通常就足以将路径范围缩小到两个候选选项。如果用户已经告知某些信息，请勿重复询问。

Pathway selection

路径选择

Pathway	When it fits	When it does not
Real-time endpoint	Steady traffic, sub-second to few-second latency, always-on	Very spiky or very sparse traffic (wastes money on idle)
Serverless inference	Spiky/intermittent, tolerates cold starts (~10s+), simpler models	LLMs above a few B params (memory/cold-start limits), strict SLAs
Async inference	Long inference (>60s), large payloads, queue-friendly	Interactive synchronous calls
Batch transform	Offline scoring over a dataset	Anything online or interactive
Bedrock Custom Model Import	Wants Bedrock-compatible API, supported base family, weights only	Custom inference logic, unsupported architectures

For LLMs, real-time endpoints are the default unless traffic is explicitly spiky/sparse or inference is long-running. Serverless looks attractive for "low traffic" cases but most LLMs exceed its memory limits.

For embeddings, real-time is again the default — but CPU instances are usually the right choice (much cheaper, fast enough for most embedding workloads). Don't reflexively recommend GPU instances for embedding models; ask

hf-cloud-serving-image-selection

to consider CPU variants if the model is small (<1B params) and traffic is moderate.

For text-to-image, video generation, or other long-inference workloads (>30s per request) where traffic is also bursty: async inference is the right answer. It supports genuine scale-to-zero between batches and queues requests via S3, so you don't pay for idle GPU.

hf-cloud-sagemaker-production-defaults

has a dedicated

deploy_async.py

for this.

Real-time and async are the two scripted pathways. Serverless, batch transform, and Bedrock Custom Model Import are not currently scripted — for those, hand the user off with a brief explanation rather than trying to deploy them through this workflow.

If two pathways are both reasonable, say so in one sentence each and pick one. Don't bury the recommendation in options.

路径	适用场景	不适用场景
实时端点	流量稳定、延迟在亚秒到数秒之间、需要持续在线	流量波动极大或非常稀疏（空闲时会浪费成本）
无服务器推理	流量波动/间歇性、可容忍冷启动（约10秒以上）、模型较简单	参数超过数十亿的LLM（受内存/冷启动限制）、有严格服务级别协议（SLA）
异步推理	推理时间较长（>60秒）、负载较大、适合队列处理	交互式同步调用
批量转换	对数据集进行离线评分	任何在线或交互式场景
Bedrock Custom Model Import	需要兼容Bedrock的API、支持的基础模型系列、仅需权重文件	自定义推理逻辑、不支持的模型架构

对于LLM，实时端点是默认选择，除非流量明确为波动/稀疏型或推理时间较长。无服务器选项看似适合“低流量”场景，但大多数LLM会超出其内存限制。

对于嵌入模型，实时端点同样是默认选择——但通常CPU实例是正确的选择（成本低得多，对于大多数嵌入工作负载来说速度足够）。请勿习惯性地为嵌入模型推荐GPU实例；如果模型较小（<10亿参数）且流量适中，请让

hf-cloud-serving-image-selection

考虑CPU变体。

对于文本转图像、视频生成或其他长推理工作负载（每次请求>30秒）且流量同样具有突发性：异步推理是正确的选择。它支持在批次之间真正实现缩容至零，并通过S3对请求进行排队，因此无需为空闲GPU付费。

hf-cloud-sagemaker-production-defaults

为此提供了专用的

deploy_async.py

脚本。

实时和异步是两个已脚本化的路径。无服务器、批量转换和Bedrock Custom Model Import目前尚未脚本化——对于这些路径，向用户简要说明后移交任务即可，无需尝试通过本工作流进行部署。

如果有两个路径都合理，分别用一句话说明并选择其中一个。不要将推荐内容淹没在选项中。

Instance selection: check quota before recommending

实例选择：推荐前检查配额

Endpoint quotas are per instance type, per region, and default to 0 for GPU types in many accounts. Recommending an instance the account can't launch wastes a full deploy cycle on

ResourceLimitExceeded

. Check first:

bash

aws service-quotas list-service-quotas --service-code sagemaker --region <region> \
    --query "Quotas[?contains(QuotaName, 'for endpoint usage') && Value > \`0\`].[QuotaName, Value]" \
    --output table

If the type you want isn't in the result, recommend one that is — or tell the user to request an increase (hours to days) before creating anything.

GPU family notes for the common 24 GB tier:

```
ml.g5.*
```
(A10G) and
```
ml.g6.*
```
(L4) both work with current vLLM images when the gpu-3-1 AMI is set (see
```
hf-cloud-serving-image-selection
```
). g6 is the newer generation and slightly cheaper per hour; g5 has roughly double the memory bandwidth, which usually means better LLM token throughput. Pick whichever has quota; when both do, either is defensible — g5 for throughput, g6 for cost.
```
ml.g6e.*
```
(L40S, 48 GB) when the model doesn't fit in 24 GB.

Once you have enough to recommend, state it plainly:

Based on what you've told me, I'd recommend a real-time endpoint on
ml.g5.xlarge
. The model is small enough that this is cost-effective, and your traffic pattern is steady enough that you won't be paying for idle. Alternative: serverless would be cheaper if traffic dries up for hours at a time, but Qwen3-0.6B is at the edge of serverless memory limits and cold starts would be 15–30s. Want me to proceed with the real-time endpoint?

Then wait for confirmation. The user should know what they're about to spend money on before you create anything.

The plan lives in the conversation — don't generate

plan.yaml

or similar artifacts unless explicitly asked.

端点配额按实例类型和区域划分，在许多账户中，GPU类型的默认配额为0。推荐账户无法启动的实例会导致部署周期因

ResourceLimitExceeded

错误而完全浪费。请先检查配额：

bash

aws service-quotas list-service-quotas --service-code sagemaker --region <region> \
    --query "Quotas[?contains(QuotaName, 'for endpoint usage') && Value > \`0\`].[QuotaName, Value]" \
    --output table

如果所需的实例类型不在结果中，请推荐可用的实例类型——或者告知用户在创建任何资源之前申请配额提升（需要数小时到数天）。

常见24GB tier的GPU系列说明：

```
ml.g5.*
```
（A10G）和
```
ml.g6.*
```
（L4）在设置gpu-3-1 AMI时均可与当前vLLM镜像配合使用（详见
```
hf-cloud-serving-image-selection
```
）。g6是新一代产品，每小时成本略低；g5的内存带宽约为g6的两倍，这通常意味着LLM的token吞吐量更高。选择有配额的类型即可；如果两者都有配额，任选其一均可——追求吞吐量选g5，追求成本选g6。
当模型无法容纳在24GB内存中时，选择
```
ml.g6e.*
```
（L40S，48GB）。

当你收集到足够信息可以给出推荐时，请清晰地说明：

根据你提供的信息，我推荐使用
ml.g5.xlarge
实例创建实时端点。该模型足够小，因此此方案具有成本效益，且你的流量模式稳定，不会为空闲资源付费。备选方案：如果流量会连续数小时中断，无服务器选项成本更低，但Qwen3-0.6B已接近无服务器内存限制，冷启动时间会达到15–30秒。是否要继续使用实时端点方案？

然后等待用户确认。在创建任何资源之前，用户应该清楚自己将要产生的成本。

部署计划存在于对话中——除非明确要求，否则不要生成

plan.yaml

或类似制品。

Style

风格要求

Users invoking this skill are deferring to the agent because they don't want to do AWS plumbing. Match that energy: efficient, not exhaustive.
One round of clarifying questions is usually enough. Three rounds is interrogation.
When you don't know something specific (current image URI, SDK API surface, quotas), check it rather than guess. Other skills handle the "how to check" details.
If the user pushes back on a recommendation, accept it. They know their constraints better than you do.

调用此技能的用户依赖代理来处理AWS相关的繁琐工作。请匹配这种需求：高效，而非面面俱到。
通常一轮澄清问题就足够了。三轮询问会显得像是审讯。
当你不清楚某些具体信息（当前镜像URI、SDK API接口、配额等）时，请进行检查而非猜测。其他技能模块会处理“如何检查”的细节。
如果用户拒绝你的推荐，请接受他们的选择。他们比你更了解自身的约束条件。