finetuning-setup

Original：🇺🇸 English
Translated
2 scriptsChecked / no sensitive code detected

Selects a base model and fine-tuning technique (SFT, DPO, or RLVR) for the user's use case by querying SageMaker Hub. Use when the user asks which model or technique to use, wants to start fine-tuning, or mentions a model name or family (e.g., "Llama", "Mistral") — always activate even for known model names because the exact Hub model ID must be resolved. Queries available models, validates technique compatibility, and confirms selections.

3installs
Sourceawslabs/agent-plugins
Added on

NPX Install

npx skill4agent add awslabs/agent-plugins finetuning-setup

Tags

Translated version includes tags in frontmatter
fine-tuningsagemakerllmmodel-selectiontraining-techniques

SKILL.md Content

View Translation Comparison →

Finetuning Setup

Guides the user through selecting a base model and fine-tuning technique based on their use case.

When to Use

  • User asks which fine-tuning technique to use
  • User wants to select or change their base model
  • User mentions a model name or family (e.g., "Llama", "Mistral") — the exact Hub model ID still needs to be resolved

Prerequisites

  • A 
    use_case_spec.md
    file exists. If not, activate the use-case-specification skill to generate it first.

Workflow

Step 1: Discover Hub

  1. List all available SageMaker Hubs in the user's region by calling the SageMaker 
    ListHubs
    API using the 
    aws___call_aws
    tool.
  2. From the results, filter out any hub whose 
    HubDescription
    contains "AI Registry" — these do not contain JumpStart models.
  3. The remaining hubs are eligible (e.g., 
    SageMakerPublicHub
    and any private hubs).
  4. If exactly one eligible hub exists, use it automatically — do not ask the user.
  5. If multiple eligible hubs exist, present them to the user and ask which one to use. Example:
    I found the following model hubs:
- SageMakerPublicHub — SageMaker Public Hub
- Private-Hub-XYZ — Private Hub models
Which hub would you like to use?
  6. Store the selected hub name for use in subsequent steps.

Step 2: Select Base Model

  1. Read 
    use_case_spec.md
    to understand the use case and success criteria.
  2. Restate the use case in one sentence.
  3. Always retrieve the full list of available SageMaker Hub model names by running: 
    python finetuning-setup/scripts/get_model_names.py <hub-name>
    — even if the user has already mentioned a model name or family. Do not skip this step or filter the results.
  4. Present all available models to the user, grouped by model family (e.g., Llama, Mistral, Gemma) for readability.
  5. Ask the user to pick the exact model ID from the list.
  6. Validate the selected model exists in the retrieved list before proceeding.
EXTREMELY IMPORTANT: NEVER recommend or suggest any particular model based on the context you have. YOU ARE ALLOWED ONLY to display the list of models as given by the script. DO NOT add your own recommendation or suggestion after displaying the list of models to tell which model is correct. Present this statement to the user: "Which model would you like to use? Please type the exact model name from the above list." and allow the user to select the model.

Step 3: Determine Finetuning Technique

  1. Consult 
    references/finetune_technique_selection_guide.md
    and recommend the best-fit technique (SFT, DPO, or RLVR) for the use case. Present the recommendation and reasoning to the user.
  2. Ask the user if they'd like to go with the recommendation or prefer a different technique.
  3. Once the user confirms a technique, retrieve the finetuning techniques available for the selected model by running: 
    python finetuning-setup/scripts/get_recipes.py <model-name> <hub-name>
    • This returns only the techniques the model actually supports, filtered to SFT, DPO, and RLVR. Only these three techniques are supported — ignore any other techniques even if the model's recipes include them.
  4. If the chosen technique is available for the model, proceed to Step 4.
  5. If the chosen technique is not available for the model, explain that the selected model does not support it on SageMaker and offer to go back to Step 2 to pick a different model that supports the chosen technique.

Step 4: Confirm Selections

Present a summary to the user:
Here's what we've selected:
- Base model: [model name]
- Fine-tuning technique: [SFT/DPO/RLVR]

References

  • references/finetune_technique_selection_guide.md
    — Technique guidance