Workflow Instruction

Workflow

1. Locate Dataset:
   • The full path may be a local file path, or an S3 URI
   • Resolve the full path to the dataset file, make sure read permissions are available, and help the user if the file is not found
2. Determine strategy and model:
   • File formatting depends on the currently selected fine-tuning strategy and fine-tuning base model.
   • If the strategy and model are already known from the conversation context (e.g., selected via the finetuning-setup skill), use them.
   • If not available in context, activate the finetuning-setup skill to determine them before proceeding.
3. Check File Formatting: Run the tool format_detector.py to make sure the file conforms to formatting requirements.
   • Send the full path directly to the format_detector script as an argument
   • Do not send the model and strategy as arguments
   • Do not download data from S3
   • Do not make local copies of data
4. Summarize Results: Tell the user if their data is ready
   • Examine the output of format_detector and compare to the known strategy and model
   • Important: training datasets and evaluation datasets have different format requirements.
     • Training datasets must match the fine-tuning strategy format (SFT, DPO, RLVR) per

       references/strategy_data_requirements.md

     • Evaluation datasets (for model evaluation) must match one of the SageMaker evaluation dataset formats.
   • Report back to the user if their current dataset is valid for its intended purpose
   • Warn the user if their dataset is valid, but for a different strategy or model
   • Warn the user if their dataset is not valid for any strategy/model pair

Messages to the User

• Introduction: "This skill checks the structure of your dataset for model fine-tuning."
• File types: This skill applies to files that are formatted according to the Amazon SageMaker AI Developer Guide

Resources

• scripts/format_detector.py is self-contained format validation script that can be run independently
• finetuning-setup skill should have already determined the fine-tuning strategy and base model
• references/strategy_data_requirements.md contains data format requirements per strategy

Script Details

• scripts/format_detector.py is self-contained format validation script that can be run independently:

# With the file path argument identified in workflow step 1
python scripts/format_detector.py local_path/to/dataset