Scikit-learn Best Practices

Code Style and Structure

• Write concise, technical responses with accurate Python examples
• Prioritize reproducibility in machine learning workflows
• Use functional programming for data pipelines
• Use object-oriented programming for custom estimators
• Prefer vectorized operations over explicit loops
• Follow PEP 8 style guidelines

Machine Learning Workflow

Data Preparation

• Always split data before any preprocessing: train/validation/test
• Use

  train_test_split()

  with

  random_state

  for reproducibility
• Stratify splits for imbalanced classification:

  stratify=y

• Keep test set completely separate until final evaluation

Feature Engineering

• Scale features appropriately for distance-based algorithms
• Use

  StandardScaler

  for normally distributed features
• Use

  MinMaxScaler

  for bounded features
• Use

  RobustScaler

  for data with outliers
• Encode categorical variables:

  OneHotEncoder

  ,

  OrdinalEncoder

  ,

  LabelEncoder

• Handle missing values:

  SimpleImputer

  ,

  KNNImputer

Pipelines

• Always use

  Pipeline

  to chain preprocessing and modeling
• Prevents data leakage by fitting transformers only on training data
• Makes code cleaner and more reproducible
• Enables easy deployment and serialization

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier

pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('classifier', RandomForestClassifier(random_state=42))
])

Column Transformers

• Use

  ColumnTransformer

  for different preprocessing per feature type
• Combine numeric and categorical preprocessing in single pipeline

Model Selection and Tuning

Cross-Validation

• Use cross-validation for reliable performance estimates

• cross_val_score()

  for quick evaluation

• cross_validate()

  for multiple metrics
• Use appropriate CV strategy:

  • KFold

    for regression

  • StratifiedKFold

    for classification

  • TimeSeriesSplit

    for temporal data

  • GroupKFold

    for grouped data

Hyperparameter Tuning

• Use

  GridSearchCV

  for exhaustive search
• Use

  RandomizedSearchCV

  for large parameter spaces
• Always tune on training/validation data, never test data
• Set

  n_jobs=-1

  for parallel processing

Model Evaluation

Classification Metrics

• Use appropriate metrics for your problem:

  • accuracy_score

    for balanced classes

  • precision_score

    ,

    recall_score

    ,

    f1_score

    for imbalanced

  • roc_auc_score

    for ranking ability
• Use

  classification_report()

  for comprehensive overview
• Examine

  confusion_matrix()

  for error analysis

Regression Metrics

• mean_squared_error

  (MSE) for general use

• mean_absolute_error

  (MAE) for interpretability

• r2_score

  for explained variance

Evaluation Best Practices

• Report confidence intervals, not just point estimates
• Use multiple metrics to understand model behavior
• Compare against meaningful baselines
• Evaluate on held-out test set only once, at the end

Handling Imbalanced Data

• Use stratified splitting and cross-validation
• Consider class weights:

  class_weight='balanced'

• Use appropriate metrics (F1, AUC-PR, not accuracy)
• Adjust decision threshold based on business needs

Feature Selection

• Use

  SelectKBest

  with statistical tests
• Use

  RFE

  (Recursive Feature Elimination)
• Use model-based selection:

  SelectFromModel

• Examine feature importances from tree-based models

Model Persistence

• Use

  joblib

  for saving and loading models
• Save entire pipelines, not just models
• Version control model artifacts
• Document model metadata

Performance Optimization

• Use

  n_jobs=-1

  for parallel processing where available
• Consider

  warm_start=True

  for iterative training
• Use sparse matrices for high-dimensional sparse data
• Consider incremental learning with

  partial_fit()

  for large data

Key Conventions

• Import from submodules:

  from sklearn.ensemble import RandomForestClassifier

• Set

  random_state

  for reproducibility
• Use pipelines to prevent data leakage
• Document model choices and hyperparameters