Machine Learning

Machine Learning

Comprehensive machine learning skill covering the full ML lifecycle from experimentation to production deployment.

When to Use This Skill

• Building machine learning pipelines
• Feature engineering and data preprocessing
• Model training, evaluation, and selection
• Hyperparameter tuning and optimization
• Model deployment and serving
• ML experiment tracking and versioning
• Production ML monitoring and maintenance

ML Development Lifecycle

1. Problem Definition

Classification Types:

• Binary classification (spam/not spam)
• Multi-class classification (image categories)
• Multi-label classification (document tags)
• Regression (price prediction)
• Clustering (customer segmentation)
• Ranking (search results)
• Anomaly detection (fraud detection)

Success Metrics by Problem Type:

2. Data Preparation

Data Quality Checks:

• Missing value analysis and imputation strategies
• Outlier detection and handling
• Data type validation
• Distribution analysis
• Target leakage detection

Feature Engineering Patterns:

• Numerical: scaling, binning, log transforms, polynomial features
• Categorical: one-hot, target encoding, frequency encoding, embeddings
• Temporal: lag features, rolling statistics, cyclical encoding
• Text: TF-IDF, word embeddings, transformer embeddings
• Geospatial: distance features, clustering, grid encoding

Train/Test Split Strategies:

• Random split (standard)
• Stratified split (imbalanced classes)
• Time-based split (temporal data)
• Group split (prevent data leakage)
• K-fold cross-validation

3. Model Selection

Algorithm Selection Guide:

4. Model Training

Hyperparameter Tuning:

• Grid Search: exhaustive, good for small spaces
• Random Search: efficient, good for large spaces
• Bayesian Optimization: smart exploration (Optuna, Hyperopt)
• Early stopping: prevent overfitting

Common Hyperparameters:

5. Model Evaluation

Evaluation Best Practices:

• Always use held-out test set for final evaluation
• Use cross-validation during development
• Check for overfitting (train vs validation gap)
• Evaluate on multiple metrics
• Analyze errors qualitatively

Handling Imbalanced Data:

• Resampling: SMOTE, undersampling
• Class weights: weighted loss functions
• Threshold tuning: optimize decision threshold
• Evaluation: use PR-AUC over ROC-AUC

6. Production Deployment

Model Serving Patterns:

• REST API (Flask, FastAPI, TF Serving)
• Batch inference (scheduled jobs)
• Streaming (real-time predictions)
• Edge deployment (mobile, IoT)

Production Considerations:

• Latency requirements (p50, p95, p99)
• Throughput (requests per second)
• Model size and memory footprint
• Fallback strategies
• A/B testing framework

7. Monitoring & Maintenance

What to Monitor:

• Prediction latency
• Input feature distributions (data drift)
• Prediction distributions (concept drift)
• Model performance metrics
• Error rates and types

Retraining Triggers:

• Performance degradation below threshold
• Significant data drift detected
• Scheduled retraining (daily, weekly)
• New training data available

MLOps Best Practices

Experiment Tracking

Track for every experiment:

• Code version (git commit)
• Data version (hash or version ID)
• Hyperparameters
• Metrics (train, validation, test)
• Model artifacts
• Environment (packages, versions)

Model Versioning

models/
├── model_v1.0.0/
│   ├── model.pkl
│   ├── metadata.json
│   ├── requirements.txt
│   └── metrics.json
├── model_v1.1.0/
└── model_v2.0.0/

CI/CD for ML

1. Continuous Integration:

• Data validation tests
• Model training tests
• Performance regression tests

2. Continuous Deployment:

• Staging environment validation
• Shadow mode testing
• Gradual rollout (canary)
• Automatic rollback

Reference Files

For detailed patterns and code examples, load reference files as needed:

• `references/preprocessing.md` - Data preprocessing patterns and feature engineering techniques
• `references/model_patterns.md` - Model architecture patterns and implementation examples
• `references/evaluation.md` - Comprehensive evaluation strategies and metrics

Integration with Other Skills

• performance - For optimizing inference latency
• testing - For ML-specific testing patterns
• database-optimization - For feature store queries
• debugging - For model debugging and error analysis

Data Preprocessing Patterns

Missing Value Strategies

Numerical Features

# Simple imputation
df['col'].fillna(df['col'].median(), inplace=True)

# Model-based imputation
from sklearn.impute import KNNImputer
imputer = KNNImputer(n_neighbors=5)
df_imputed = imputer.fit_transform(df)

Categorical Features

# Mode imputation
df['col'].fillna(df['col'].mode()[0], inplace=True)

# New category for missing
df['col'].fillna('MISSING', inplace=True)

Feature Scaling

from sklearn.preprocessing import StandardScaler, MinMaxScaler, RobustScaler

# Standard scaling (mean=0, std=1) - use for most models
scaler = StandardScaler()

# MinMax scaling [0,1] - use for neural networks
scaler = MinMaxScaler()

# Robust scaling - use when outliers present
scaler = RobustScaler()

X_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

Categorical Encoding

from sklearn.preprocessing import LabelEncoder, OneHotEncoder
import category_encoders as ce

# One-hot encoding (low cardinality)
encoder = OneHotEncoder(sparse=False, handle_unknown='ignore')

# Target encoding (high cardinality)
encoder = ce.TargetEncoder(cols=['category_col'])

# Frequency encoding
df['col_freq'] = df['col'].map(df['col'].value_counts(normalize=True))

Feature Engineering Examples

Temporal Features

df['hour'] = df['timestamp'].dt.hour
df['day_of_week'] = df['timestamp'].dt.dayofweek
df['is_weekend'] = df['day_of_week'].isin([5, 6]).astype(int)
df['month'] = df['timestamp'].dt.month

# Cyclical encoding
df['hour_sin'] = np.sin(2 * np.pi * df['hour'] / 24)
df['hour_cos'] = np.cos(2 * np.pi * df['hour'] / 24)

Numerical Transformations

# Log transform (right-skewed data)
df['col_log'] = np.log1p(df['col'])

# Box-Cox transform
from scipy.stats import boxcox
df['col_bc'], lambda_param = boxcox(df['col'] + 1)

# Polynomial features
from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(degree=2, include_bias=False)
X_poly = poly.fit_transform(X)

Interaction Features

# Manual interactions
df['feature_interaction'] = df['feat1'] * df['feat2']
df['feature_ratio'] = df['feat1'] / (df['feat2'] + 1e-8)

# Automated interactions
from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(degree=2, interaction_only=True)

Outlier Handling

# IQR method
Q1 = df['col'].quantile(0.25)
Q3 = df['col'].quantile(0.75)
IQR = Q3 - Q1
lower_bound = Q1 - 1.5 * IQR
upper_bound = Q3 + 1.5 * IQR
df_clean = df[(df['col'] >= lower_bound) & (df['col'] <= upper_bound)]

# Clipping
df['col_clipped'] = df['col'].clip(lower=lower_bound, upper=upper_bound)

# Z-score method
from scipy import stats
z_scores = np.abs(stats.zscore(df['col']))
df_clean = df[z_scores < 3]

Data Leakage Prevention

Common Leakage Sources:

1. Future information in features 2. Target information in features 3. Train-test contamination during preprocessing

Prevention:

# Always fit on train, transform on test
scaler.fit(X_train)
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)

# Use pipelines
from sklearn.pipeline import Pipeline
pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('model', LogisticRegression())
])
pipeline.fit(X_train, y_train)