Scikit Learn
Split data, cross-validate, tune hyperparameters, and pick models with scikit-learn while an agent drafts correct sklearn code.
Overview
Scikit-learn is an agent skill for the Build phase that teaches model selection, cross-validation, and hyperparameter evaluation patterns using scikit-learn.
Install
npx skills add https://github.com/k-dense-ai/scientific-agent-skills --skill scikit-learnWhat is this skill?
- train_test_split with stratify and three-way train/val/test splits
- KFold and StratifiedKFold cross-validation patterns with code
- Model selection tooling aligned to scikit-learn model_selection module
- Reproducibility via random_state=42 conventions throughout examples
- Comprehensive evaluation and hyperparameter tuning orientation
- Documents KFold with n_splits=5 and StratifiedKFold for imbalanced classification
Adoption & trust: 621 installs on skills.sh; 27.6k GitHub stars; 3/3 security scanners passed (skills.sh audits).
What problem does it solve?
You know you need proper splits and CV but keep reaching for leaky holdouts or inconsistent sklearn boilerplate.
Who is it for?
Indie ML builders validating classifiers or regressors in Python before promoting a model to inference or a batch API.
Skip if: Deep learning-only stacks with no sklearn, or teams that only need high-level product analytics without training code.
When should I use this skill?
Implementing or reviewing scikit-learn model selection, cross-validation, or hyperparameter tuning in Python ML workflows.
What do I get? / Deliverables
You get copy-paste-correct train/val/test and CV loops ready to wire into training jobs and comparison reports.
- Train/validation/test split code
- Cross-validation loop templates
- Model comparison evaluation scaffolding
Recommended Skills
Journey fit
Model selection and evaluation are core ML engineering tasks during product build when you train and compare predictors. The reference targets training pipelines and evaluation logic—backend/data work—not marketing or deploy checklists.
How it compares
Use this sklearn-focused reference instead of generic ML chat that omits stratified splits and fold discipline.
Common Questions / FAQ
Who is scikit-learn for?
Developers and solo data builders who train classical ML models in Python and want agent-generated sklearn evaluation code.
When should I use scikit-learn?
During Build when designing train-test splits, cross-validation, or hyperparameter search for scikit-learn estimators.
Is scikit-learn safe to install?
It is documentation-style procedural knowledge; review the Security Audits panel on this page and audit any generated training code before running on sensitive data.
SKILL.md
READMESKILL.md - Scikit Learn
# Model Selection and Evaluation Reference ## Overview Comprehensive guide for evaluating models, tuning hyperparameters, and selecting the best model using scikit-learn's model selection tools. ## Train-Test Split ### Basic Splitting ```python from sklearn.model_selection import train_test_split # Basic split (default 75/25) X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42) # With stratification (preserves class distribution) X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.25, stratify=y, random_state=42 ) # Three-way split (train/val/test) X_train, X_temp, y_train, y_temp = train_test_split(X, y, test_size=0.3, random_state=42) X_val, X_test, y_val, y_test = train_test_split(X_temp, y_temp, test_size=0.5, random_state=42) ``` ## Cross-Validation ### Cross-Validation Strategies **KFold** - Standard k-fold cross-validation - Splits data into k consecutive folds ```python from sklearn.model_selection import KFold kf = KFold(n_splits=5, shuffle=True, random_state=42) for train_idx, val_idx in kf.split(X): X_train, X_val = X[train_idx], X[val_idx] y_train, y_val = y[train_idx], y[val_idx] ``` **StratifiedKFold** - Preserves class distribution in each fold - Use for imbalanced classification ```python from sklearn.model_selection import StratifiedKFold skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42) for train_idx, val_idx in skf.split(X, y): X_train, X_val = X[train_idx], X[val_idx] y_train, y_val = y[train_idx], y[val_idx] ``` **TimeSeriesSplit** - For time series data - Respects temporal order ```python from sklearn.model_selection import TimeSeriesSplit tscv = TimeSeriesSplit(n_splits=5) for train_idx, val_idx in tscv.split(X): X_train, X_val = X[train_idx], X[val_idx] y_train, y_val = y[train_idx], y[val_idx] ``` **GroupKFold** - Ensures samples from same group don't appear in both train and validation - Use when samples are not independent ```python from sklearn.model_selection import GroupKFold gkf = GroupKFold(n_splits=5) for train_idx, val_idx in gkf.split(X, y, groups=group_ids): X_train, X_val = X[train_idx], X[val_idx] y_train, y_val = y[train_idx], y[val_idx] ``` **LeaveOneOut (LOO)** - Each sample used as validation set once - Use for very small datasets - Computationally expensive ```python from sklearn.model_selection import LeaveOneOut loo = LeaveOneOut() for train_idx, val_idx in loo.split(X): X_train, X_val = X[train_idx], X[val_idx] y_train, y_val = y[train_idx], y[val_idx] ``` ### Cross-Validation Functions **cross_val_score** - Evaluate model using cross-validation - Returns array of scores ```python from sklearn.model_selection import cross_val_score from sklearn.ensemble import RandomForestClassifier model = RandomForestClassifier(n_estimators=100, random_state=42) scores = cross_val_score(model, X, y, cv=5, scoring='accuracy') print(f"Scores: {scores}") print(f"Mean: {scores.mean():.3f} (+/- {scores.std() * 2:.3f})") ``` **cross_validate** - More comprehensive than cross_val_score - Can return multiple metrics and fit times ```python from sklearn.model_selection import cross_validate model = RandomForestClassifier(n_estimators=100, random_state=42) cv_results = cross_validate( model, X, y, cv=5, scoring=['accuracy', 'precision', 'recall', 'f1'], return_train_score=True, return_estimator=True # Returns fitted estimators ) print(f"Test accuracy: {cv_results['test_accuracy'].mean():.3f}") print(f"Test precision: {cv_results['test_precision'].mean():.3f}") print(f"Fit time: {cv_results['fit_time'].mean():.3f}s") ``` **cross_val_predict** - Get predictions for each sample when it was in validation set - Useful for analyzing errors ```python from sklearn.model_selection import cross_val_predict model = RandomForestClassifier(n_estimators=100, random_state=42) y_pred = cross_val_predict(model, X, y, cv=5) # Now can analyze predictions vs actual