Microsoft Foundry
- 487k installs
- 1.3k repo stars
- Updated July 26, 2026
- microsoft/azure-skills
microsoft-foundry is an agent skill that runs Azure AI Foundry fine-tuning workflows for SFT, DPO, and RFT with scripts for data validation, training, deployment, and evaluation.
About
The microsoft-foundry skill guides fine-tuning on Azure AI Foundry with supervised fine-tuning, direct preference optimization, and reinforcement fine-tuning with graders. It spans dataset preparation, validation, training job submission, monitoring, grader calibration, deployment, evaluation, and cleanup across quickstart, full pipeline, dataset creation, iterative training, and diagnose-poor-results workflows. Bundled Python scripts cover submit_training, monitor_training, calibrate_grader, check_training curves, deploy_model via ARM REST, evaluate_model with an LLM judge, convert_dataset formats, generate distillation data, score_dataset quality, and cleanup resources. Five rules insist on baseline evaluation first, validated data before submit, RFT grader calibration targeting 25-50% base-model failure, checkpoint evaluation before deploy, and token cost measurement alongside accuracy. The skill explicitly excludes general deployment without fine-tuning, agent creation, and prompt-only optimization, routing those to sibling azure-skills modules. Error handling documents API version issues, globalStandard training types, RFT endpoint scaling, deployment race conditions, and con.
- Supports SFT, DPO, and RFT fine-tuning with dataset prep, training, deployment, and evaluation pipelines.
- Ships Python scripts for submit, monitor, calibrate_grader, check_training, deploy, evaluate, and cleanup.
- Validators under scripts/validate cover SFT, DPO, and RFT data formats before job submission.
- Five training rules include baseline first, data validation, RFT grader calibration, and checkpoint evaluation.
- Documents platform gotchas: openai SDK version, globalStandard OSS models, RFT S2+ scaling, and deployment races.
Microsoft Foundry by the numbers
- 486,627 all-time installs (skills.sh)
- Security screen: MEDIUM risk (skills.sh audit)
- Data as of Jul 28, 2026 (Skillselion catalog sync)
microsoft-foundry capabilities & compatibility
Free skill; requires a paid Azure AI Foundry subscription and Azure MCP.
- Capabilities
- sft, dpo, and rft training type selection · dataset validation and format conversion · training job submit and monitor scripts · rft grader calibration and reward hacking guidan · arm rest model deployment · llm judge evaluation and training curve analysis · synthetic distillation data generation · resource cleanup for files and deployments
- Works with
- azure · openai
- Use cases
- api development · data analysis · orchestration
- Runs
- Local or remote
- Pricing
- Bring your own API key
What microsoft-foundry says it does
Provision Azure resources when needed, deploy the agent, and smoke-test it.
For **hosted agents** (custom container or code), use `azd deploy`. Prefer **direct code deployment through azd** (no Docker/ACR required)
Each successful `azd deploy` creates an immutable agent version
npx skills add https://github.com/microsoft/azure-skills --skill microsoft-foundryAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 487k |
|---|---|
| repo stars | ★ 1.3k |
| Security audit | 2 / 3 scanners passed |
| Last updated | July 26, 2026 |
| Repository | microsoft/azure-skills ↗ |
How do I fine-tune an Azure AI Foundry model with the right training type, validated data, grader calibration, and safe deployment?
Fine-tune Azure AI Foundry models with SFT, DPO, or RFT using validated datasets, training scripts, deployment, and evaluation.
Who is it for?
Developers training custom Azure AI Foundry models who need scripted SFT, DPO, or RFT pipelines with validation and evaluation.
Skip if: Skip for general model deploy without training, agent creation, or prompt optimization without a training job.
When should I use this skill?
Fine-tune, SFT, DPO, RFT, training data, grader, distillation, training job, large file upload, calibrate grader, deploy fine-tuned model, or evaluate fine-tuned model.
What you get
Submitted and monitored fine-tuning jobs with evaluated checkpoints and a deployed fine-tuned model endpoint when results pass review.
- Validated training datasets
- Submitted fine-tuning jobs
- Deployed and evaluated fine-tuned model
By the numbers
- Documents five training rules including 25-50% RFT grader failure target on the base model.
- Lists eleven Python automation scripts from submit_training through cleanup.
- Provides five workflow guides: quickstart, full pipeline, dataset creation, iterative training, and diagnose poor result
Files
Microsoft Foundry Skill
This skill helps developers work with Microsoft Foundry resources, covering model discovery and deployment, complete dev lifecycle of AI agent, evaluation workflows, and troubleshooting.
Pre-Execution Requirements
MANDATORY: Before executing ANY workflow, you MUST first call the Azure MCP `foundry` tool and inspect the available Foundry MCP tools and related parameters. Treat this initialfoundrycall as a discovery/help step. For this skill, Azure MCPfoundryis the required entry point for Foundry-related MCP operations.
Sub-Skills
MANDATORY: Before executing ANY workflow-specific steps, you MUST read the corresponding sub-skill document. Do not call workflow-specific MCP tools for a workflow without reading its skill document. This applies even if you already know the MCP tool parameters — the skill document contains required workflow steps, pre-checks, and validation logic that must be followed. This rule applies on every new user message that triggers a different workflow, even if the skill is already loaded.
This skill includes specialized sub-skills for specific workflows. Use these instead of the main skill when they match your task:
| Sub-Skill | When to Use | Reference |
|---|---|---|
| deploy | Containerize, build, push to ACR, create/update/clone agent deployments | deploy |
| invoke | Send messages to an agent, single or multi-turn conversations | invoke |
| observe | Evaluate agent quality, run batch evals, analyze failures, optimize prompts, improve agent instructions, compare versions, set up CI/CD monitoring, and enable continuous production evaluation | observe |
| trace | Query traces, analyze latency/failures, correlate eval results to specific responses via App Insights customEvents | trace |
| troubleshoot | View hosted agent logs, query telemetry, diagnose failures | troubleshoot |
| create | Create new hosted agent applications. Supports Microsoft Agent Framework, LangGraph, or custom frameworks in Python or C#, across responses or invocations protocols. | create |
| faos-optimize | Convert existing Python agent code to a FAOS (Foundry Agent Optimization Service) optimization-ready version by wiring evaluator-targeted instructions/model/temperature knobs, then stop for review before deployment. | faos-optimize |
| eval-datasets | Harvest production traces into evaluation datasets, manage dataset versions and splits, track evaluation metrics over time, detect regressions, and maintain full lineage from trace to de |
Guides
- Azure Skills for Claude Code: The Complete Guide to microsoft/azure-skills
microsoft/azure-skills is Microsoft's official repo of 27 Azure skills for Claude Code, covering deploy, AKS, diagnostics, compliance, cost, and Azure AI Foundry. Combined they have 9,775,059 installs, with microsoft-foundry alone at 446,557 installs (skills.sh registry, July 2026). This guide maps every skill to the Azure job it does.
Related skills
Ai Research ReproductionOrchestrate trustworthy, auditable reproduction of deep learning repositories directly from their READMEs.176k512Run TrainSafely execute selected deep learning training commands with standardized evidence capture.176k512Explore RunSafely run isolated exploratory experiments with clear recording and conservative selection before committing changes.176k512Paper Context ResolverFetch precise reproduction-critical details like dataset splits, preprocessing steps, or evaluation protocols from the original academic paper when the repo README leav141k512Repo Intake And PlanScan unfamiliar AI research repositories and receive a minimal, trustworthy reproduction target before investing significant time.140k512Env And Assets BootstrapCreate a reproducible, conservative conda environment plus required checkpoints, datasets and caches before attempting to run any AI research paper reproduction.140k512How it compares
Fine-tuning pipeline skill, not general Foundry model deploy or prompt-only optimization.
FAQ
Who is microsoft-foundry for?
Developers running Azure AI Foundry fine-tuning who need SFT, DPO, or RFT scripts, validators, and deployment guidance.
When should I use microsoft-foundry?
When preparing training data, submitting or monitoring fine-tuning jobs, calibrating RFT graders, or deploying and evaluating fine-tuned checkpoints.
Is microsoft-foundry safe to install?
Review the Security Audits panel on this page before installing in production.