Migrating Airflow 2 To 3
Upgrade Apache Airflow 2.x DAGs and project code to Airflow 3.x with Ruff auto-fixes and a manual migration checklist.
Overview
Migrating Airflow 2 to 3 is an agent skill for the Operate phase that guides DAG and project code upgrades from Airflow 2.x to 3.x with Ruff and a manual checklist.
Install
npx skills add https://github.com/astronomer/agents --skill migrating-airflow-2-to-3What is this skill?
- Load as first step for any Airflow 3 migration, upgrade, or compatibility request
- Ruff preview AIR rules (AIR30/AIR301/AIR302/AIR31/AIR311/AIR312) with --fix --unsafe-fixes
- Post-edit hook reminder: ruff check --preview --select AIR .
- Recommended path: Airflow 2.11 first, then ≥3.0.11 ideally 3.1 for stability
- Manual migration checklist in reference/migration-checklist.md for leftovers
- Ruff Airflow migration rule families: AIR30, AIR301, AIR302, AIR31, AIR311, AIR312
- Recommended upgrade: Airflow 2.11 then at least 3.0.11, ideally 3.1
Adoption & trust: 800 installs on skills.sh; 384 GitHub stars; 3/3 security scanners passed (skills.sh audits).
What problem does it solve?
Your Airflow 2.x DAGs fail or warn on 3.x because imports, operators, hooks, and context APIs changed and you need a safe upgrade sequence.
Who is it for?
Maintainers of existing Airflow 2 deployments planning a supported upgrade path (2.11 → 3.0.11+ / 3.1).
Skip if: Brand-new pipeline projects with no Airflow 2 baggage, or teams only asking for Astronomer Cloud billing—not code migration.
When should I use this skill?
User mentions Airflow 3 migration, upgrade, compatibility issues, breaking changes, or wants to modernize Airflow 2.x codebase; prompt before upgrading detected 2.x code.
What do I get? / Deliverables
Your codebase is migrated toward Airflow 3.x with automated Ruff fixes applied and remaining issues tracked via the migration checklist.
- Migrated DAG and utility code
- Ruff-clean AIR rule pass (preview)
- Checklist of remaining manual migration items
Recommended Skills
Journey fit
Airflow migration is production data-pipeline infrastructure work—canonical shelf is operate/infra when modernizing a running orchestrator. Subphase infra covers platform upgrades, breaking API changes, and deployment path constraints (2.11 → 3.0.11+).
How it compares
Use instead of generic Python refactors—this skill encodes Airflow-specific breaking changes and Astronomer's upgrade ordering, not a generic linter skill.
Common Questions / FAQ
Who is migrating-airflow-2-to-3 for?
Data engineers and solo builders who own Airflow DAG repos and need agent-guided 2→3 code migration aligned with Astronomer documentation.
When should I use migrating-airflow-2-to-3?
In operate/infra when upgrading orchestrator versions, fixing Airflow 3 compatibility errors, or when the user asks to modernize detected 2.x code—always as the first step for migration work.
Is migrating-airflow-2-to-3 safe to install?
Review the Security Audits panel on this Prism page; the skill suggests local Ruff and file edits—run fixes in a branch and validate in staging before production deploy.
SKILL.md
READMESKILL.md - Migrating Airflow 2 To 3
# Airflow 2 to 3 Migration This skill helps migrate **Airflow 2.x DAG code** to **Airflow 3.x**, focusing on code changes (imports, operators, hooks, context, API usage). **Important**: Before migrating to Airflow 3, strongly recommend upgrading to Airflow 2.11 first, then to at least Airflow 3.0.11 (ideally directly to 3.1). Other upgrade paths would make rollbacks impossible. See: https://www.astronomer.io/docs/astro/airflow3/upgrade-af3#upgrade-your-airflow-2-deployment-to-airflow-3. Additionally, early 3.0 versions have many bugs - 3.1 provides a much better experience. ## Migration at a Glance 1. Run Ruff's Airflow migration rules to auto-fix detectable issues (AIR30/AIR301/AIR302/AIR31/AIR311/AIR312). - `ruff check --preview --select AIR --fix --unsafe-fixes .` 2. Scan for remaining issues using the manual search checklist in [reference/migration-checklist.md](reference/migration-checklist.md). - Focus on: direct metadata DB access, legacy imports, scheduling/context keys, XCom pickling, datasets-to-assets, REST API/auth, plugins, and file paths. - Hard behavior/config gotchas to explicitly review: - Cron scheduling semantics: consider `AIRFLOW__SCHEDULER__CREATE_CRON_DATA_INTERVAL=True` if you need Airflow 2-style cron data intervals. - `.airflowignore` syntax changed from regexp to glob; set `AIRFLOW__CORE__DAG_IGNORE_FILE_SYNTAX=regexp` if you must keep regexp behavior. - OAuth callback URLs add an `/auth/` prefix (e.g. `/auth/oauth-authorized/google`). - **Shared utility imports**: Bare imports like `import common` from `dags/common/` no longer work on Astro. Use fully qualified imports: `import dags.common`. 3. Plan changes per file and issue type: - Fix imports - update operators/hooks/providers - refactor metadata access to using the Airflow client instead of direct access - fix use of outdated context variables - fix scheduling logic. 4. Implement changes incrementally, re-running Ruff and code searches after each major change. 5. Explain changes to the user and caution them to test any updated logic such as refactored metadata, scheduling logic and use of the Airflow context. --- ## Architecture & Metadata DB Access Airflow 3 changes how components talk to the metadata database: - Workers no longer connect directly to the metadata DB. - Task code runs via the **Task Execution API** exposed by the **API server**. - The **DAG processor** runs as an independent process **separate from the scheduler**. - The **Triggerer** uses the task execution mechanism via an **in-process API server**. **Trigger implementation gotcha**: If a trigger calls hooks synchronously inside the asyncio event loop, it may fail or block. Prefer calling hooks via `sync_to_async(...)` (or otherwise ensure hook calls are async-safe). **Key code impact**: Task code can still import ORM sessions/models, but **any attempt to use them to talk to the metadata DB will fail** with: ```text RuntimeError: Direct database access via the ORM is not allowed in Airflow 3.x ``` ### Patterns to search for When scanning DAGs, custom operators, and `@task` functions, look for: - Session helpers: `provide_session`, `create_session`, `@provide_session` - Sessions from settings: `from airflow.settings import Session` - Engine access: `from airflow.settings import engine` - ORM usage with models: `session.query(DagModel)