Authoring Dags
Create new Apache Airflow DAGs with discover-plan-implement workflow and Astronomer `af` CLI conventions.
Overview
Authoring DAGs is an agent skill for the Build phase that guides creating Apache Airflow DAGs via discover-plan-implement steps and `af` CLI best practices.
Install
npx skills add https://github.com/astronomer/agents --skill authoring-dagsWhat is this skill?
- Four-step workflow: Discover → Plan → Implement (with approval gate on structure)
- Uses Astronomer `af` CLI; optional `astro otto` or `uv tool install astro-airflow-mcp`
- Best-practice DAG patterns and conventions for new pipeline code
- Stop hook reminds to validate with testing-dags skill after authoring
- Pairs with testing-dags for test → debug → fix → retest cycles
- 3-phase authoring workflow: Discover, Plan, Implement
Adoption & trust: 818 installs on skills.sh; 384 GitHub stars; 3/3 security scanners passed (skills.sh audits).
What problem does it solve?
You need a new Airflow pipeline but lack a consistent discover-plan-implement ritual and DAG conventions for your environment.
Who is it for?
Builders on Astronomer/Airflow who want agent-guided DAG creation with CLI-aware patterns before running tests.
Skip if: Teams that only need one-off SQL scripts without orchestration, or debugging-only sessions better served by testing-dags alone.
When should I use this skill?
User wants to create a new DAG, write pipeline code, or asks about DAG patterns and conventions.
What do I get? / Deliverables
A new DAG file follows approved structure and Astronomer patterns, with an explicit handoff to invoke testing-dags for validation.
- New or updated DAG Python module following proposed structure
- Plan approval checkpoint before implementation
Recommended Skills
Journey fit
How it compares
Workflow skill for writing DAGs—not a substitute for the testing-dags skill’s test-debug-retest loop.
Common Questions / FAQ
Who is authoring-dags for?
Solo builders and small data teams using Airflow on Astronomer who want agents to follow official DAG authoring and `af` CLI workflows.
When should I use authoring-dags?
In Build when creating a new DAG, drafting pipeline code, or asking about DAG structure and conventions—then follow with testing-dags before shipping.
Is authoring-dags safe to install?
It may run shell hooks and CLI commands in your environment; review Security Audits on this Prism page and restrict repo access as you would for any DevOps skill.
Workflow Chain
Then invoke: testing dags
SKILL.md
READMESKILL.md - Authoring Dags
# DAG Authoring Skill This skill guides you through creating and validating Airflow DAGs using best practices and `af` CLI commands. > **For testing and debugging DAGs**, see the **testing-dags** skill which covers the full test -> debug -> fix -> retest workflow. --- ## Running the CLI These commands assume `af` is on PATH. Run via `astro otto` to get it automatically, or install standalone with `uv tool install astro-airflow-mcp`. --- ## Workflow Overview ``` +-----------------------------------------+ | 1. DISCOVER | | Understand codebase & environment | +-----------------------------------------+ | +-----------------------------------------+ | 2. PLAN | | Propose structure, get approval | +-----------------------------------------+ | +-----------------------------------------+ | 3. IMPLEMENT | | Write DAG following patterns | +-----------------------------------------+ | +-----------------------------------------+ | 4. VALIDATE | | Check import errors, warnings | +-----------------------------------------+ | +-----------------------------------------+ | 5. TEST (with user consent) | | Trigger, monitor, check logs | +-----------------------------------------+ | +-----------------------------------------+ | 6. ITERATE | | Fix issues, re-validate | +-----------------------------------------+ ``` --- ## Phase 1: Discover Before writing code, understand the context. ### Explore the Codebase Use file tools to find existing patterns: - `Glob` for `**/dags/**/*.py` to find existing DAGs - `Read` similar DAGs to understand conventions - Check `requirements.txt` for available packages ### Query the Airflow Environment Use `af` CLI commands to understand what's available: | Command | Purpose | |---------|---------| | `af config connections` | What external systems are configured | | `af config variables` | What configuration values exist | | `af config providers` | What operator packages are installed | | `af config version` | Version constraints and features | | `af dags list` | Existing DAGs and naming conventions | | `af config pools` | Resource pools for concurrency | **Example discovery questions:** - "Is there a Snowflake connection?" -> `af config connections` - "What Airflow version?" -> `af config version` - "Are S3 operators available?" -> `af config providers` --- ## Phase 2: Plan Based on discovery, propose: 1. **DAG structure** - Tasks, dependencies, schedule 2. **Operators to use** - Based on available providers 3. **Connections needed** - Existing or to be created 4. **Variables needed** - Existing or to be created 5. **Packages needed** - Additions to requirements.txt **Get user approval before implementing.** --- ## Phase 3: Implement Write the DAG following best practices (see below). Key steps: 1. Create DAG file in appropriate location 2. Update `requirements.txt` if needed 3. Save the file --- ## Phase 4: Validate **Use `af` CLI as a feedback loop to validate your DAG.** ### Step 1: Check Import Errors After saving, check for parse errors (Airflow will have already parsed the file): ```bash af dags errors ``` - If your file appears -> **fix and retry** - If no errors -> **continue** Common causes: missing imports, syntax errors, missing packages. ### Step 2: Verify DAG Exists ```b