Now liveThe Skillselion MCP - thousands of ranked skills, loaded into your agent mid-task. No install.Get it →
vasilyu1983 avatar

Dev Ai Coding Metrics

vasilyu1983/ai-agents-public

  • 81 installs
  • 73 repo stars
  • Updated July 13, 2026
  • vasilyu1983/ai-agents-public

Helps with ai & agent building tasks.

About

dev-ai-coding-metrics is a Claude Code skill for ai & agent building. It helps solo builders move faster with AI-assisted development.

  • dev-ai-coding-metrics
  • AI & Agent Building
  • AI-coding skill

Dev Ai Coding Metrics by the numbers

  • 81 all-time installs (skills.sh)
  • +2 installs in the week ending Jul 27, 2026 (Skillselion tracking)
  • Ranked #5,182 of 16,546 AI & Agent Building skills by installs in the Skillselion catalog
  • Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/vasilyu1983/ai-agents-public --skill dev-ai-coding-metrics

Add your badge

Show developers this skill is listed on Skillselion. Paste this into your README.

Listed on Skillselion
Installs81
repo stars★ 73
Last updatedJuly 13, 2026
Repositoryvasilyu1983/ai-agents-public ↗

What it does

Helps with ai & agent building tasks.

Files

SKILL.mdMarkdownGitHub ↗

AI Coding Agent Metrics for Engineering Teams

Measure what matters when adopting AI coding tools. This skill provides metrics frameworks, measurement methodology, ROI models, and reporting templates for engineering managers, VPs of Engineering, and CTOs evaluating or scaling AI coding agents.

When to Use This Skill

  • Evaluating AI coding tool ROI before or after purchase
  • Building a metrics program for AI-assisted development
  • Reporting AI tool impact to leadership or board
  • Designing controlled experiments to measure AI effectiveness
  • Comparing productivity across AI-equipped and traditional teams
  • Tracking adoption health and identifying stall patterns
  • Assessing quality impact of AI-generated code
  • Running developer experience surveys for AI tools

Quick Reference

TaskReferenceAsset
Track tool adoptionadoption-metrics.mdadoption-survey-template.md
Measure productivityproductivity-metrics.mdmetric-dashboard-template.md
Monitor code qualityquality-metrics.mdmetric-dashboard-template.md
Calculate ROIroi-framework.mdroi-calculator-template.md
Assess developer experiencedeveloper-experience-metrics.mdadoption-survey-template.md
Design experimentsbenchmarking-methodology.mdexperiment-design-template.md
Report to executivesroi-framework.mdexecutive-report-template.md
Measure AI coding impactthis skill—
Context engineering for AIdev-context-engineering—
Per-task agent ROIai-agents—
Observability for systemsqa-observability—

---

Core Metrics Taxonomy

Five measurement categories. Start with Adoption (you can't optimize what people aren't using), then layer in the others.

1. Adoption Metrics

Track whether and how developers use AI tools.

MetricFormulaTarget (Mature)Source
License Utilizationactive_users / licensed_seats>85%License admin
DAU/WAU Ratiodaily_active / weekly_active>0.6Tool telemetry
Feature Breadthfeatures_used / features_available>0.5Tool telemetry
Acceptance Ratesuggestions_accepted / suggestions_shown25-35%Copilot API / tool logs
Organic Usage Ratiovoluntary_sessions / total_sessions>0.8Survey + telemetry

Deep dive: references/adoption-metrics.md — 8 additional metrics, adoption curve phases, tool-specific tracking, stall patterns.

2. Velocity Metrics

Measure speed and throughput changes.

MetricFormulaExpected AI ImpactSource
Deploy Frequencydeploys / time_period+15-30%CI/CD pipeline
Lead Time for Changescommit_to_production-20-40%Git + CI/CD
Cycle Timeticket_start_to_deploy-15-35%Project management + Git
PR Throughputmerged_PRs / developer / week+20-40%Git platform
Time to First Commitonboard_date_to_first_commit-30-50%Git + HR data

Deep dive: references/productivity-metrics.md — DORA adaptations, SPACE framework, cycle time decomposition, confounding variables.

3. Quality Metrics

Track whether AI helps or hurts code quality.

MetricFormulaWatch DirectionSource
Bug Densitybugs / KLOCShould decreaseIssue tracker
Defect Escape Rateprod_bugs / total_bugsShould decreaseIssue tracker
Rework Ratefollowup_PRs / total_PRsWatch for increaseGit platform
Test Coveragecovered_lines / total_linesShould increaseCI coverage
Vulnerability Ratenew_vulns / sprintWatch for increaseSAST tools

Critical warning: Early studies show mixed quality results. AI can increase velocity while also increasing bug density if guardrails are missing. Monitor both.

Deep dive: references/quality-metrics.md — complexity tracking, security metrics, technical debt, quality guardrails.

4. Economic Metrics

Calculate costs, benefits, and ROI.

MetricFormulaBenchmarkSource
Cost per Seat(license + infra + training) / developers$20-50/dev/monthFinance
Hours Saved/Dev/Weekmeasured_or_estimated_time_savings2-8 hrs (varies widely)Survey + telemetry
ROI(net_benefits - costs) / costs × 100100-300% yr1 (vendor data)Calculated
Payback Periodtotal_investment / monthly_net_benefit2-6 monthsCalculated
Break-Even Adoptioncost / (max_benefit × developers)25-40% of teamCalculated

Caveat: Most published ROI figures come from tool vendors. Independent studies show lower but still positive returns. Always triangulate.

Deep dive: references/roi-framework.md — cost model, value model, formulas, executive reporting, benchmarks with caveats.

5. Experience Metrics

Measure developer satisfaction and cognitive impact.

MetricFormulaTargetSource
AI Tool Satisfactionsurvey_score (1-5 Likert)>3.8/5.0Quarterly survey
Tool NPSpromoters% - detractors%>30Quarterly survey
Cognitive LoadNASA-TLX adaptation (1-7)<4.0/7.0Post-task survey
Give-Up Ratestarted_AI_finished_manual / total<20%Telemetry
Trust Calibrationappropriate_review_rate>80%Code review data

Deep dive: references/developer-experience-metrics.md — survey design, cognitive load measurement, friction indicators, trust metrics.

---

Measurement Maturity Model

Where is your organization in measuring AI coding impact?

LevelNameCharacteristicsKey Action
L0No MeasurementNo tracking beyond license countInstall basic telemetry, run first survey
L1Basic TrackingLicense utilization + adoption rate trackedAdd DORA metrics baseline, first ROI estimate
L2Structured ProgramDORA + adoption + quality metrics active, quarterly surveyDesign controlled experiment, build dashboard
L3Evidence-BasedControlled experiments, statistical rigor, executive reportingCross-team benchmarking, predictive models
L4OptimizedContinuous measurement, automated dashboards, data-driven tool selectionIndustry benchmarking, publish findings

L0 → L1 Quick Start (2 hours)

1. Pull license utilization from admin console 2. Run the adoption survey (assets/adoption-survey-template.md) 3. Calculate basic ROI estimate (assets/roi-calculator-template.md) 4. Present 1-page summary to leadership (assets/executive-report-template.md)

L1 → L2 (2-4 weeks)

1. Establish DORA metric baselines (references/productivity-metrics.md) 2. Set up quality tracking (references/quality-metrics.md) 3. Build three-tier dashboard (assets/metric-dashboard-template.md) 4. Schedule quarterly developer experience surveys

L2 → L3 (1-3 months)

1. Design first controlled experiment (assets/experiment-design-template.md) 2. Apply statistical rigor (references/benchmarking-methodology.md) 3. Create executive reporting cadence (assets/executive-report-template.md) 4. Cross-reference with dev-context-engineering maturity model for context quality impact

L3 → L4 (ongoing)

1. Automate data collection and dashboards 2. Build predictive models (adoption → productivity correlation) 3. Benchmark against industry data 4. Contribute findings to community (conference talks, blog posts)

---

Metric Selection Decision Tree

Not every org needs every metric. Start from what you're trying to prove.

WHAT ARE YOU TRYING TO PROVE?
  │
  ├─ "Should we buy AI coding tools?"
  │   └─ START: roi-framework.md → roi-calculator-template.md
  │       Metrics: cost per seat, estimated hours saved, break-even adoption rate
  │
  ├─ "Are developers actually using the tools?"
  │   └─ START: adoption-metrics.md → adoption-survey-template.md
  │       Metrics: DAU/WAU, acceptance rate, feature breadth, organic usage
  │
  ├─ "Are we shipping faster?"
  │   └─ START: productivity-metrics.md → metric-dashboard-template.md
  │       Metrics: DORA metrics, cycle time, PR throughput
  │
  ├─ "Is code quality suffering?"
  │   └─ START: quality-metrics.md → metric-dashboard-template.md
  │       Metrics: bug density, defect escape rate, rework rate, vulnerability rate
  │
  ├─ "Are developers happy with AI tools?"
  │   └─ START: developer-experience-metrics.md → adoption-survey-template.md
  │       Metrics: satisfaction, NPS, cognitive load, give-up rate
  │
  ├─ "How do we compare to industry?"
  │   └─ START: benchmarking-methodology.md → experiment-design-template.md
  │       Metrics: DORA benchmarks, adoption curves, ROI ranges
  │
  └─ "Should we expand or cut the program?"
      └─ COMBINE: roi-framework.md + adoption-metrics.md + executive-report-template.md
          Metrics: ROI trend, adoption trajectory, satisfaction trend, quality delta

---

Dashboard Design Principles

Three-Tier Hierarchy

TierAudienceRefreshMetricsPurpose
ExecutiveC-Suite, VP EngMonthly4-6 KPIsInvestment decision, program health
Team LeadEng ManagersWeekly8-10 metricsTeam optimization, coaching
DeveloperIndividual devsReal-timePersonal statsSelf-improvement (opt-in only)

Design Rules

1. Lead with outcomes, not activity — show deploy frequency, not lines of code 2. Always show trend lines — a single number is meaningless without direction 3. Include confidence indicators — mark metrics with low sample sizes or high variance 4. Never rank individuals — aggregate to team level minimum (team size ≥5) 5. Pair speed with quality — never show velocity without adjacent quality metrics 6. Show cost alongside benefit — ROI is a ratio, not a cherry-picked benefit number

See: assets/metric-dashboard-template.md for full layout.

---

Anti-Patterns

Anti-PatternWhy It's HarmfulFix
Lines of Code as productivityAI inflates LOC; rewards verbosity over clarityUse outcome metrics (features shipped, bugs resolved)
Individual developer trackingCreates surveillance culture, erodes trustAggregate to team level, minimum team size 5
Vanity metrics only"90% adoption!" means nothing if output quality dropsAlways pair adoption with quality and satisfaction
Measuring too earlyFirst 4 weeks are learning curve, not steady stateAllow 8-12 week adoption curve before measuring impact
Vendor benchmarks as gospelVendor studies select favorable conditionsTriangulate with independent research; discount vendor data 30-50%
Ignoring the denominator"Shipped 40% more PRs" — but were they smaller?Normalize metrics (features/sprint, not PRs/sprint)
Correlation → causationTeam adopted AI and got a new senior devUse controlled experiments (benchmarking-methodology.md)
Surveying without actingDevelopers report friction → nothing changesClose the loop: share results + action plan within 2 weeks
One metric to rule them allSingle metric always gets gamedUse balanced scorecard (adoption + velocity + quality + experience)
Comparing incomparable teamsFrontend team vs infra team → meaningless comparisonSegment by project type, stack, and task complexity

---

Cross-References

SkillRelationship
dev-context-engineeringContext quality directly affects AI tool effectiveness — L0-L4 maturity model correlates with metric outcomes
ai-agentsPer-task token economics and agent ROI (this skill covers team/org-level metrics)
qa-observabilityOpenTelemetry integration for automated metric collection
product-managementOKR integration — AI metrics feed into engineering OKRs
startup-business-modelsUnit economics context for ROI calculations
dev-workflow-planningCycle time and planning metrics overlap

---

Do / Avoid

Do:

  • Start with adoption metrics — you can't optimize what people aren't using
  • Establish baselines before rolling out AI tools (8-week minimum)
  • Use the balanced scorecard approach (adoption + velocity + quality + experience)
  • Run quarterly developer experience surveys
  • Report with confidence intervals, not point estimates
  • Cross-reference with context maturity (dev-context-engineering) — structured repos get more AI benefit

Avoid:

  • Don't track individual developer productivity with AI tools
  • Don't use lines of code as a metric for anything
  • Don't measure impact in the first 4 weeks (adoption curve)
  • Don't rely on vendor-published benchmarks without independent validation
  • Don't survey developers without acting on the results
  • Don't compare teams without controlling for confounding variables
  • Don't present ROI without showing the cost model assumptions

---

Web Verification

55 curated sources in data/sources.json across 7 categories:

CategorySourcesKey Items
Developer Productivity Research~10DORA, SPACE, METR, ETH Zurich, McKinsey, Nicole Forsgren
AI Tool Adoption Data~8GitHub Copilot studies, Stack Overflow, GitClear, Harvard BS
Industry Case Studies~8Block/Square, Stripe, Klarna, Coinbase, Shopify, Amazon
Frameworks & Methodologies~8DX Company, LinearB, Haystack, Jellyfish, Swarmia
Measurement Tools~7Copilot Metrics API, OpenTelemetry, Grafana, PostHog
Consulting Reports~7McKinsey, BCG, HBR, Gartner, Forrester
Academic Research~7arXiv (Peng et al., Ziegler et al., METR), ACM, IEEE

Verify current data before final answers. Priority areas:

  • DORA State of DevOps report updates (annual)
  • GitHub Copilot Metrics API changes
  • New independent productivity studies (academic, not vendor)
  • ETH Zurich context effectiveness research updates
  • METR evaluation methodology updates

Fact-Checking

  • Use web search/web fetch to verify current external facts, versions, pricing, tool features, or published benchmarks before final answers.
  • Prefer independent/academic sources over vendor marketing; report source links and dates.
  • If web access is unavailable, state the limitation and mark guidance as unverified.

---

Navigation

References

FileContentLines
adoption-metrics.mdAdoption tracking, curve phases, tool-specific data sources, stall patterns~300
productivity-metrics.mdDORA for AI teams, SPACE framework, cycle time decomposition~350
quality-metrics.mdDefect metrics, complexity, test coverage, security, technical debt~280
roi-framework.mdCost/value models, ROI formulas, executive reporting, benchmarks~320
developer-experience-metrics.mdSatisfaction surveys, cognitive load, friction, trust, onboarding~260
benchmarking-methodology.mdA/B comparison, before/after design, statistical rigor, reporting~300

Assets (Copy-Ready Templates)

FilePurpose
metric-dashboard-template.mdThree-tier dashboard layout (Executive / Team Lead / Developer)
adoption-survey-template.md15-question developer survey with Likert scales and scoring
roi-calculator-template.mdSpreadsheet-ready ROI formulas and sensitivity analysis
executive-report-template.mdMonthly 1-page + quarterly deep-dive report templates
experiment-design-template.mdControlled experiment planning with statistical requirements

Related skills

mattpocockSetup Matt Pocock SkillsScaffold the per-repo configuration that Matt Pocock’s engineering agent skills rely on so they understand the issue tracker, triage labels, and domain documentation la495k196kopen.feishu.cnLark AppsConnect Claude, Cursor or custom agents directly to Lark (Feishu) for messaging, document automation, approval workflows and enterprise data access.402kjuliusbrusseeCavemanSlash token usage by roughly 75% while keeping every technical detail intact when working with Claude Code, Cursor or similar agents.391k94.7klarksuiteLark Skill MakerQuickly turn any Lark/Feishu OpenAPI call or multi-step workflow into a reusable agent skill with its own SKILL.md.386k16kxixu-meRunning Claude Code Via Litellm CopilotRun Claude Code at a fraction of the cost by routing requests through LiteLLM to the GitHub Copilot Chat API.270k72runcomfy-comCodex PetGenerate a complete Codex Pet spritesheet and metadata from one reference image without needing an OpenAI key or Codex Pro.246k10
AI & Agent Buildingagents
Browse by category
AI & AgentsFrontend & DesignBackend & DataDev ToolsAutomationTesting & ReviewSecurityMonitoring & CloudDocs & Planning
Design & frontend
Design systemsAnimation & motionAccessibility (a11y)CSS & TailwindUI designUXDesign to codeshadcn/uiLanding pagesBest design skills
Most searched skills
find-skillscavemanfrontend-designbrainstormingsystematic-debuggingstop-slopemil-design-engui-ux-pro-maxkarpathy-guidelinesweb-design-guidelinesreact-best-practicesgrill-meimprovemicrosoft-foundryagent-browser
Browse all listings
All agent skillsAll MCP serversAll marketplacesAll plugins

This week in AI coding

Five minutes, every Monday - the tools, releases and tactics for developers.

unsubscribe anytime.