Track Test Coverage and Defect Metrics in Azure DevOps

Measuring quality without metrics is guesswork. Azure DevOps provides the data — test results, bug work items, pipeline runs — but it takes deliberate configuration to turn that data into actionable metrics.

Code coverage in Azure Pipelines

Publishing coverage from Jest

- script: |
    npm test -- \
      --coverage \
      --coverageReporters=cobertura \
      --coverageDirectory=coverage
  displayName: Run tests with coverage
 
- task: PublishCodeCoverageResults@1
  displayName: Publish code coverage
  inputs:
    codeCoverageTool: Cobertura
    summaryFileLocation: coverage/cobertura-coverage.xml
    reportDirectory: coverage
  condition: always()

Publishing coverage from pytest

- script: |
    pip install pytest-cov
    pytest tests/ \
      --cov=src \
      --cov-report=xml:coverage.xml \
      --cov-fail-under=80
  displayName: Run tests with coverage
 
- task: PublishCodeCoverageResults@1
  inputs:
    codeCoverageTool: Cobertura
    summaryFileLocation: coverage.xml
  condition: always()

The pipeline summary now shows a Code Coverage tab with line, branch, and function coverage percentages.

Coverage gate (fail if below threshold)

# In Jest config:
coverageThreshold:
  global:
    lines: 80
    branches: 75
    functions: 80
 
# In pytest:
--cov-fail-under=80

If coverage drops below the threshold, the pipeline fails — preventing coverage regression from being merged.

Requirement coverage metrics

Query: uncovered user stories

Find user stories with no linked test cases:

Work Item Type = User Story
AND Iteration = @CurrentIteration
AND State = Active
AND [Not linked to: Test Case with Tests link]

Save as a chart widget on your dashboard. Any bar in this chart represents a testing gap.

Requirement coverage formula

Coverage % = (Stories with all test cases passed / Total stories) × 100

Azure DevOps calculates this automatically in the Requirements tab of a test plan.

Defect metrics

Defect density query

Total bugs found:
  Type = Bug
  AND Created >= @StartOf("sprint")

Bugs by component:
  Type = Bug AND Area Path UNDER "App/Checkout"
  → count gives defect density for checkout module

Bug escape rate dashboard

Bugs found in testing:
  Type = Bug AND Tags CONTAINS "found-in-testing"
  AND Created >= @StartOf("month")

Bugs found in production:
  Type = Bug AND Tags CONTAINS "found-in-production"
  AND Created >= @StartOf("month")

Escape rate = production bugs / (testing bugs + production bugs) × 100

Tag every bug with found-in-testing or found-in-production when it's created. This enables escape rate tracking over time.

Building a metrics scorecard

Create a monthly metrics export:

Q3 2025 QA Metrics Scorecard

Test Coverage:
  Requirement coverage: 94%  (target: >90%) ✓
  Code coverage:        82%  (target: >80%) ✓
  Automated %:          68%  (target: >65%) ✓

Defect Metrics:
  Bugs found (QA):      47
  Bugs found (prod):     3
  Escape rate:           6%  (target: <10%) ✓
  Defect density:       1.4  (target: <2.0) ✓

Pipeline Health:
  Avg pass rate:        96.8% (target: >95%) ✓
  Flaky test count:        4  (target: <5)  ✓
  Mean pipeline time:   11m   (target: <15m) ✓

Common errors and fixes

Error: Coverage report shows 0% despite tests running Fix: The test runner must be configured to instrument source files. Check that the --cov=src or --coverage flag points to your source directory, not the test directory.

Error: Cobertura XML not found by PublishCodeCoverageResults Fix: The path in summaryFileLocation must be relative to $(System.DefaultWorkingDirectory). Use $(System.DefaultWorkingDirectory)/coverage/cobertura-coverage.xml for explicit paths.

Error: Bug escape rate query includes testing environment bugs Fix: Standardise the tagging: use environment:staging and environment:production tags. Filter queries by environment tag to separate test environment bugs from production escapes.