Azure DevOps Services Explained: Boards, Repos, Pipelines, Test Plans

Azure DevOps bundles five distinct services into a single platform. You can use them all together or connect individual services to other tools — Azure Pipelines works with GitHub repos, Azure Boards works with external issue trackers, and so on. Understanding each service individually is the foundation for understanding how they work together.

This article covers all five services from a QA engineer's perspective.

Azure Boards

What it is: A work item tracking system similar to Jira, Linear, or GitHub Issues.

What QA engineers do here:

• Review user stories and acceptance criteria before writing test cases
• Create and track bugs through the defect lifecycle
• Query and filter work items using saved queries
• View and contribute to the sprint board and backlog
• Build dashboard widgets showing QA metrics

The Boards hierarchy

Epic (quarters)
  └── Feature (weeks to months)
        └── User Story (days to a sprint)
              └── Task (hours)
              └── Bug (when a defect is found)

Key QA uses of Azure Boards

Acceptance Criteria field: Every User Story should have this filled. It's your testing contract — if the AC says "error message must include the field name", your test case validates exactly that.

Bug states: New → Active → Resolved → Closed. As QA, you move bugs from Resolved to Closed after verifying the fix. Never close a bug without verifying it — this breaks the bug's resolution date metrics.

Queries for QA: The most useful queries for QA teams:

// All open bugs in current sprint
Type = Bug AND Iteration = @CurrentIteration AND State <> Closed

// High-priority bugs without an owner
Type = Bug AND Priority <= 2 AND Assigned To = [Unassigned]

// Bugs found in testing vs bugs found in production
Type = Bug AND Tags CONTAINS "found-in-testing"

Azure Repos

What it is: A full Git repository hosting service, similar to GitHub or GitLab.

What QA engineers do here:

• Store test automation code alongside the application code (or in a separate test repo)
• Review pull requests that include test changes
• Set up branch policies that require tests to pass before merging

Branch policies for QA enforcement

Branch policies on the main or release branches can enforce quality gates. Go to Repos → Branches → Branch Policies:

✓ Require a minimum of 2 reviewers
✓ Check for linked work items (ensures every PR has a user story linked)
✓ Build validation: must pass the CI pipeline including tests
✓ Required reviewer: QA lead must approve for changes to test-critical paths

Test code organisation

A clean repo structure for test automation:

tests/
  e2e/
    checkout.spec.ts
    auth.spec.ts
  api/
    orders.test.ts
    users.test.ts
  fixtures/
    test-data.json
  utils/
    helpers.ts
playwright.config.ts
azure-pipelines.yml

Keeping tests in the same repo as the application code ensures they're always versioned together — no test-to-code mismatch when checking out a specific release.

Azure Pipelines

What it is: A CI/CD automation service that builds, tests, and deploys code.

What QA engineers do here:

• Set up automated test stages in the pipeline
• Monitor pipeline runs for test failures
• Investigate flaky tests vs genuine product regressions
• Configure scheduled runs for nightly regression suites

Pipeline YAML structure for testing

trigger:
  branches:
    include:
      - main
      - feature/*
 
pool:
  vmImage: ubuntu-latest
 
stages:
  - stage: Build
    jobs:
      - job: BuildApp
        steps:
          - task: NodeTool@0
            inputs:
              versionSpec: '20.x'
          - script: npm ci
          - script: npm run build
 
  - stage: Test
    dependsOn: Build
    jobs:
      - job: UnitTests
        steps:
          - script: npm test -- --reporter=junit --output-file=results/unit.xml
          - task: PublishTestResults@2
            inputs:
              testResultsFormat: JUnit
              testResultsFiles: results/unit.xml
            condition: always()
 
      - job: E2ETests
        steps:
          - script: npx playwright test --reporter=junit
          - task: PublishTestResults@2
            inputs:
              testResultsFormat: JUnit
              testResultsFiles: playwright-report/results.xml
            condition: always()
          - task: PublishPipelineArtifact@1
            inputs:
              targetPath: playwright-report
              artifact: e2e-report
            condition: always()

The condition: always() rule

Always add condition: always() to PublishTestResults tasks. Without it, the task only runs when the previous step succeeded — meaning a test failure prevents test results from being uploaded, which defeats the entire purpose.

Azure Test Plans

What it is: A dedicated test management service for manual test cases, exploratory testing, and linking automated results to formal test plans.

What QA engineers do here:

• Write and organise test cases
• Execute manual tests using Test Runner
• Track test execution progress across a sprint
• Link automated test results to formal test plans
• Generate test reports for stakeholders

Test Plans vs pipelines: the distinction

Azure Pipelines runs automated tests and stores the results. Azure Test Plans manages the test inventory — the catalogue of what should be tested. They connect when a pipeline publishes results and those results are associated with test cases in Test Plans.

Azure Test Plans:        What to test (inventory)
Azure Pipelines:         Running the tests (execution)
Link between them:       PublishTestResults task

Requirement-based test suites

The most powerful feature for traceability. Create a Query-based suite using:

Type = User Story AND Iteration = @CurrentIteration AND State = Active

Azure DevOps automatically populates the suite with all User Stories in the current sprint. Test cases linked to those User Stories appear inside them. As stories complete and new ones enter the sprint, the suite updates automatically.

Azure Artifacts

What it is: A package manager and feed hosting service for NuGet, npm, Maven, PyPI, and Universal Packages.

What QA engineers typically do here:

• Publish and consume internal test utility packages
• Cache package downloads for faster pipeline runs
• Version and share test fixtures, page object models, or shared test utilities as internal packages

Using Artifacts for test utilities

If your QA team maintains a shared page object model library or a test data generation utility, Artifacts lets you publish it as a private package:

# Publish a shared test utility as an npm package
- task: Npm@1
  inputs:
    command: 'publish'
    publishRegistry: 'useFeed'
    publishFeed: 'MyOrg/qa-utilities'

Other teams can then install it:

npm install @myorg/qa-utilities --registry https://pkgs.dev.azure.com/MyOrg/...

This eliminates copy-paste maintenance of common test code across multiple repos.

How the five services work together

Here's the full flow for a feature going through development and testing:

1. Boards:    QA reviews User Story #847 acceptance criteria
2. Boards:    QA creates test cases linked to #847
3. Repos:     Dev writes code and opens PR
4. Pipelines: PR triggers CI pipeline — unit tests run automatically
5. Repos:     Pipeline passes → QA reviews and approves PR
6. Repos:     Code merged to main
7. Pipelines: Deployment pipeline runs — deploys to staging environment
8. Test Plans: QA runs manual tests against staging using Test Runner
9. Boards:    QA raises Bug #891 from failed test case → linked to #847
10. Repos:    Dev fixes bug, opens new PR
11. Pipelines: CI runs again — tests pass
12. Test Plans: QA verifies fix → closes Bug #891, marks test case Passed
13. Boards:    User Story #847 → state changes to Closed
14. Boards:    Sprint dashboard shows 100% pass rate for #847's test cases

Every step flows through Azure DevOps. The traceability is automatic — you can start at a User Story and trace all the way to which pipeline run validated it and which test cases passed.

Common errors and fixes

Error: PublishTestResults task shows no results after running Fix: Verify the testResultsFiles glob pattern matches your output file. Test with an explicit path first (**/junit-results.xml), then generalise.

Error: Branch policy blocks merges even when tests pass Fix: Ensure the build validation policy points to the correct pipeline definition. Policies on main must reference the pipeline that runs on PRs to main.

Error: Azure Artifacts feed returns 401 in the pipeline Fix: Add the project build service account ({Project} Build Service) to the Artifacts feed as a Contributor. Go to Artifacts → Feed Settings → Permissions.

Error: Test Plans shows "No test cases" even after creating test cases Fix: Test cases must be added to a Test Suite inside a Test Plan. Creating a test case work item doesn't automatically add it to a plan — you must add it explicitly or use a query-based suite.