How to Create QA Dashboards in Azure DevOps

A well-configured Azure DevOps dashboard gives stakeholders instant visibility into quality: pass rates, open bug counts, test coverage, and sprint progress — all in one place without asking the QA team for status updates.

---

Dashboard types in Azure DevOps

• Team dashboards: scoped to a team, visible to team members
• Project dashboards: visible to all project members
• Personal dashboards: private to the individual

QA teams typically maintain one team dashboard for daily operational visibility and contribute to a project dashboard for executive/stakeholder summaries.

---

Creating a QA dashboard

1. Go to Overview → Dashboards
2. Click + New dashboard
3. Name: QA — Sprint Quality View
4. Select: Team dashboard → QA Team
5. Click Create

---

Essential QA widgets

1. Test Results Trend

Shows pass/fail rate over time for an automated pipeline.

Configuration:

• Widget: Test Results Trend (Advanced)
• Pipeline: select your regression pipeline
• Duration: Last 30 days
• Metrics: Pass percentage, Failure count
• Chart type: Line chart

Interpretation: A rising pass rate line means quality is improving. A sudden drop signals a regression was introduced.

2. Test Plan Progress

Shows execution progress for the current sprint test plan.

Configuration:

• Widget: Test Plan Progress
• Test plan: select current sprint plan
• Show: Passed, Failed, Not Run counts with percentage

3. Bug Distribution by Severity

Configuration:

• Widget: Chart for Work Items
• Query: Type = Bug AND State <> Closed AND Iteration = @CurrentIteration
• Chart type: Pie/Donut chart
• Group by: Severity

Interpretation: A large "Critical" or "1 - Critical" slice means the sprint is at release risk.

4. Bug by State (Resolution Trend)

Configuration:

• Widget: Chart for Work Items
• Query: Type = Bug AND Iteration = @CurrentIteration
• Chart type: Stacked bar
• Group by: State
• Stacked by: Priority

5. Open High-Priority Bugs

A live list of critical issues:

Configuration:

• Widget: Query Results
• Query:

  Type = Bug
  AND Priority <= 2
  AND State <> Closed
  AND Iteration = @CurrentIteration
  

• Columns: ID, Title, Priority, Severity, Assigned To, State

6. Sprint Burndown

Shows work remaining in the sprint:

Configuration:

• Widget: Sprint Burndown
• Team: QA Team
• Current sprint: auto-selected

7. Pipeline Build Status

Shows whether the latest regression pipeline passed:

Configuration:

• Widget: Build History or Deployment Status
• Pipeline: Nightly regression pipeline
• Shows: last 10 runs with pass/fail indicators

8. Requirement Coverage

Shows how many user stories have test coverage:

Configuration:

• Widget: Requirements quality (available in Test Plans section)
• Test plan: current sprint
• Shows: % requirements with test cases, % executed, % passed

---

Recommended dashboard layout

┌─────────────────┬─────────────────┬─────────────────┐
│  Sprint Burndown│ Test Plan Prog. │ Pipeline Status │
│  (large)        │ (medium)        │ (medium)        │
├─────────────────┴─────────────────┼─────────────────┤
│  Test Results Trend (large)       │ Bug Severity    │
│  (pass rate over 30 days)         │ Distribution    │
│                                   │ (pie chart)     │
├───────────────────────────────────┼─────────────────┤
│  Open P1/P2 Bugs (query list)     │ Bug by State    │
│  (scrollable table)               │ (stacked bar)   │
└───────────────────────────────────┴─────────────────┘

---

Stakeholder-facing project dashboard

A simpler view for leadership and product owners:

Widget	What it shows
Test Results Trend	Pass rate over the last 4 sprints
Bug Trend	Total bugs opened vs closed per sprint
Deployment Status	Last 3 deployments to production
Team Velocity	Sprint story points completed

---

Common errors and fixes

Error: Test Results Trend widget shows "No data" Fix: Requires at least 2 pipeline runs with published test results. Also check the pipeline filter — the widget needs the correct pipeline selected, not "all pipelines".

Error: Query-based widgets show "Access denied" Fix: Shared queries used by widgets must be in the Shared Queries folder, not the personal queries folder. Move the query and update the widget.

Error: Sprint Burndown shows wrong team's data Fix: The widget uses the team selected when the dashboard was created. Create a new team dashboard scoped to the QA team, or re-create the dashboard as a QA Team dashboard.

Error: Dashboard layout resets after browser refresh Fix: Click the save button (floppy disk icon) after arranging widgets. Azure DevOps has an autosave but it's unreliable for layout changes.