Building a Delivery Health Dashboard

The Purpose of a Delivery Dashboard

A delivery health dashboard exists to answer one question for leadership: "Should I be worried?" If the answer is no, the dashboard confirms it in 30 seconds. If the answer is yes, the dashboard shows exactly where the problem is and how severe it is.

Most dashboards fail because they display too much data without interpretation, or they show lagging indicators that only confirm problems after they've already caused damage. An effective dashboard shows leading indicators that predict problems before they materialise.

The Three-Layer Architecture

Layer 1: Executive Summary (5-second view)

The top of the dashboard answers "are we healthy?" at a glance:

• Overall RAG status: One colour for the entire delivery system
• Sprint Goal status: On track / At risk / Off track
• Key number: The single most important metric this period (e.g. "3 of 4 DORA metrics improving")
• Top risk: The one thing leadership should know about

This layer is for executives who have 30 seconds. If they need more, they scroll down.

Layer 2: Delivery Metrics (2-minute view)

The middle layer shows the health indicators that inform the RAG status:

Flow metrics:

• Throughput: items completed per sprint (trend line, not just current)
• Cycle time: median days from start to done (trend line)
• WIP: current items in progress vs WIP limit

Quality metrics:

• Change failure rate: % of deployments causing incidents
• Escaped defects: bugs found in production this sprint
• Rework rate: % of items that came back for fixes

Predictability metrics:

• Sprint Goal success rate (last 6 sprints)
• Carryover rate (last 6 sprints)
• Deployment frequency (trend)

Team health:

• Blocker count and average age
• Dependency health (% on track)
• Unplanned work percentage

Layer 3: Detail and Context (5-minute view)

The bottom layer provides context for anyone who needs to understand the "why" behind the numbers:

• Blocker details: what's blocked, who owns resolution, how long it's been blocked
• Dependency tracker: cross-team dependencies with owners, dates, and RAG
• Risk register: top 5 risks with mitigation status
• Decisions needed: items requiring leadership input
• Upcoming milestones: next 2-4 weeks of key dates

Choosing the Right Metrics

The biggest mistake is tracking too many metrics. Start with 5-7 that directly inform decisions. Add more only when you have a specific question they answer.

The "So What?" test: For every metric on the dashboard, ask: "If this number changed significantly, what would we do differently?" If the answer is "nothing," remove it.

Leading vs lagging indicators:

• Leading: WIP count, blocker age, dependency health, unplanned work % (predict future problems)
• Lagging: velocity, escaped defects, sprint goal success (confirm past performance)

A good dashboard has both, but leading indicators are more valuable because they enable proactive intervention.

RAG Status Definitions

RAG (Red/Amber/Green) only works when definitions are explicit and consistent. Without clear criteria, RAG becomes subjective and teams default to "Green" to avoid difficult conversations.

Green: All key metrics within target range. Sprint Goal on track. No unresolved blockers older than 48 hours. Dependencies on track.

Amber: One or more metrics trending in the wrong direction. Sprint Goal at risk but recoverable. Blockers exist but have resolution plans. One dependency at risk.

Red: Multiple metrics outside target range. Sprint Goal unlikely to be met. Unresolved blockers older than 5 days. Critical dependency failed. Immediate leadership attention needed.

The watermelon rule: A dashboard that's always Green is lying. If your dashboard hasn't shown Amber or Red in 3 months, your thresholds are too generous or your team isn't being honest. Recalibrate.

Building the Dashboard

Data Sources

• Jira / Azure DevOps: Sprint data, throughput, cycle time, WIP, blockers, carryover
• CI/CD pipeline: Deployment frequency, lead time, build success rate
• Incident management: Change failure rate, MTTR, incident count
• Manual input: RAG status, risk commentary, decisions needed, dependency updates

Tools

• Jira dashboards + Confluence: Good for teams already in the Atlassian ecosystem. Limited visualisation.
• Looker / Tableau / Power BI: Best for complex data from multiple sources. Requires data engineering.
• Google Sheets / Excel: Surprisingly effective for small teams. Easy to maintain. Low barrier.
• Swarmia / LinearB / Jellyfish: Purpose-built engineering metrics platforms. Automated DORA + flow metrics.

Update Cadence

• Automated metrics (throughput, cycle time, deployment frequency): real-time or daily
• Manual inputs (RAG commentary, risk updates, decisions): weekly (Friday afternoon)
• Review cadence: Weekly delivery review meeting uses the dashboard as the agenda

Using the Dashboard in Governance

The dashboard is not a report — it's a conversation tool. Use it to structure delivery governance:

Weekly delivery review (30 min): 1. Walk the executive summary (2 min) 2. Review any Amber/Red metrics (5 min each) 3. Blocker triage: who owns resolution? By when? (5 min) 4. Dependency check: anything at risk? (5 min) 5. Decisions needed from leadership (5 min)

Fortnightly steering committee:

• Present Layer 1 only (executive summary)
• Deep-dive on one specific area if needed
• Focus on decisions and escalations, not status

Quarterly delivery review:

• Trend analysis across the quarter
• DORA metric progression
• Investment recommendations for next quarter

Common Dashboard Anti-Patterns

The data dump: 50 metrics on one page with no hierarchy or interpretation. Nobody reads it. Fix: ruthlessly prioritise. 5-7 metrics maximum on the primary view.

The always-Green dashboard: RAG thresholds are so generous that problems are never surfaced. Fix: calibrate thresholds against actual delivery outcomes. If you missed a deadline but the dashboard was Green, your thresholds are wrong.

The blame dashboard: Metrics are used to identify underperforming individuals or teams. Fix: dashboards measure the system, not people. Remove any individual-level data.

The stale dashboard: Updated monthly or "when someone remembers." Fix: automate what you can. For manual inputs, assign ownership and make it part of the weekly rhythm.

The vanity dashboard: Shows metrics that look good but don't drive decisions (total stories completed all-time, cumulative deployments). Fix: apply the "So What?" test. Remove anything that doesn't inform action.

---

Download the [Delivery Dashboard template](/templates) for a ready-to-use executive delivery health format.