Site Reliability Engineering and Observability Maturity Assessment
A 51-question diagnostic across 8 domains that evaluates whether an organization runs its production services to agreed, user-centered reliability targets, sees inside them when they fail, changes them safely, recovers f...
Overview
A 51-question diagnostic across 8 domains that evaluates whether an organization runs its production services to agreed, user-centered reliability targets, sees inside them when they fail, changes them safely, recovers fast and learns, including services built on AI models and agents.
Detailed Description
Most outages are not exotic: a configuration push that went to every region at once, a retry storm against a slow dependency, a page that fired 40 times a night until people muted it, a runbook last touched two years ago, a postmortem whose actions never closed. AI adds new ones: an LLM provider that hits its rate limit at peak, a prompt change that quietly halves answer quality, an agent that restarts the wrong cluster, and a triage assistant whose confident guess becomes the root cause. This assessment follows the reliability practice end to end: tiers, the SRE operating and engagement model, shared ownership and skills; user-centric SLIs, SLOs, error budgets and their use in decisions; telemetry standards, end-to-end traces, debugging, burn-rate alerting, cost and sensitive data; readiness, progressive delivery, flags and configuration, change to incident linkage and AI changes; resilience patterns, capacity, load, chaos, recovery and provider fallback; detection, command, on-call health, runbooks, status, postmortems and AI-assisted triage; toil, self-service, automated remediation, AI detection and agent actions; and KPIs, action follow-through, themes, game days, the cost and value of reliability, benchmarking, prediction and improvement. The model is self-contained: it owns every document it uses, and needs no other assessment. It tracks the DORA measures that bear on stability (change fail rate, rework rate and failed deployment recovery time) and leaves throughput measures (lead time and deployment frequency) to delivery assessments.
Assessment Details
Audience
Head of SRE or reliability engineering, platform engineering lead and engineering VP. Secondary: service owners and product engineering leads, incident commanders and on-call responders, observability engineers, AI and ML service teams, and business owners of critical services.
Purpose
Shows where reliability targets are missing or ignored, where teams cannot see or debug what fails, where changes reach every user before anyone checks them, where recovery and fallback are untested, where on-call burns people out, where AI services and agents run without SLOs or limits, and where postmortem lessons never turn into fixes.
Effort
Initial assessment: 4 to 6 hours of facilitated sessions with SRE, platform and observability engineering, two or three service teams including an AI service team, incident command and on-call leads, and an engineering or business leader. Evidence gathering: the SLO Register and budget history, alert and page data for one quarter, deployment and incident records, recent postmortems, test and game day results, and toil data.
Cadence
Yearly, and after a major outage, a platform or cloud migration, the launch of a customer-facing AI or agent service, or a change in SRE operating model.
Framework Alignment
KPIs
- Tier 1 Services Meeting Their SLO - Share of Tier 1 services that met their SLO over the quarter. Target: 95% or more (model target).
- Time to Mitigate Major Incidents - Detection to mitigation for major incidents at the median and 90th percentile. Target: median within 60 minutes and 90th percentile within 4 hours, or the tier target where tighter (model target).
- Change Fail Rate - Share of deployments to Tier 1 services that caused a failure needing remediation. Target: under 15% (model target).
- On-Call Shifts Within the Page Limit - Share of on-call shifts with no more pages than the On-Call Policy limit. Target: 90% or more (model target).
- SRE Time Spent on Toil - Share of SRE time spent on toil, no team above the 50% cap. Target: under 30% (model target; cap from the Google SRE Book).
Dependencies
Assessment Outline
(8 domains · 51 questions)-
1
1 Reliability Governance and Operating Model
The reliability policy and service tiers, the SRE operating model, engagement entry and exit, shared production ownership and SRE skills proven in practice
- 1.1 Reliability Policy and Service Tiers 1 questions
- 1.2 SRE Operating Model 1 questions
- 1.3 SRE Engagement Model 1 questions
- 1.4 Shared Production Ownership 1 questions
- 1.5 SRE Skills 1 questions
-
2
2 SLOs and Error Budgets
User-centric SLIs, SLO definition and agreement, the error budget policy, SLO reporting and decisions on breaches, SLO recalibration, dependency reliability and SLOs for AI and LLM services
- 2.1 User-Centric SLIs 1 questions
- 2.2 SLO Definition and Agreement 1 questions
- 2.3 Error Budget Policy 1 questions
- 2.4 SLO Reporting 1 questions
- 2.5 SLO Review and Recalibration 1 questions
- 2.6 Dependency Reliability 1 questions
- 2.7 SLOs for AI and LLM Services 1 questions
-
3
3 Observability
The instrumentation standard and context propagation, end-to-end telemetry including AI and agent traces, exploratory debugging, burn-rate alerting and noise review, synthetic and real user monitoring, telemetry cost and cardinality, sensitive data in telemetry and the observability platform
- 3.1 Telemetry Standard 1 questions
- 3.2 Telemetry Coverage 1 questions
- 3.3 Exploratory Debugging 1 questions
- 3.4 SLO-Based Alerting 1 questions
- 3.5 Synthetic and Real User Monitoring 1 questions
- 3.6 Telemetry Cost and Cardinality 1 questions
- 3.7 Sensitive Data in Telemetry 1 questions
- 3.8 Observability Platform 1 questions
-
4
4 Release Safety and Production Readiness
Production readiness before launch, progressive delivery with automated analysis and rollback, feature flag and configuration change safety, linking incidents to changes and AI model, prompt and provider changes
- 4.1 Production Readiness 1 questions
- 4.2 Progressive Delivery and Automated Rollback 1 questions
- 4.3 Feature Flag and Configuration Change Safety 1 questions
- 4.4 Change to Incident Correlation 1 questions
- 4.5 AI Model, Prompt and Provider Changes 1 questions
-
5
5 Resilience and Capacity
Failure-handling patterns, capacity forecasting and headroom, load testing, chaos experiments, recovery testing and fallback when AI model providers fail
- 5.1 Resilience Patterns 1 questions
- 5.2 Capacity Planning 1 questions
- 5.3 Load Testing 1 questions
- 5.4 Chaos Engineering 1 questions
- 5.5 Disaster Recovery Testing 1 questions
- 5.6 Provider and Model Dependency Failure 1 questions
-
6
6 Incident Response and On-Call
SLO-driven detection, incident command, sustainable on-call, runbooks, incident status communication with human approval of external posts, blameless postmortems and AI-assisted triage
- 6.1 SLO-Driven Detection 1 questions
- 6.2 Incident Command 1 questions
- 6.3 Sustainable On-Call 1 questions
- 6.4 Runbooks 1 questions
- 6.5 Incident Status Communication 1 questions
- 6.6 Blameless Postmortems 1 questions
- 6.7 AI-Assisted Triage 1 questions
-
7
7 Toil, Automation and Agent Actions
Toil measurement and caps, self-service reliability tooling, automated remediation, AI in operations for detection and correlation, and controls on AI agents acting in production
- 7.1 Toil Reduction 1 questions
- 7.2 Self-Service Reliability Tooling 1 questions
- 7.3 Automated Remediation 1 questions
- 7.4 AI Detection and Correlation 1 questions
- 7.5 Agent Actions in Production 1 questions
-
8
8 Reliability Learning, Measurement and Value
Reliability KPIs, action prioritization and verified closure, cross-service risk themes, knowledge sharing and game days, the cost and value of reliability, benchmarking, predictive reliability and continual improvement
- 8.1 Reliability KPIs 1 questions
- 8.2 Reliability Action Tracking 1 questions
- 8.3 Reliability Risk Themes 1 questions
- 8.4 Knowledge Sharing and Game Days 1 questions
- 8.5 Cost and Value of Reliability 1 questions
- 8.6 Benchmarking 1 questions
- 8.7 Predictive Reliability 1 questions
- 8.8 Continuous Improvement 1 questions
At a Glance
Still have questions? Get in touch
Business Bay, Dubai, UAE
ITSGovernance