How to set up your service maturity rubric

Production readiness is one thing, but a true service maturity approach goes beyond checklist items and allows you to effectively monitor and improve the overall quality, reliability, and security of your services. The OpsLevel Rubric is designed to help you ensure production readiness and measure overall service maturity against checks that you set across various levels. Here, we’ll explain how the rubric works in OpsLevel, and how you could set yours up.

What’s the Rubric in OpsLevel?

If checks are the individual tests that evaluate a component of a service, the rubric is a collection of these checks across multiple levels (e.g., bronze, silver, gold) grouped into categories like security, reliability, etc. Once you’ve categorized your checks across the levels, a service must pass all those checks to achieve that level’s rating. 

The rubric makes it straightforward for engineering teams to know where their services stand—both in general and in particular—so you know exactly where to focus your efforts if something isn’t passing, and what to prioritize to bump up to the next level.

Defining service levels in your Rubric

The rubric grades your services into different levels across different categories. Both the number of and names of these levels and categories are totally customizable. Our default levels are bronze, silver, and gold, across categories like service ownership, security, reliability, scalability, observability, etc. but you might want to call your levels A+, B, C, D- or simply Levels 1, 2, 3 and change up the categories. Below we offer some guidance about how to conceptualize and build out these levels, regardless of what you call them.

Bronze: the mission-critical level

Bronze is the most important minimum threshold a service needs to meet, typically reserved for metadata and a small number of critical checks. If a service isn’t up to bronze standards, then it shouldn’t get deployed to production, because it’s not meeting your organization’s core requirements. 

Silver: the core level

The next step up is silver, which has more baseline requirements once a service has cleared the bare minimum. A silver service is not just production ready, it's operationally mature. For example, a service has vulnerability scanning set up in your org’s scanning tool.

Gold: the aspirational level

The top level, gold, is more for future-proofing and aspirational standards that will be critical in the future, but are less of a priority now. For example, in silver, your service got added to the scanning tool, and in gold, you see that vulnerability scanning shows no vulnerabilities in the service.

How OpsLevel’s Rubric is different

By now, you may have noticed that the levels in our rubric don't represent a percentage of completeness (e.g., bronze doesn’t mean you met 60% of checks). We think framing it that way makes it hard to see if a service is passing a minimum standard check or not, since all checks are of equal importance.

For example, let’s say your service passes 95% of checks and is at gold, but you introduce a new bronze-level mission-critical check that it’s not passing. It would still show a 92% pass rate in other tools, but in OpsLevel, by not passing a critical check, it would get knocked back down to bronze. This helps to ensure that nothing slips through the cracks—especially where mission-critical checks are concerned. (Although this is a great example of when to use campaigns to roll out a new check before it's officially added to your rubric.)

This is because how we frame the rubric reflects our focus on service maturity versus a mere production readiness checklist—ultimately helping our customers deliver secure and mature services at scale. 

Now that you’ve got a basic idea of the framework for setting up your rubric, start determining the levels and categories that work best for your services. Check our technical documentation for a walkthrough of how to set it all up in OpsLevel.