Datadog + OpsLevel Benefits
Integrating OpsLevel and Datadog makes it easy for anyone in your engineering organization to see the statuses of all your monitors in real time, alongside the complete context of your service catalog.
For SREs, platform engineering, or engineering management, OpsLevel can be used to ensure Datadog monitors are configured correctly and active on all the relevant services.
Integrating with Datadog
This guide will walk you through how to integrate Datadog with OpsLevel Alert sources to make it easier for your organization to view up-to-date monitor status alongside your Service Catalog. You can also integrate OpsLevel and Datadog using Datadog Checks to build Datadog monitor status into your Service Maturity Rubric. To configure Datadog checks, check out the Datadog Checks Integration guide.
How it Works
OpsLevel’s Datadog integration listens for any status changes in your associated Datadog monitors.
Note: Synthetic testing monitors are supported.
The relationship between Datadog monitors and linked alert sources on OpsLevel services follows the pattern below:
|Datadog monitor||Linked OpsLevel service|
|No Data||gray No Data state|
|Alert||red Alert state|
|Warn||orange Warn state|
|OK||green OK state|
Datadog Usage Checks
For guidance on setting up a check to verify Datadog is used on all services in your catalog, read more on Alert Source Usage Checks here.
Configuring the Datadog Integration in OpsLevel requires both the Datadog API Key and Datadog Application Key. When retrieving the keys from Datadog, the logged in user must be an Admin user with the following permissions enabled:
- Monitors Read (monitors_read)
- Monitors Write (monitors_write)
OpsLevel needs authentication keys from users with these permissions so that the integration can find and update Datadog monitors in order to use webhooks to send updated monitor status to OpsLevel.
OpsLevel support is available at email@example.com or via your shared OpsLevel Slack channel.
Installing the Datadog integration can be completed in seconds.
1. In the OpsLevel app, from the left-hand menu, navigate to Integrations and click the New Integration tile.
2. Click the Datadog tile and then enter your API and APP keys.
Note: Ensure that the authentication keys' associated user has both the monitor_read and monitor_write permissions enabled.
3. Make sure @webhook-OpsLevel gets added to any Datadog monitors that you attach to OpsLevel Services. OpsLevel can automatically do this on your behalf if the Set Webhooks On Monitors option is enabled. If you prefer to manage your Datadog monitors directly (e.g. through Datadog’s API or with config-as-code), disable this option.
On the next screen you’ll be presented with a detailed getting started guide to configure webhook delivery from Datadog to OpsLevel.
How to attach a Datadog monitor to a service in OpsLevel
First, log into your Datadog account and navigate to the Monitors page.
1. Click on the monitor you want to be automatically attached.
2. Add a tag following the service:service_alias pattern. Any of the aliases registered for the service can be used. Tip: The tag in the Datadog monitor is case-insensitive.
3. Ready! Your monitor will be attached to the service automatically.
Note: If a tag was removed from the Datadog monitor, the link with the service will not be automatically removed! If you want to remove it, you have to do it manually.
First, navigate to the Operations tab of an OpsLevel service.
1. As shown in the screenshot below, Select Add Alert Sources
2. In the drop-down menu, find and select the correct Datadog monitor (you can type its name to search)
3. Click the Add Alert Sources button on the dialog box to save your selection(s)