Your services have dashboards, alerts, and SLOs in Datadog. Your test suite doesn’t. Here’s how to get test metrics into Datadog without writing custom instrumentation.
The Problem
You run hundreds of tests in CI every day, but the results live in CI logs. To answer “is our test suite healthy?” you check individual pipeline runs. There’s no trend line, no alert when pass rates drop, and no way to correlate test health with deployment metrics.
What Teams Usually Try
1. Custom metric scripts in CI
- Parse test output, POST to Datadog’s API in an
after_scriptblock - Breaks when test runner output format changes
- Every team reinvents this differently
2. Datadog CI Visibility
- Full-featured but requires Datadog’s test runner instrumentation
- Per-test pricing can add up at scale
- Tightly coupled to Datadog — hard to switch later
3. Manual monitoring
- Someone checks CI periodically
- Problems are discovered after they’ve compounded
A Simpler Approach
Gaffer already stores your test results and coverage reports. With OpenTelemetry export, it pushes metrics to Datadog automatically — pass rates, failure counts, test totals, and coverage percentages arrive as standard OTLP metrics.
No test runner plugins. No CI script changes beyond what you already have for Gaffer uploads.
Setup
If you’re already uploading test results to Gaffer, adding Datadog export takes a few steps:
- Go to Settings > OpenTelemetry in the Gaffer dashboard
- Click “Add destination” and select Datadog
- Paste your Datadog API key and select your site region
- Click “Test” to verify connectivity
For full setup instructions including Grafana Cloud and generic OTLP endpoints, see the OpenTelemetry integration guide.
Dashboard Template
We’ve published a Datadog dashboard template you can import directly. It includes:
- Overview gauges — current pass rate, total tests, failed count, skipped count
- Pass rate over time — line chart of
gaffer.tests.pass_rateover 30 days, grouped by branch - Test counts by result — stacked bar chart of passed/failed/skipped per day
- Coverage trends — line and branch coverage percentages over time
Import the Dashboard
- Download the dashboard template
- In Datadog, go to Dashboards > New Dashboard
- Click the gear icon and select Import Dashboard JSON
- Paste or upload the JSON file
- Use the
$projecttemplate variable to filter by Gaffer project
Available Metrics
| Metric | Type | Description |
|---|---|---|
gaffer.tests.pass_rate | Gauge (%) | Pass rate of the test run |
gaffer.tests.total | Gauge | Total test count |
gaffer.tests.passed | Gauge | Passed test count |
gaffer.tests.failed | Gauge | Failed test count |
gaffer.tests.skipped | Gauge | Skipped test count |
gaffer.coverage.line_percent | Gauge (%) | Line coverage percentage |
gaffer.coverage.branch_percent | Gauge (%) | Branch coverage percentage |
All metrics include gaffer.project.name, gaffer.branch, and gaffer.commit_sha as attributes for filtering.
What You Get
- Alerts — set a Datadog monitor on
gaffer.tests.pass_ratedropping below a threshold - Correlation — overlay test health with deploy events, error rates, and service metrics
- Trend visibility — spot gradual test suite degradation before it becomes a problem
- Team dashboards — share test health alongside service dashboards that your team already checks daily