Gaffer

Gaffer

OpenTelemetry

Export test run and coverage metrics to Datadog, Grafana Cloud, or any OTLP-compatible backend.

🚧

Beta Feature

OpenTelemetry metrics export is currently in beta. Please share feedback as you try it out.

Gaffer can export test run and coverage metrics as OpenTelemetry (OTLP) data to your existing observability stack. Track pass rates, failure counts, and coverage trends alongside your service metrics in Datadog, Grafana Cloud, or any OTLP-compatible backend.

Features

  • Provider presets — built-in support for Datadog and Grafana Cloud, plus a generic OTLP option for any compatible endpoint
  • Branch filtering — limit metric export to specific branches using glob patterns
  • Test run metrics — pass rate, total/passed/failed/skipped counts per run
  • Coverage metrics — line and branch coverage percentages per upload
  • Test connectivity — verify your configuration from the dashboard before waiting for real data
  • Encrypted API keys — credentials are encrypted at rest with AES-256-GCM

Setup by Provider

Datadog

  1. In Datadog, go to Organization Settings > API Keys and create (or copy) an API key
  2. In Gaffer, navigate to Settings > OpenTelemetry and click "Add destination"
  3. Select Datadog as the provider
  4. Paste your API key and select your Datadog site (e.g., datadoghq.com, datadoghq.eu, us3.datadoghq.com, us5.datadoghq.com, ap1.datadoghq.com)
  5. Choose which event types to export (test runs, coverage, or both)
  6. Click "Create"
  7. Click "Test" to verify connectivity
  8. Verify in Datadog: go to Metrics > Explorer and search for gaffer.tests.pass_rate

Grafana Cloud

  1. In Grafana Cloud, go to your stack's OpenTelemetry section to get your OTLP gateway credentials (instance ID, region, API token)
  2. In Gaffer, navigate to Settings > OpenTelemetry and click "Add destination"
  3. Select Grafana as the provider
  4. Enter your Grafana Cloud region, instance ID, and API token
  5. Choose event types and click "Create"
  6. Click "Test" to verify connectivity
  7. Verify in Grafana: go to Explore and query for gaffer_tests_pass_rate

Generic OTLP

Any backend that accepts OTLP over HTTP can be used as a destination.

  1. In Gaffer, navigate to Settings > OpenTelemetry and click "Add destination"
  2. Select Generic OTLP as the provider
  3. Enter your OTLP HTTP endpoint URL (must be HTTPS)
  4. Enter an API key or bearer token for authentication
  5. Choose event types and click "Create"
  6. Click "Test" to verify connectivity

Configuration Options

Event Types

Choose which events export metrics:

  • Test runs — exports pass rate, total/passed/failed/skipped counts when a test run completes
  • Coverage — exports line and branch coverage percentages when a coverage report is uploaded

At least one event type must be enabled.

Branch Filters

Restrict which branches trigger metric export using glob patterns:

  • main — exact match
  • release/* — matches release/v1.0, release/2024-01, etc.
  • feature-* — matches feature-auth, feature-dashboard, etc.

Leave branch filters empty to export metrics for all branches.

Enabled Toggle

Each destination can be enabled or disabled independently. Disabled destinations remain configured but do not export metrics.

Available Metrics

Test Run Metrics

Metric Type Unit Description
gaffer.tests.pass_rate Gauge % Pass rate of the test run
gaffer.tests.total Gauge {test} Total test count
gaffer.tests.passed Gauge {test} Passed test count
gaffer.tests.failed Gauge {test} Failed test count
gaffer.tests.skipped Gauge {test} Skipped test count

Coverage Metrics

Metric Type Unit Description
gaffer.coverage.line_percent Gauge % Line coverage percentage
gaffer.coverage.branch_percent Gauge % Branch coverage percentage

Dimensions (Attributes)

Metrics include these attributes for filtering and grouping:

Attribute Included On Description
gaffer.project.id All metrics Project identifier
gaffer.project.name All metrics Project name
gaffer.branch All metrics Git branch name (when available)
gaffer.commit_sha All metrics Git commit SHA (when available)
gaffer.report_format Test run metrics only Report format (e.g., ctrf)

Dashboard Templates

Get started faster with a pre-built Datadog dashboard. See Datadog Test Metrics Dashboard for a downloadable JSON template and import instructions.

Testing

Each destination has a "Test" button in the dashboard. Clicking it sends a test metric payload to your configured endpoint and reports the HTTP status code. Use this to verify connectivity and authentication before waiting for real test data.

Troubleshooting

Test button returns an error?

  • Check your API key — make sure the key is correct and has the necessary permissions
  • Verify the site/region — Datadog and Grafana Cloud require the correct region to be selected
  • Check endpoint URL — for generic OTLP, ensure the URL is correct and reachable
  • Network issues — if your backend is behind a firewall, ensure it accepts connections from Gaffer's servers

Metrics not appearing in your backend?

  • Check the destination is enabled — disabled destinations do not export metrics
  • Verify branch filters — if configured, only matching branches trigger export
  • Check event types — make sure the relevant event type (test runs or coverage) is enabled
  • Check "Last Sent" — the destinations list shows the last delivery time and any errors
  • Allow propagation time — some backends take a few minutes to index new metrics

Security

  • Encrypted API keys — credentials are encrypted at rest using AES-256-GCM
  • HTTPS recommended — Datadog and Grafana Cloud presets use HTTPS endpoints; for generic OTLP, use HTTPS to protect data in transit