Upload CTRF Reports for Analytics, Trends & Sharing

CTRF (Common Test Report Format) is a universal JSON schema for test results. It provides a standardized way to report test outcomes across any testing tool, framework, or language.

Why Use CTRF?

• Universal: Works with any test framework that has a CTRF reporter
• Consistent: Same format across Jest, Playwright, pytest, RSpec, and more
• Rich data: Includes timing, retries, flaky test detection, and metadata
• Open standard: Community-driven, MIT-licensed specification at ctrf.io

Supported Frameworks

CTRF has reporters for most popular test frameworks:

Framework	Package	Language
Playwright	playwright-ctrf-json-reporter	JavaScript/TypeScript
Jest	jest-ctrf-json-reporter	JavaScript/TypeScript
Vitest	vitest-ctrf-json-reporter	JavaScript/TypeScript
Cypress	cypress-ctrf-json-reporter	JavaScript/TypeScript
Mocha	mocha-ctrf-json-reporter	JavaScript/TypeScript
pytest	pytest-ctrf	Python
Go	ctrf-go-json-reporter	Go
JUnit	junit-json-reporter-ctrf	Java

See the full list at ctrf.io.

Installation

Playwright

npm install playwright-ctrf-json-reporter --save-dev

playwright.config.ts

import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [
    ['playwright-ctrf-json-reporter', { outputFile: 'ctrf-report.json' }],
    ['list'], // Also show in console
  ],
});

Jest

npm install jest-ctrf-json-reporter --save-dev

jest.config.js

module.exports = {
  reporters: [
    'default',
    ['jest-ctrf-json-reporter', { outputFile: 'ctrf-report.json' }],
  ],
};

Vitest

npm install vitest-ctrf-json-reporter --save-dev

vitest.config.ts

import { defineConfig } from 'vitest/config';

export default defineConfig({
  test: {
    reporters: ['default', 'vitest-ctrf-json-reporter'],
  },
});

pytest

pip install pytest-ctrf

pytest --ctrf ctrf-report.json

Uploading CTRF Reports

Once you have a CTRF JSON file, upload it to Gaffer:

curl -X POST https://app.gaffer.sh/api/upload \
  -H "X-API-Key: YOUR_UPLOAD_TOKEN" \
  -F "[email protected]" \
  -F 'tags={"commitSha":"abc123","branch":"main"}'

GitHub Actions

- name: Run tests
  run: npm test

- name: Upload CTRF report to Gaffer
  if: always()
  uses: gaffer-sh/gaffer-uploader@v1
  with:
    gaffer_api_key: ${{ secrets.GAFFER_UPLOAD_TOKEN }}
    report_path: ./ctrf-report.json
    commit_sha: ${{ github.sha }}
    branch: ${{ github.ref_name }}

GitLab CI

test:
  script:
    - npm ci
    - npm test
  after_script:
    - |
      curl -X POST https://app.gaffer.sh/api/upload \
        -H "X-API-Key: $GAFFER_UPLOAD_TOKEN" \
        -F "[email protected]" \
        -F 'tags={"commitSha":"'"$CI_COMMIT_SHA"'","branch":"'"$CI_COMMIT_REF_NAME"'"}'

CTRF Report Structure

A CTRF report contains standardized test result data:

{
  "results": {
    "tool": {
      "name": "playwright"
    },
    "summary": {
      "tests": 42,
      "passed": 40,
      "failed": 1,
      "pending": 0,
      "skipped": 1,
      "other": 0,
      "start": 1703520000000,
      "stop": 1703520060000
    },
    "tests": [
      {
        "name": "should login successfully",
        "status": "passed",
        "duration": 1234,
        "retries": 0,
        "flaky": false
      },
      {
        "name": "should display dashboard",
        "status": "failed",
        "duration": 5678,
        "message": "Element not found: #dashboard"
      }
    ]
  }
}

Troubleshooting

ENOENT: no such file or directory writing the CTRF report

ENOENT: no such file or directory, open 'ctrf/coverage/report.json'

The fix depends on which reporter you’re using, because the CTRF packages don’t share an option shape.

Jest (jest-ctrf-json-reporter) uses two separate options: outputDir (the directory) and outputFile (the filename only, default ctrf-report.json). The reporter calls mkdirSync(outputDir, { recursive: true }) for you, so a nested outputDir works as long as it’s actually passed via that option. The common mistake is collapsing both into outputFile:

jest.config.js

// ❌ Wrong: outputFile is filename-only; the path separators are not understood as a directory tree
['jest-ctrf-json-reporter', { outputFile: 'ctrf/coverage/report.json' }]

// ✅ Correct: outputDir for the directory (will be created), outputFile for the filename
['jest-ctrf-json-reporter', { outputDir: 'ctrf/coverage', outputFile: 'report.json' }]

Playwright (playwright-ctrf-json-reporter) takes a single outputFile that is a full path, and does not create the parent directory. Either ensure the directory exists in CI before the run, or write to a flat path:

# Option A: pre-create the directory in CI
- name: Ensure CTRF output directory exists
  run: mkdir -p ctrf/coverage

- name: Run tests
  run: npx playwright test

playwright.config.ts

// Option B: use a flat path
reporter: [
  ['playwright-ctrf-json-reporter', { outputFile: 'ctrf-report.json' }],
  ['list'],
],

If you’re using a different CTRF reporter, check its README for whether the directory is created automatically, or default to the pre-mkdir -p step.

tests array missing or empty

If the report is generated but Gaffer reports zero tests parsed, the reporter probably wasn’t invoked.

• Default reporter overridden. The reporters array in jest.config.js (or the equivalent in your framework) replaces the default. Make sure both 'default' and the CTRF reporter are listed, otherwise tests run but no CTRF file is written.
• Tests crashed before reporting. If the suite throws during setup (a beforeAll that fails to connect to a service, for example), some reporters bail before writing the file. Check the CI logs for the underlying error before assuming the reporter is broken.

Benefits with Gaffer

When you upload CTRF reports to Gaffer, you get:

• Structured analytics: Pass rates, duration trends, flaky test detection
• Cross-framework comparison: Compare results from different test suites
• Failure patterns: See which tests fail most frequently
• Historical tracking: See how tests perform over time

Tip

CTRF is particularly useful when you have multiple test frameworks in your project. Instead of learning different report formats, use CTRF everywhere for consistent analytics.

Next Steps

CI Provider Guides:

• GitHub Actions - Use the official Gaffer Action
• GitLab CI - GitLab pipeline integration
• CircleCI - CircleCI workflow integration
• Jenkins - Jenkins pipeline integration
• Bitbucket Pipelines - Bitbucket integration
• Azure DevOps - Azure Pipelines integration

Reference:

• Upload API - Full API documentation
• cURL Guide - Manual uploads and debugging

Get Started

Gaffer’s free tier includes 500 MB of storage with 7-day retention. Upload your first CTRF report in under 5 minutes.

Start Uploading CTRF Reports - Free