Quarantining flaky tests with Trunk

Quarantining isolates known flaky tests to prevent them from blocking CI jobs while continuing to run and track their results. The system identifies flaky tests at runtime and overrides their exit codes when they fail, allowing your CI pipeline to pass without requiring code changes to disable problematic tests. Why use quarantining: It acts as a crucial stopgap, minimizing the disruption from known flaky tests while your team works on fixing them. By quarantining flaky tests, you unblock critical CI pipelines—especially your merge queue—and regain development velocity without losing visibility, as these tests continue to run and upload results. This constant stream of data allows you to prioritize fixing the worst offenders based on their ongoing impact.

Broken tests are not quarantine candidates. Quarantining is designed for flaky tests — tests that intermittently fail and can be safely skipped to unblock CI while being investigated. A broken test represents a real failure that should not be hidden from CI results. Only tests with a Flaky status are eligible for auto-quarantine.

What does “Quarantined” mean?

A quarantined test continues running in CI and uploading results to Trunk Flaky Tests, but its failures won’t block your pipeline. The Trunk Analytics CLI checks with Trunk’s backend to determine if failed tests are quarantined, then overrides the exit code for those failures. When all failures in a CI job come from quarantined tests, the entire job passes. Why this matters: You maintain complete test coverage and historical data while preventing known problematic tests from disrupting your development cycle.

A CI job runs three tests where test_checkout fails. Without quarantine the job exits 1 and the PR is blocked; with the failing test wrapped in a quarantine box the job exits 0 and the PR merges.

How tests get quarantined

Tests can be quarantined through two methods:

Manual Quarantine - You explicitly select specific tests using override settings
Auto-Quarantine (when enabled) - Tests already flagged by Trunk’s flaky detection are automatically quarantined

Tests are auto-quarantined only if detected as flaky or flagged as flaky manually. Tests with a Broken status are not auto-quarantined — they represent real failures that should be investigated and fixed. For manually quarantined tests, all failures are quarantined regardless of test state.

Enable quarantining

Toggling the Enable Test Quarantining switch makes quarantining possible but does not quarantine any tests on its own.A test failure will only be ignored by CI if the test is already manually quarantined, or if the test has previously been identified as flaky and the Auto-Quarantine option is enabled.Actively quarantining tests will significantly change CI results, as failures from quarantined tests no longer cause builds to fail. Learn more about the effects of quarantining.

With quarantining enabled, the Analytics Uploader will compare failed test cases against known flaky tests. If a test is known to be flaky, it will be quarantined. If all failed tests are quarantined, the exit code of the test command will be overridden to return 0 and the CI job will pass.

Quarantining settings

To enable quarantining, navigate to Settings → Repositories → [repository] → Flaky Tests → toggle Enable Test Quarantining on.

Here’s what each of these options does:

Setting	Description
Enable Test Quarantining	This primary toggle activates the quarantining feature set, unlocking both manual override options and the ability to enable auto-quarantining. For any quarantining to work, the necessary configurations must also be made in your CI pipeline.
Auto-Quarantine Flaky Tests	When enabled, any test already identified by Trunk as “flaky” will be automatically quarantined. This saves you from having to manually quarantine each flaky test as it’s discovered.
Manual Quarantine Permissions	Controls who can set Always Quarantine or Never Quarantine overrides on individual tests. Set to Anyone to allow any user with repository access, or Admins only to restrict to organization admins. Disabled when Enable Test Quarantining is off. Only admins can change this setting.
Manual Test Status Override Permissions	Controls who can use Flag as Flaky to set a test’s status. Set to Anyone to allow any user with repository access, or Admins only to restrict to organization admins. Only admins can change this setting.
Summary Flaky Tests Reports	When enabled, Trunk posts pull request comments on each PR summarizing tests that passed, failed, flaked, were skipped, or were quarantined. Enabled by default.

Collection-level quarantining settings

Test Collections have their own quarantining settings that override the repository-level settings for any uploads routed to that collection. This lets you apply different quarantining policies to different subsets of your test suite. To configure collection quarantining, navigate to Flaky Tests → Collections → [collection name] → Settings → Quarantining. The same two toggles are available at the collection level:

Setting	Description
Enable Test Quarantining	Activates quarantining for this collection. When enabled, this setting overrides the repository-level quarantining setting for uploads that belong to this collection. New test collections have this enabled by default.
Auto-Quarantine Flaky Tests	Automatically quarantines any test in this collection that Trunk has identified as flaky. This option is only available when quarantining is enabled for the collection.

Only organization admins can change collection quarantining settings. Members can view the settings page but cannot toggle the controls.

New test collections have quarantining enabled by default. You can disable it from the collection’s Settings page at any time. Auto-Quarantine remains off by default and must be turned on separately.

When you disable collection quarantining, auto-quarantine is also disabled automatically. Re-enabling quarantining for the collection does not restore auto-quarantine — you must turn it back on separately.

Quarantining with Sharded or Parallelized Tests

There are two options for handling quarantining. Option 1: Wrapping each test invocation Wrap each command and specify its JUnit output path. Trunk captures the exit code and automatically uploads results. Example

# run test 1
./trunk-analytics-cli test --org-url-slug=[org] --token=[token] --junit-paths=test1_output/*.xml -- npm run test1
# run test 2
./trunk-analytics-cli test --org-url-slug=[org] --token=[token] --junit-paths=test2_output/*.xml -- npm run test2

Option 2: Handling quarantining during upload For complex setups where Trunk can’t wrap test commands, run tests first and let the upload step be the final gate. When quarantining is enabled, the upload inspects the provided JUnit results and decides whether to return exit code 0 or 1 based on the outcomes. Advanced: Handling build errors outside test runs To handle build issues that occur outside test runs, use the —test-process-exit-code option. This provides a fallback exit code if the upload runs without detecting any JUnit results. Example

./trunk-analytics-cli test --junit-paths "test_output.xml" \
   --org-url-slug <TRUNK_ORG_URL_SLUG> \
   --token $TRUNK_API_TOKEN \
   --junit-paths="**/results/*.xml" \
   --test-process-exit-code=1
   <YOUR_TEST_COMMAND>

The CLI only recognizes tests defined in JUnit. If multiple test executions occur and one fails due to a build error, Flaky Tests won’t detect it and will assume the exit code came from test failures. If those failures are quarantined, the job may incorrectly be reported as successful. To prevent this:

Upload results for each test execution separately, or
Generate a JUnit that records build errors.

Updates in CI

If you’re using the provided GitHub Actions workflow to upload test results to Flaky Tests, you can quarantine flaky tests by wrapping the test command or as a follow-up step. If you’re using the Trunk Analytics CLI directly or other CI providers, check the instructions in the Using The Trunk Analytics CLI Directly tab.

GitHub Actions Workflow
Using The Trunk Analytics CLI Directly

Using the Trunk Analytics Uploader Action in your GitHub Actions Workflow files may need modifications to your workflow files to support quarantining.If you upload your test results as a second step after you run your tests, you need to add continue-on-error: true on your test step so your CI job will continue even on failures.Here’s an example file.

	name: Run Tests And Upload Results
	on:
	  workflow_dispatch:
	jobs:
	  upload-test-results:
	    runs-on: ubuntu-latest
	    timeout-minutes: 60
	    steps:
	    - name: Run Tests
	      id: unit_tests
	      shell: bash
	      run: <COMMAND TO RUN TESTS>
	      continue-on-error: true
	
	    - name: Upload test results
	      if: always()
	      uses: trunk-io/analytics-uploader@v2
	      with:
	        junit-paths: <TEST OUTPUT PATH>
	        org-slug: my-trunk-org-slug
	        token: ${{ secrets.TRUNK_API_TOKEN }}

If you want to run the test command and upload in a single step, the test command must be run via the Analytics Uploader through the run: <COMMAND TO RUN TESTS> parameter.This will override the response code of the test command. Make sure to set continue-on-error: false so un-quarantined tests are blocking.

name: Run Tests And Upload Results
on:
  workflow_dispatch:
jobs:
  upload-test-results:
    runs-on: ubuntu-latest
    timeout-minutes: 60
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Run tests and upload results
        uses: trunk-io/analytics-uploader@v2
        with:
          junit-paths: <TEST OUTPUT PATH>
          run: <COMMAND TO RUN TESTS>
          token: ${{ secrets.TRUNK_API_TOKEN }}
          org-slug: my-trunk-org-slug

Using Flaky Tests as a separate stepIf you upload your test results as a second step after you run your tests, you need to ensure your test step continues on errors so the upload step that’s run after can quarantine failed tests. When quarantining is enabled, the trunk-analytics-cli upload command will return an error if there are unquarantined failures and return a status code 0 if all tests are quarantined.

<run my tests> || true # doesn't fail job on failure
|
    ./trunk-analytics-cli upload \
        --org-url-slug $TRUNK_ORG_URL_SLUG \
        --token $TRUNK_API_TOKEN \
        --junit-paths $JUNIT_PATH

Using Flaky Tests as a single stepYou can also wrap the test command with the Trunk Analytics CLI. When wrapping the command with the Trunk Analytics CLI, if there are unquarantined tests, the command will return an error. If there are no unquarantined tests, the command will return a status code 0.

./trunk-analytics-cli test \
    --org-url-slug <TRUNK_ORG_URL_SLUG> \
    --token $TRUNK_API_TOKEN \
    --junit-paths $JUNIT_PATH \
    --allow-empty-test-results \
    <Test Command>

Overriding individual tests

If you have tests that should never be quarantined or should always be quarantined regardless of their current health status, you can do this by overriding individual tests.

quarantine override on the test details page

You can set a quarantine override from two places: a test’s details page, or the Flaky Tests table. From the test details page Open the test’s details page and click the Quarantine button to reveal the Choose quarantine status control. Select Always to quarantine the test’s failures or Never to keep it from being quarantined, add the required comment, and click Save. The control also shows whether repository auto-quarantining is on. To clear an override later, reopen the control and click Remove Quarantine. From the Flaky Tests table Open the actions menu (the ⋮ button at the end of any row in the Flaky Tests table). Alongside Copy link and Create ticket, two quarantine actions are available:

Quarantine test / Unquarantine test: toggles the always-quarantine override for that test.
Never Quarantine test / Remove Never Quarantine: toggles the never-quarantine override. When set, the test is never quarantined, even if auto-quarantining is enabled for the repo.

Access to these options is controlled by the Manual Quarantine Permissions setting. When set to Admins only, non-admin users see the quarantine controls disabled with the tooltip “Only admins can set manual quarantine.” When a manual override is active, a banner shows who set it and when.

Setting	Behavior
Always Quarantine	Quarantine a test failure even if the health status is healthy.
Never Quarantine	Never quarantine failures, even if the health status is flaky, and auto-quarantining is enabled for the repo.

To review a history of all quarantine changes on a test, check the Events tab on the test details page. The Events tab shows every override, setting change, and comment, along with the author and timestamp for each entry. To see all quarantined runs of a test, set the Quarantined filter to Only on the Test History tab.

Tracking quarantined jobs in the dashboard

Once quarantining is active, the Quarantining tab provides a central hub for monitoring its impact and effectiveness. This tab serves as a complete audit log of every CI job saved by the feature, allowing you to:

Visualize Trends: A 30-day chart shows the number of jobs quarantined per day.
Inspect Individual Jobs: A detailed table lists every quarantined job. Click any entry to see the specific tests that were quarantined.
Isolate Critical Workflows: Use the filter to see how quarantining impacts specific branches, such as preventing flaky failures in your Merge Queue.
Measure ROI: Use the data to quantify the number of builds saved and developer time reclaimed for your organization.

Audit logs

Trunk provides audit logs for all setting changes and overwrites for individual tests. You can access the audit log by navigating to Settings → Repositories → [repository] → Flaky Tests → Audit logs under the Enable Test Quarantining heading.

Quarantining API and webhooks

For advanced use cases, you can interact with quarantining features programmatically.

API: Use the Flaky Tests API to fetch a list of all currently quarantined tests in your project.
Webhooks: Subscribe to the test_case.quarantining_setting_changed event to trigger automated workflows whenever a test’s quarantine override is modified. Learn more about Webhooks.

Service Availability and Graceful Degradation

Trunk Analytics CLI is designed to fail safe when our quarantine service is unavailable. Read more at Quarantine Service Availability

​What does “Quarantined” mean?

​How tests get quarantined

​Enable quarantining

​Quarantining settings

​Collection-level quarantining settings

​Quarantining with Sharded or Parallelized Tests

​Updates in CI

​Overriding individual tests

​Tracking quarantined jobs in the dashboard

​Audit logs

​Quarantining API and webhooks

​Service Availability and Graceful Degradation

What does “Quarantined” mean?

How tests get quarantined

Enable quarantining

Quarantining settings

Collection-level quarantining settings

Quarantining with Sharded or Parallelized Tests

Updates in CI

Overriding individual tests

Tracking quarantined jobs in the dashboard

Audit logs

Quarantining API and webhooks

Service Availability and Graceful Degradation